Family Options Study

3-Year Impacts of Housing

and Services Interventions for

Homeless Families

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

Family Options Study

3-Year Impacts of Housing

and Services Interventions for

Homeless Families

Prepared for

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Prepared by

Daniel Gubits

Marybeth Shinn

Michelle Wood

Stephen Bell

Samuel Dastrup

Claudia D. Solari

Scott R. Brown

Debi McInnis

Tom McCall

Utsav Kattel

Project quality advisors

Jill Khadduri

Jacob Klerman

Abt Associates, Inc.

In partnership with

Vanderbilt University

October 2016

FAMILY OPTIONS STUDY

ACKNOWLEDGMENTS

The authors of this report gratefully acknowledge (Abt Associates) led the site recruitment and implementation the efforts of many individuals who have assisted in

activities, with contributions from site research liaisons Burt,

carrying out the Family Options Study since it began

Culhane, Shinn, Donald Chamberlin (independent consul-

in 2008. They particularly thank the Government Technical

tant), Wendy Vaulton (formerly with the Center for Social

Representative (GTR), Anne Fletcher, for her unwavering

Innovation), Matt White (Abt Associates), and Suzanne Zerger

guidance and support throughout all phases of the research.

(formerly with the Center for Social Innovation). Satyendra Pa-

Other staff members at the U.S. Department of Housing and

trabansh (formerly with Abt Associates) designed the complex

Urban Development (HUD), Office of Policy Development and

random assignment algorithm used to assign families, and Sage

Research (PD&R) have provided valuable guidance and tech-

Computing developed the randomization software. Scott Brown

nical oversight for the study. In particular, the authors thank

(Vanderbilt University), Lauren Dunton, Nichole Fiore, and

Paul Dornan, the study’s former GTR, for his substantial input

Meghan Henry (Abt Associates) monitored random assignment,

into the study’s design and specification of study interventions.

collecting updates about the availability of intervention slots

HUD’s Office of Special Needs Assistance Programs has been

and the status of families enrolled in each intervention.

an active partner in the design and implementation of the

The Family Options Study involved an extensive field

Family Options Study, and the study team is grateful for that

data-collection effort to collect information directly from study

office’s guidance and support in developing the study design

participants at several junctures. Abt SRBI led the participant

and implementation plan. In addition, the authors thank Lynn

data-collection activities under the direction of Ricki Jarmon

Rodgers of PD&R for her assistance with acquiring HUD Public

and Brenda Rodriquez, the study’s survey director. Ashley

and Indian Housing Information Center (PIC) and Tenant

Bradbury and Brianna Roche (Abt SRBI) and Debi McInnis (Abt

Rental Assistance Certification System (TRACS) data.

Associates) also played key roles in coordinating and monitor-

The authors also give special thanks to the families participat-

ing data-collection activities. Kathy Gill and Lynn Reneau (Abt

ing in the Family Options Study, who have continued to share

SRBI) served as survey field managers and oversaw the 17 field

their experiences and open their lives to the study team.

interviewers working in the study sites.

The Family Options Study has benefited from the contributions

Samuel Dastrup (Abt Associates) led the cost data-collection

of a number of technical experts. The study design reflects

efforts, assisted by Scott Brown, Burt, Dunton, Fiore, Katherine

guidance from experts in random assignment methodology and

Buck (Abt Associates), and Galen Savidge (Abt Associates).

from subject matter experts knowledgeable about and close

Dunton and Claudia Solari (Abt Associates) managed the

to the operational realities of the homeless assistance services

collection and analysis of Homeless Management Information

delivery system. The authors are especially grateful to Ellen

System (HMIS) data from the 12 study sites, and Steven Brown

Bassuk (Center for Social Innovation), Martha Burt (indepen-

(formerly with Abt Associates) coordinated analysis of HUD

dent consultant), Dennis Culhane (University of Pennsylvania),

administrative data from PIC and TRACS.

Larry Orr (Johns Hopkins University), and Beth Weitzman

Under the direction of Gubits, Scott Brown (Vanderbilt

(New York University) for their help in designing the study

University), Tom McCall, Utsav Kattel, Nancy McGarry, and

interventions and the research approach.

Don Laliberty (Abt Associates) played key roles in estimating

Stephen Bell (Abt Associates) and Marybeth Shinn (Vanderbilt

impacts using survey and administrative data.

University) are the study’s Co-Principal Investigators, Michelle

The study’s Co-Project Quality Advisors, Jill Khadduri and

Wood (Abt Associates) is the Project Director, and Daniel

Jacob Klerman (Abt Associates), provided technical guidance

Gubits (Abt Associates) is the study’s Director of Analysis.

throughout all phases of the project and thoughtful and con-

Implementing the study design required the efforts of a large

structive comments on this report. Missy Robinson (Abt Associ-

team to recruit sites, develop site-specific implementation

ates) did an excellent job producing this complex document.

plans, and conduct random assignment. Brooke Spellman

3-Year Impacts of Housing and Services Interventions for Homeless Families

ii

FAMILY OPTIONS STUDY

Acknowledgments

The study would not be possible without the enthusiasm and

Minnesota; (11) Phoenix, Arizona; and (12) Salt Lake City,

dedication of local service providers, Continuum of Care (CoC)

Utah. The study team is grateful for their many contributions

leaders, and public housing agencies in the 12 participating

and continued commitment to the study. The study team

communities: (1) Alameda County, California; (2) Atlanta,

also thanks the HMIS administrators in each of the CoCs for

Georgia; (3) Baltimore, Maryland; (4) Boston, Massachusetts;

providing HMIS data for participating families.

(5) the New Haven and Bridgeport regions of Connecticut;

A grant to Vanderbilt University by the Eunice Kennedy Shriver

(6) Denver, Colorado; (7) Honolulu, Hawaii; (8) Kansas

National Institute of Child Health and Human Development

City, Missouri; (9) Louisville, Kentucky; (10) Minneapolis,

supported child data collection and analysis for this study.

DISCLAIMER

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S.

Department of Housing and Urban Development or the U.S. government.

3-Year Impacts of Housing and Services Interventions for Homeless Families

iii

FAMILY OPTIONS STUDY

FOREWORD

The 2010 release of Opening Doors: Federal Strategic conveying significantly improved housing outcomes to Plan To Prevent and End Homelessness established

formerly homeless families. We also continue to see the

an ambitious goal to end homelessness among

radiating benefits to families of stable housing that were

children, families, and youth by 2020. Until now, the U.S.

observed in the short-term outcomes, such as reductions in

Department of Housing and Urban Development (HUD) has

adult psychological distress, experiences of intimate partner

had little empirical evidence to guide its policy and program

violence, school mobility among children, food insecurity,

decisions toward achieving this goal.

and sleep and behavior problems in children. Offering a

homeless family a voucher cost roughly $4,000 more than

This report is the culmination of an 8-year research effort.

leaving families to find their own way out of shelter over

Launched in 2008, HUD’s Family Options Study represents,

the full 3-year followup period—a modest investment to

to date, the most rigorously designed experimental study

achieve substantially better outcomes for both parents and

to determine the effectiveness and relative costs of different

children. Among the crisis interventions studied- including

interventions that communities may implement to assist

rapid re-housing, transitional housing, and emergency

families experiencing homelessness—long term housing

shelter—housing stability and other outcomes for families

subsidy, project-based transitional housing, community-

were relatively similar, but, as it was in the short-term study

based rapid re-housing, and usual care. More than 2,200

findings, rapid re-housing was the least costly intervention

homeless families, including more than 5,000 children, in

for communities to implement over the full 3-year followup

12 communities were randomly assigned to one of these

period.

four interventions. The families were tracked for 3 years and

were extensively interviewed at baseline, 20 months after

The study findings, in both the short and long terms, pro-

random assignment, and again at 37 months after random

vide clear evidence that, when long-term housing assistance

assignment to assess outcomes related to housing stability,

is offered to families experiencing homelessness, families are

family preservation, adult well-being, child well-being, and

able to both obtain and sustain stable housing and that this

self-sufficiency.

housing assistance can have a powerful impact on improv-

ing the lives of poor families and children.

HUD released the short-term outcomes report in July 2015,

documenting the outcomes of families 20 months after

random assignment and presenting striking evidence of the

power of offering a long-term subsidy, such as a voucher,

to a homeless family in domains that extended far beyond

housing stability. This current report, which documents

Katherine M. O’Regan

outcomes of families 37 months after random assignment,

Assistant Secretary for Policy Development & Research

amplifies the remarkable short-term findings, demonstrating

U.S. Department of Housing and Urban Development

once again the power of a long-term housing subsidy in

3-Year Impacts of Housing and Services Interventions for Homeless Families

iv

FAMILY OPTIONS STUDY

TABLE OF CONTENTS

List of Abbreviations and Acronyms ................................. xv

Chapter 2. Description of Usual Care (UC) and

Outcomes Measured in the Study. ............................... 9

Executive Summary ......................................................... xvi

2.1. The Emergency Shelter Experi ence of Usual

Study Interventions .................................................. xvi

Care (UC) Families ............................................... 9

Random Assignment Design .................................... xvii

2.1.1. Length of Time UC Families Spent in

Meaning of Intention-to-Treat (ITT) Impact

Emergency Shelters .................................. 9

Estimates ......................................................... xviii

2.2. Use of Other Homeless and Housing Assis-

Data Sources ............................................................. xix

tance Programs by Usual Care (UC) Families ...... 11

Hypothesized Effects of the Interventions .................. xx

2.3. Outcomes for Families Randomly Assigned to

Conceptual Framework for the SUB and CBRR

Usual Care (UC) ................................................. 13

Interventions ............................................... xx

2.3.1. Measures of Housing Stability ................ 13

Hypotheses for Comparisons Involving the

2.3.2. Housing Stability of the UC Group ....... 14

SUB and CBRR Interventions and Usual

2.3.3. Measures of Family Preservation ............ 16

Care............................................................. xx

2.3.4. Family Preservation in the UC Group .... 17

Conceptual Framework for the PBTH

2.3.5. Measures of Adult Well-Being ................ 18

Intervention ................................................ xxi

2.3.6. Adult Well-Being in the UC Group ........ 18

Hypotheses for Comparisons Involving the

PBTH Intervention...................................... xxi

2.3.7. Measures of Child Well-Being ............... 19

Baseline Characteristics of the Study Sample .......... xxii

2.3.8. Characteristics of Focal Children in

the UC Group ........................................ 21

Intervention Eligibility Screening and Family

Decisions ......................................................... xxii

2.3.9. Characteristics of Focal Children by

Age-Specific Outcomes in the UC Group .. 23

Study Findings Over the 3-Year Followup Period .... xxii

2.3.10. Measures of Self-Sufficiency .................. 25

Usual Care Group During the Followup

Period ...................................................... xxiii

2.3.11. Self-Sufficiency Indicators for the UC

Group .................................................... 27

Program Use for Pairwise Comparisons ........... xxiv

2.4. Summary ............................................................ 29

Impact Estimates for Pairwise Comparisons .... xxvii

Do Certain Interventions Work Better When

Chapter 3. Impacts of Permanent Housing Subsidy

Offered to Families Facing Greater

(SUB) Compared With Usual Care (UC) .................... 30

Difficulties?................................................ xxx

3.1. Permanent Housing Subsidy (SUB) Intervention .. 30

Intervention Costs ............................................ xxx

3.2. Program Use by Families in the Permanent

Conclusions.......................................................... xxxiii

Housing Subsidy (SUB) Versus Usual Care

(UC) Comparison ............................................... 31

Chapter 1. Introduction...................................................... 1

3.3. Impacts on Housing Stability in the Permanent

1.1. The Research Sample ........................................... 1

Housing Subsidy (SUB) Versus Usual Care

1.2. Interventions Studied and Rationales ................... 3

(UC) Comparison ............................................... 35

1.3. Evaluation Design ................................................. 4

3.4. Impacts on Family Preservation in the

1.4. Findings From the Enrollment Phase .................... 6

Permanent Housing Subsidy (SUB) Versus

Usual Care (UC) Comparison ............................. 39

1.5. Short-Term Findings (20 Months After Enrollment) .6

3.5. Impacts on Adult Well-Being in the Permanent

1.6. Questions for the 3-Year Analysis ........................ 7

Housing Subsidy (SUB) Versus Usual Care

1.7. Organization of the Report ................................... 8

(UC) Comparison ............................................... 40

3-Year Impacts of Housing and Services Interventions for Homeless Families

v

FAMILY OPTIONS STUDY

Table of Contents

3.6. Impacts on Child Well-Being in the Permanent

5.4. Impacts on Family Preservation in the Project-

Housing Subsidy (SUB) Versus Usual Care

Based Transitional Housing (PBTH) Versus

(UC) Comparison ............................................... 40

Usual Care (UC) Comparison ............................. 66

3.7. Impacts on Self-Sufficiency in the Permanent

5.5. Impacts on Adult Well-Being in the Project-

Housing Subsidy (SUB) Versus Usual Care

Based Transitional Housing (PBTH) Versus

(UC) Comparison ............................................... 41

Usual Care (UC) Comparison ............................. 68

3.8. Summary of the Permanent Housing Subsidy

5.6. Impacts on Child Well-Being in the Project-

(SUB) Versus Usual Care (UC) Comparison

Based Transitional Housing (PBTH) Versus

Across Domains .................................................. 45

Usual Care (UC) Comparison ............................. 69

5.7. Impacts on Self-Sufficiency in the Project-

Chapter 4. Impacts of Community-Based Rapid Re-

Based Transitional Housing (PBTH) Versus

Housing (CBRR) Compared With Usual Care (UC). .. 47

Usual Care (UC) Comparison ............................. 69

4.1. Community-Based Rapid Re-Housing (CBRR)

5.8. Summary of Project-Based Transitional

Intervention ........................................................ 47

Housing (PBTH) Versus Usual Care (UC)

4.2. Program Use by Families in the Community-

Comparison Across Domains .............................. 72

Based Rapid Re-Housing (CBRR) Versus Usual

Care (UC) Comparison ....................................... 48

Chapter 6. Impacts of Permanent Housing Subsidy

4.3. Impacts on Housing Stability in the Community-

(SUB) Compared With Community-Based Rapid

Based Rapid Re-Housing (CBRR) Versus Usual

Re-Housing (CBRR), SUB Compared With Project-

Care (UC) Comparison ....................................... 52

Based Transitional Housing (PBTH), and CBRR

Compared With PBTH .............................................. 73

4.4. Impacts on Family Preservation in the

Community- Based Rapid Re-Housing (CBRR)

6.1. Analysis Samples for Pairwise Comparisons........ 73

Versus Usual Care (UC) Comparison .................. 54

6.2. The Permanent Housing Subsidy (SUB) Versus

4.5. Impacts on Adult Well-Being in the Community-

Community-Based Rapid Re-Housing (CBRR)

Based Rapid Re-Housing (CBRR) Versus Usual

Comparison ........................................................ 74

Care (UC) Comparison ....................................... 55

6.2.1. Program Use by Families in the SUB-

4.6. Impacts on Child Well-Being in the Community-

Versus-CBRR Comparison ........................ 74

Based Rapid Re-Housing (CBRR) Versus Usual

6.2.2. Impacts of the SUB Intervention

Care (UC) Comparison ....................................... 56

Compared With the CBRR Intervention ..... 75

4.7. Impacts on Self-Sufficiency in the Community-

6.3. The Permanent Housing Subsidy (SUB) Versus

Based Rapid Re-Housing (CBRR) Versus Usual

Project-Based Transitional Housing (PBTH)

Care (UC) Comparison ....................................... 56

Comparison ........................................................ 82

4.8. Summary of the Community-Based Rapid

6.3.1. Program Use by Families in the SUB-

Re-Housing (CBRR) Versus Usual Care (UC)

Versus-PBTH Comparison ........................ 82

Comparison Across Domains .............................. 59

6.3.2. Impacts of the SUB Intervention

Compared With the PBTH Intervention ..... 83

Chapter 5. Impacts of Project-Based Transitional

Housing (PBTH) Compared With Usual Care (UC). .. 60

6.4. The Community-Based Rapid Re-Housing

(CBRR) Versus Project-Based Transitional

5.1. Project-Based Transitional Housing (PBTH)

Housing (PBTH) Comparison ............................. 90

Intervention ........................................................ 60

6.4.1. Program Use by Families in the CBRR-

5.2. Program Use by Families in the Project-Based

Versus-PBTH Comparison ........................ 91

Transitional Housing (PBTH) Versus Usual

Care (UC) Comparison ....................................... 61

6.4.2. Impacts of the CBRR Intervention

Compared With the PBTH Intervention ..... 92

5.3. Impacts on Housing Stability in the Project-

Based Transitional Housing (PBTH) Versus

Chapter 7. Impacts of Pooled Comparisons ...................... 98

Usual Care (UC) Comparison ............................. 65

3-Year Impacts of Housing and Services Interventions for Homeless Families

vi

FAMILY OPTIONS STUDY

Table of Contents

Chapter 8. Do Certain Interventions Work Better When

10.8. Implications for Theory .................................. 125

Offered to Families Who Face Greater Difficulties? .... 99

10.9. Summary of Findings and Implications for

8.1. Descriptive Results for Psychosocial Challenges

Policy ............................................................. 125

and Housing Barriers ........................................ 100

Chapter 11. References ................................................... 127

8.2. Differential Impacts Depending on Psychoso-

cial Challenges .................................................. 101

Chapter 12. Additional reading ...................................... 130

8.2.1. SUB Versus UC ....................................... 101

8.2.2. CBRR Versus UC .................................... 101

Appendix A. Data Sources and Data Set Construction ......A1

8.2.3. PBTH Versus UC .................................... 101

A.1. Random Assignment Data ..................................A2

8.2.4. SUB Versus CBRR ................................... 101

Family Options Study Sample ............................A2

8.2.5. SUB Versus PBTH ................................... 103

Eligibility Determination ....................................A3

8.2.6. CBRR Versus PBTH ................................. 103

A.2. Baseline Data Collection .....................................A3

8.3. Differential Impacts Depending on Housing

A.3. Enrollment Verification Data ..............................A4

Barriers ............................................................. 104

A.4. 6-, 12-, and 27-Month Tracking Surveys............A4

8.3.1. SUB Versus UC ....................................... 104

A.5. 20- and 37-Month Followup Adult Surveys ......A4

8.3.2. CBRR Versus UC .................................... 104

A.6. Focal Child Selection .........................................A5

8.3.3. PBTH Versus UC .................................... 104

Focal Child Selection ..........................................A5

8.3.4. SUB Versus CBRR ................................... 104

Focal Children Sample Sizes ...............................A6

8.3.5. SUB Versus PBTH ................................... 104

Parent-on-Child Module in the 37-Month

8.3.6. CBRR Versus PBTH ................................. 106

Adult Survey................................................A9

8.4. Summary .......................................................... 106

A.7. 20- and 37-Month Followup Child Surveys .......A9

A.8. 20- and 37-Month Followup Child

Chapter 9. Intervention Costs......................................... 107

Assessments .................................................... A9

9.1. Introduction and Review of Methodology ......... 107

A.9. 20- and 37-Month Followup HOME

9.1.1. Cost Data-Collection and Analysis

Inventories ..................................................... A9

Methodology .......................................... 109

A.10. Qualitative Surveys ........................................A10

9.2. Per-Family Monthly Costs ................................ 111

A.11. Additional Details About Surveys and Data

9.3. Cost of All Program Use During the Followup

Collection .........................................................A10

Period by Families Assigned to Each

Implementation of 37-Month Data Collection ....A10

Intervention ...................................................... 113

Efforts To Improve Response Rates ...................A11

9.4. Monthly Cost of All Program Use at the Time

Incentives .........................................................A11

of the Followup Survey by Families Assigned

to Each Intervention ......................................... 115

Completion Rates .............................................A11

A.12. Administrative Data .......................................A13

Chapter 10. Conclusions ................................................ 118

HMIS Data ........................................................A13

10.1. Questions Addressed in the 3-Year Analysis.... 118

PIC and TRACS Data Files ................................A14

10.2. Meaning of Intention-to-Treat Impact

NDNH Data ......................................................A15

Estimates .......................................................... 119

Child Welfare Agency Administrative Data .......A16

10.3. Usual Care (UC) ............................................. 121

A.13. Program Usage Data File ................................A16

10.4. Permanent Housing Subsidy (SUB) ................ 122

10.5. Community-Based Rapid Re-housing (CBRR) . 123

Appendix B. Construction of Outcomes ...........................B1

10.6. Project-Based Transitional Housing (PBTH) .... 124

B.1. Measures of Housing Stability ............................B1

10.7. Impacts by Level of Family Challenge............. 125

B.2. Measures of Family Preservation.........................B2

3-Year Impacts of Housing and Services Interventions for Homeless Families

vii

FAMILY OPTIONS STUDY

Table of Contents

B.3. Measures of Adult Well-Being ............................B3

Appendix D. Analysis of 37-Month Survey Nonresponse ... D1

B.4. Measures of Child Well-Being ............................B6

D.1. Introduction ..................................................... D1

Child Outcomes Across All Age Groups .............B6

Balance in Impact Comparison Groups After

Behavioral Strengths and Challenges ..................B6

Nonresponse .............................................. D1

Child Outcomes for Specific Age Groups ............B8

Respondents Versus Nonrespondents ............... D1

B.5. Measures of Self-Sufficiency ............................B12

Overview of Findings ........................................ D1

B.6. Measures of Earnings and Employment From

D.2. Balance in Impact Comparison Groups After

NDNH Data ......................................................B15

Nonresponse ..................................................... D2

Data Cleaning and Processing ...........................B15

Survey Nonresponse .......................................... D2

Defining Outcomes ...........................................B15

Equivalence at Baseline of Analysis Sample by

Impact Comparison .................................... D2

B.7. Measures of Formal Child Separations From

Child Welfare Agency Records .........................B16

D.3. Respondents Versus Nonrespondents ............. D14

B.8. Program Use Outcomes ....................................B16

Do Respondents Differ From Nonrespondents

on Baseline Characteristics? ...................... D14

Appendix C. Analysis Methods ........................................ C1

Unweighted Impact Estimates ......................... D29

C.1. Methodology ..................................................... C1

Appendix E. Impacts on Use of Transitional Housing .......E1

Pooled Comparisons ......................................... C1

Impact Estimation Model for Family and Adult

Appendix F. Impact Estimates for Pooled Comparisons ...F1

Outcomes ................................................... C2

Appendix G. Intervention Costs—Methodology, Sites,

Impact Estimation Model for Child Well-Being

and Programs ........................................................... G1

Outcomes ................................................... C3

G.1. Programs in the Cost Study .............................. G1

Impact Estimation Model for Moderator

Analysis ...................................................... C3

G.2. Elements of Program-Level per-Family Monthly

Program Cost .................................................... G2

C.2. Covariates ........................................................ C3

In-Kind and Partner Costs ................................ G3

Continuous Variables ........................................ C4

Capital Costs .................................................... G4

Binary Variables ................................................ C4

Participant Contributions ................................. G4

Binary Variables for Child Outcomes Only ....... C5

G.3. Costs of SUB Intervention ................................. G5

Additional Covariates for Pooled Comparisons

and Selected Outcomes .............................. C5

G.4. Calculating Costs .............................................. G5

C.3. Missing Data and Imputations ........................... C5

Per-Family Monthly Program Cost .................... G6

C.4. Family/Adult Weights ....................................... C5

Cost of All Program Use During the Followup

Period ......................................................... G6

C.5. Child Weights ................................................... C6

Cost of All Program Use at the Time of the

C.6. Partial Paired t-Test for Differences in Means

Followup Survey ........................................ G6

Between Followup Waves .................................. C7

C.7. Multiple Comparisons Adjustment for Confir-

Appendix H. Supplemental Analyses ............................... H1

matory Hypothesis Tests .................................... C7

Statement of the Problem .................................. C7

Appendix I. Length of Baseline Stay in Emergency Shelter ... I1

Study Response to the Problem ......................... C8

Implementing the Multiple Comparisons

Procedure ................................................... C8

3-Year Impacts of Housing and Services Interventions for Homeless Families

viii

FAMILY OPTIONS STUDY

Table of Contents

List of Exhibits

Exhibit ES-1. Six Pairwise Comparisons Among the Four

Exhibit 2-9. Adult Well-Being Outcomes in the UC Group ..19

Interventions ................................................. xvii

Exhibit 2-10. Child Well-Being Outcomes in the UC Group

Exhibit ES-2. Total Number of Families Assigned to Each

Measured for Children Across Age Groups ......21

Intervention and Number of Followup

Exhibit 2-11. Child Well-Being Developmental Outcomes

Survey Respondents .................................... xviii

in the UC Group for Children in Specific

Exhibit ES-3. Data Sources Used in the Analysis of 3-Year

Age Groups .....................................................24

Impacts .......................................................... xix

Exhibit 2-12. Self-Sufficiency Outcomes in the UC Group ....28

Exhibit ES-4. Program Use of UC Families for 32 Months

Exhibit 2-13. Work for Pay at 20 and 37 Months After RA

After RA ....................................................... xxiii

in the UC Group .............................................29

Exhibit ES-5. SUB, CBRR, and PBTH: Program Use for

Exhibit 2-14. Earnings and Employment in the UC Group ...29

32 Months After RA ...................................... xxiv

Exhibit 3-1. Subsidy Type Provided by Site ........................31

Exhibit ES-6. Program Use Since RA for All Pairwise

Impact Comparisons .................................... xxvi

Exhibit 3-2. SUB Versus UC: Program Use Since RA ...........32

Exhibit ES-7. Summary of Impacts for Six Policy

Exhibit 3-3. Number of Months of Subsidy Receipt

Comparisons .............................................. xxviii

During Followup Period by SUB Families

Who Ever Used Offered Permanent Subsidy ....33

Exhibit ES-8. Cost of Program Use Since RA for Each

Intervention Contrast .................................. xxxii

Exhibit 3-4. SUB Versus UC: Program Use in Each

Month of Followup Period ..............................34

Exhibit 1-1. Location of Study Sites ......................................2

Exhibit 3-5. SUB Versus UC: Impacts on Housing

Exhibit 1-2. Family Characteristics .......................................3

Stability at 37 Months .....................................36

Exhibit 1-3. Six Pairwise Comparisons Among the Four

Exhibit 3-6. SUB Versus UC: Percent of Families With

Interventions .....................................................5

Any Stay in Emergency Shelter and Any

Exhibit 2-1. Length of Time Spent in Emergency Shelters

Permanent Housing Subsidy During Month,

by UC Families ................................................10

by Month After RA ..........................................38

Exhibit 2-2. Percent of UC Families With at Least

Exhibit 3-7. SUB Versus UC: Impacts on Family Preser-

1-Night Stay in Emergency Shelter During

vation at 37 Months ........................................39

Month, by Number of Months After RA...........10

Exhibit 3-8. SUB Versus UC: Impacts on Child Welfare

Exhibit 2-3. Program Use Since RA for UC Families ............11

Outcomes ........................................................39

Exhibit 2-4. Program Use of UC Families for 32 Months

Exhibit 3-9. SUB Versus UC: Impacts on Adult Well-

After RA ..........................................................12

Being at 37 Months .........................................40

Exhibit 2-5. Housing Stability Outcomes for UC Families ..15

Exhibit 3-10. SUB Versus UC: Impacts on Child Well-

Being Across Age Groups at 37 Months ...........41

Exhibit 2-6. Homeless, Doubled Up, and Living in Own

Place in the UC Group at 20 and 37 Months

Exhibit 3-11. SUB Versus UC: Impacts on Child Well-

After RA ..........................................................16

Being Developmental Outcomes by Age

Group at 37 Months ........................................42

Exhibit 2-7. Family Preservation Outcomes in the UC

Group..............................................................17

Exhibit 3-12. SUB Versus UC: Impacts on Self-Sufficiency

at 37 Months ...................................................43

Exhibit 2-8. Child Welfare Outcomes in the UC Group ......18

Exhibit 3-13. SUB Versus UC: Earnings and Employment ....44

3-Year Impacts of Housing and Services Interventions for Homeless Families

ix

FAMILY OPTIONS STUDY

Table of Contents

Exhibit 4-1. CBRR Versus UC: Program Use Since RA ........49

Exhibit 5-8. PBTH Versus UC: Impacts on Adult

Well-Being at 37 Months .................................68

Exhibit 4-2. Number of Months of Rapid Re-Housing

Receipt During Followup Period by CBRR

Exhibit 5-9. PBTH Versus UC: Impacts on Child

Families Who Ever Used Rapid Re-housing .....50

Well-Being Across Age Groups at 37 Months ..69

Exhibit 4-3. CBRR Versus UC: Program Use in Each

Exhibit 5-10. PBTH Versus UC: Impacts on Child

Month of Followup Period ..............................51

Well-Being Developmental Outcomes by

Age Group at 37 Months .................................70

Exhibit 4-4. CBRR Versus UC: Impacts on Housing

Stability at 37 Months .....................................52

Exhibit 5-11. PBTH Versus UC: Impacts on Self-Sufficiency

at 37 Months ...................................................71

Exhibit 4-5. CBRR Versus UC: Percent of Families With

Any Stay in Emergency Shelter and Any

Exhibit 5-12. PBTH Versus UC: Earnings and Employment ..71

Permanent Housing Subsidy During Month,

Exhibit 6-1. Sample Sizes in the Six Pairwise Comparisons...74

by Number of Months After RA .......................53

Exhibit 6-2. SUB Versus CBRR: Program Use Since RA .......75

Exhibit 4-6. CBRR Versus UC: Impacts on Family

Preservation at 37 Months ...............................54

Exhibit 6-3. SUB Versus CBRR: Impacts on Housing

Stability at 37 Months .....................................76

Exhibit 4-7. CBRR Versus UC: Impacts on Child Welfare

Outcomes ........................................................54

Exhibit 6-4. SUB Versus CBRR: Impacts on Family

Preservation at 37 Months ...............................77

Exhibit 4-8. CBRR Versus UC: Impacts on Adult

Well-Being at 37 Months .................................55

Exhibit 6-5. SUB Versus CBRR: Impacts on Child

Welfare Outcomes ...........................................77

Exhibit 4-9. CBRR Versus UC: Impacts on Child

Well-Being Across Age Groups at 37 Months ..56

Exhibit 6-6. SUB Versus CBRR: Impacts on Adult

Well-Being at 37 Months .................................78

Exhibit 4-10. CBRR Versus UC: Impacts on Child

Well-Being Developmental Outcomes by

Exhibit 6-7. SUB Versus CBRR: Impacts on Child

Age Group at 37 Months .................................57

Well-Being Across Age Groups at 37 Months ..79

Exhibit 4-11. CBRR Versus UC: Impacts on Self-Sufficiency

Exhibit 6-8. SUB Versus CBRR: Impacts on Child

at 37 Months ...................................................58

Well-Being Developmental Outcomes by

Age Group at 37 months .................................80

Exhibit 4-12. CBRR Versus UC: Earnings and Employment ..58

Exhibit 6-9. SUB Versus CBRR: Impacts on Self-

Exhibit 5-1. PBTH Versus UC: Program Use Since RA ........62

Sufficiency at 37 Months .................................81

Exhibit 5-2. Number of Months of Transitional Housing

Exhibit 6-10. SUB Versus CBRR: Earnings and Employment ..81

Receipt During Followup Period by PBTH

Families Who Ever Used TH ...........................63

Exhibit 6-11. SUB Versus PBTH: Program Use Since RA .......83

Exhibit 5-3. PBTH Versus UC: Program Use in Each

Exhibit 6-12. SUB Versus PBTH: Impacts on Housing

Month of Followup Period ..............................64

Stability at 37 Months .....................................84

Exhibit 5-4. PBTH Versus UC: Impacts on Housing

Exhibit 6-13. SUB Versus PBTH: Impacts on Family

Stability at 37 Months .....................................66

Preservation at 37 Months ...............................85

Exhibit 5-5. PBTH Versus UC: Percent of Families With

Exhibit 6-14. SUB Versus PBTH: Impacts on Child

Any Stay in Emergency Shelter and Any

Welfare Outcomes ...........................................86

Permanent Housing Subsidy During Month,

by Number of Months after RA ........................67

Exhibit 6-15. SUB Versus PBTH: Impacts on Adult

Well-Being 37 Months .....................................86

Exhibit 5-6. PBTH Versus UC: Impacts on Family

Preservation at 37 Months ...............................67

Exhibit 6-16. SUB Versus PBTH: Impacts on Child

Well-Being Across Age Groups at 37 Months ..87

Exhibit 5-7. PBTH Versus UC: Impacts on Child Welfare

Outcomes ........................................................68

3-Year Impacts of Housing and Services Interventions for Homeless Families

x

FAMILY OPTIONS STUDY

Table of Contents

Exhibit 6-17. SUB Versus PBTH: Impacts on Child

Exhibit 9-3. Summary Statistics of Per-Family Monthly

Well-Being Developmental Outcomes by

Program Cost by Program Type .....................112

Age Group at 37 Months .................................88

Exhibit 9-4. Comparison of Cost Summary Statistics

Exhibit 6-18. SUB Versus PBTH: Impacts on Self-

Across Program Types ...................................113

Sufficiency at 37 Months .................................89

Exhibit 9-5. Cost of Program Use Since RA for Each

Exhibit 6-19. SUB Versus PBTH: Earnings and Employment ..89

Intervention Contrast ....................................114

Exhibit 6-20. CBRR Versus PBTH: Program Use Since RA.....91

Exhibit 9-6. Average Per-Family Monthly Costs for

Exhibit 6-21. CBRR Versus PBTH: Impacts on Housing

Program Use at Time of the Followup

Stability at 37 Months .....................................92

Survey, by Comparison .................................116

Exhibit 6-22. CBRR Versus PBTH: Impacts on Family

Exhibit 10-1. Summary of Statistically Significant Impact

Preservation at 37 Months ...............................93

Results by Policy Comparison: Executive

Summary Outcomes 37 Months After RA ......120

Exhibit 6-23. CBRR Versus PBTH: Impacts on Child

Welfare Outcomes ...........................................93

Exhibit A-1. Data Sources Used in the Study .......................A1

Exhibit 6-24. CBRR Versus PBTH: Impacts on Adult

Exhibit A-2. Sample Enrollment Period and Number of

Well-Being at 37 Months .................................94

Families Enrolled by Intervention and Site ......A3

Exhibit 6-25. CBRR Versus PBTH: Impacts on Child

Exhibit A-3. Length of Time From Random Assignment

Well-Being Across Age Groups at 37 Months ..95

to the 37-Month Followup Survey ...................A5

Exhibit 6-26. CBRR Versus PBTH: Impacts on Child

Exhibit A-4. Focal Child Sample Distribution, by Site and

Well-Being Developmental Outcomes by

Intervention Group at 37 Months ....................A7

Age Group at 37 Months .................................96

Exhibit A-5. Overall Family Options Study Survey

Exhibit 6-27. CBRR Versus PBTH: Impacts on Self-

Completion Rates ..........................................A12

Sufficiency at 37 Months .................................97

Exhibit A-6. Survey Response Status for Family Options

Exhibit 6-28. CBRR Versus PBTH: Earnings and

Study Baseline and Followup Surveys ............A12

Employment ....................................................97

Exhibit A-7. Total Number of Families Assigned to Each

Exhibit 7-1. Sample Sizes in the Four Pooled Comparisons ..98

Intervention and Number of Followup

Exhibit 8-1. Percentage of Adult Respondents Reporting

Survey Respondents ......................................A12

Psychosocial Challenges at the Time of

Exhibit A-8. Survey Response Status for Family Options

Study Enrollment (for families interviewed

Study Baseline, Followup, and Tracking

at 37 months) ................................................100

Surveys ..........................................................A13

Exhibit 8-2. Percentage of Adult Respondents Reporting

Exhibit A-9. HMIS Participation Rates for Emergency

That a Condition Was a Big or Small

Shelter and Transitional Housing Providers

Problem in Finding a Place To Live at the

in the Study Sites, 2011 .................................A14

Time of Study Enrollment (for families

interviewed at 37 months) .............................100

Exhibit A-10. HMIS Match Rates With the Family Options

Sample by Site Through 37 Months ...............A14

Exhibit 8-3. Impacts at 37 Months Moderated by

Baseline Psychosocial Challenges Index .........102

Exhibit A-11. Sample Families in PIC/TRACS Data and

Those Assigned to the SUB Intervention in

Exhibit 8-4. Impacts at 37 Months Moderated by

PIC/TRACS Data by Site Through 37 Months ...A15

Baseline Housing Barriers Index ....................105

Exhibit A-12. Program Types and Their Data Sources in

Exhibit 9-1. Programs Included in the Estimates of

the Program Usage Data ................................A17

Monthly Costs ...............................................109

Exhibit A-13. Data Source Reliability by Program Use Data

Exhibit 9-2. Average Per-Family Monthly Cost of

Item...............................................................A17

Suppor tive Services and Housing or Shelter

Across Program Types ...................................111

3-Year Impacts of Housing and Services Interventions for Homeless Families

xi

FAMILY OPTIONS STUDY

Table of Contents

Exhibit B-1. Children’s Completion Rates at 37 Months

Exhibit D-15. Summary of Baseline Equivalence Testing

for WJ III Letter-Word Identification and

of 37-Month Respondents Versus Non-

Applied Problems Tests ...................................B9

respondents, by Impact Comparison ............ D14

Exhibit C-1. Confirmatory Hypothesis Tests in 37-Month

Exhibit D-16. Equivalence at Baseline of Analysis Sample

Analysis .......................................................... C9

for Respondents Versus Nonrespondents for

SUB, 37-Month Adult Survey ....................... D15

Exhibit D-1. Survey Nonresponse Incidence by Impact

Comparison—37-Month Adult Survey ........... D2

Exhibit D-17. Equivalence at Baseline of Analysis Sample

for Respondents Versus Nonrespondents for

Exhibit D-2. Characteristics Examined in Baseline

CBRR, 37-Month Adult Survey ..................... D16

Equivalency Testing ....................................... D3

Exhibit D-18. Equivalence at Baseline of Analysis Sample

Exhibit D-3. Summary of Equivalence Testing in Impact

for Respondents Versus Nonrespondents for

Comparisons, 37-Month Adult Survey ........... D3

PBTH, 37-Month Adult Survey ..................... D17

Exhibit D-4. Equivalence at Baseline of Analysis Sample

for SUB Versus UC Impact Comparison,

Exhibit D-19. Equivalence at Baseline of Analysis Sample

37-Month Adult Survey .................................. D4

for Respondents Versus Nonrespondents for

UC, 37-Month Adult Survey ......................... D18

Exhibit D-5. Equivalence at Baseline of Analysis Sample

for CBRR Versus UC Impact Comparison,

Exhibit D-20. Equivalence at Baseline of Analysis Sample

37-Month Adult Survey .................................. D5

for Respondents Versus Nonrespondents

for SUB Versus UC Impact Comparison,

Exhibit D-6. Equivalence at Baseline of Analysis Sample

37-Month Adult Survey ................................ D19

for PBTH Versus UC Impact Comparison,

37-Month Adult Survey .................................. D6

Exhibit D-21. Equivalence at Baseline of Analysis Sample

for Respondents Versus Nonrespondents

Exhibit D-7. Equivalence at Baseline of Analysis Sample

for CBRR Versus UC Impact Comparison,

for SUB Versus CBRR Impact Comparison,

37-Month Adult Survey ................................ D20

37-Month Adult Survey .................................. D7

Exhibit D-22. Equivalence at Baseline of Analysis Sample

Exhibit D-8. Equivalence at Baseline of Analysis Sample

for Respondents Versus Nonrespondents

for SUB Versus PBTH Impact Comparison,

for PBTH Versus UC Impact Comparison,

37-Month Adult Survey .................................. D8

37-Month Adult Survey ................................ D21

Exhibit D-9. Equivalence at Baseline of Analysis Sample

Exhibit D-23. Equivalence at Baseline of Analysis Sample

for CBRR Versus PBTH Impact Comparison,

for Respondents Versus Nonrespondents

37-Month Adult Survey .................................. D9

for SUB Versus CBRR Impact Comparison,

Exhibit D-10. Equivalence at Baseline of Analysis Sample

37-Month Adult Survey ................................ D22

for SUB + CBRR + PBTH Versus UC Impact

Exhibit D-24. Equivalence at Baseline of Analysis Sample

Comparison, 37-Month Adult Survey ........... D10

for Respondents Versus Nonrespondents

Exhibit D-11. Equivalence at Baseline of Analysis Sample

for SUB Versus PBTH Impact Comparison,

for SUB + PBTH Versus CBRR Impact

37-Month Adult Survey ................................ D23

Comparison, 37-Month Adult Survey ........... D11

Exhibit D-25. Equivalence at Baseline of Analysis Sample

Exhibit D-12. Equivalence at Baseline of Analysis Sample

for Respondents Versus Nonrespondents for

for SUB + CBRR Versus PBTH Impact

CBRR Versus PBTH Impact Comparison,

Comparison, 37-Month Adult Survey ........... D12

37-Month Adult Survey ................................ D24

Exhibit D-13. Equivalence at Baseline of Analysis Sample

Exhibit D-26. Equivalence at Baseline of Analysis Sample

for CBRR + PBTH Versus SUB Impact

for Respondents Versus Nonrespondents

Comparison, 37-Month Adult Survey ........... D13

for SUB + CBRR + PBTH Versus UC Impact

Comparison, 37-Month Adult Survey ........... D25

Exhibit D-14. Summary of Equivalence Testing of

37-Month Respondents Versus Non-

respondents, by Assigned Intervention ......... D14

3-Year Impacts of Housing and Services Interventions for Homeless Families

xii

FAMILY OPTIONS STUDY

Table of Contents

Exhibit D-27. Equivalence at Baseline of Analysis Sample

Exhibit F-7. SUB+CBRR Versus PBTH: Impacts on

for Respondents Versus Nonrespondents

Housing Stability at 37 Months .......................F6

for SUB + PBTH Versus CBRR Impact

Exhibit F-8. SUB+CBRR Versus PBTH: Impacts on

Comparison, 37-Month Adult Survey ........... D26

Family Preservation at 37 Months ...................F6

Exhibit D-28. Equivalence at Baseline of Analysis Sample

Exhibit F-9. SUB+CBRR Versus PBTH: Impacts on Adult

for Respondents Versus Nonrespondents

Well-Being at 37 Months .................................F7

for SUB + CBRR Versus PBTH Impact

Comparison, 37-Month Adult Survey ........... D27

Exhibit F-10. SUB+CBRR Versus PBTH: Impacts on Child

Well-Being Across Age Groups at 37 Months ..F7

Exhibit D-29. Equivalence at Baseline of Analysis Sample

for Respondents Versus Nonrespondents

Exhibit F-11. SUB+CBRR Versus PBTH: Impacts on Child

for PBTH + CBRR Versus SUB Impact

Well-Being Developmental Outcomes by

Comparison, 37-Month Adult Survey ........... D28

Age Group at 37 Months .................................F8

Exhibit D-30. Executive Summary Impact Estimates,

Exhibit F-12. SUB+CBRR Versus PBTH: Impacts on

Estimated Without Nonresponse Weights .... D29

Self-Sufficiency at 37 Months ..........................F9

Exhibit E-1. SUB Versus UC: Impacts on Use of ES and

Exhibit F-13. SUB+PBTH Versus CBRR: Impacts on

TH in Months 0 to 32 After RA .......................E2

Housing Stability at 37 Months .....................F10

Exhibit E-2. CBRR Versus UC: Impacts on Use of ES and

Exhibit F-14. SUB+PBTH Versus CBRR: Impacts on

TH in Months 0 to 32 After RA .......................E2

Family Preservation at 37 Months .................F10

Exhibit E-3. PBTH Versus UC: Impacts on Use of ES and

Exhibit F-15. SUB+PBTH Versus CBRR: Impacts on Adult

TH in Months 0 to 32 After RA .......................E3

Well-Being at 37 Months ...............................F11

Exhibit E-4. SUB Versus CBRR: Impacts on Use of ES

Exhibit F-16. SUB+PBTH Versus CBRR: Impacts on Child

and TH in Months 0 to 32 After RA .................E3

Well-Being Across Age Groups at 37 Months ...F11

Exhibit E-5. SUB Versus PBTH: Impacts on Use of ES

Exhibit F-17. SUB+PBTH Versus CBRR: Impacts on Child

and TH in Months 0 to 32 After RA .................E4

Well-Being Developmental Outcomes by

Age Group at 37 Months ...............................F12

Exhibit E-6. CBRR Versus PBTH: Impacts on Use of ES

and TH in Months 0 to 32 After RA .................E4

Exhibit F-18. SUB+PBTH Versus CBRR: Impacts on

Self-Sufficiency at 37 Months ........................F13

Exhibit F-1. SUB+CBRR+PBTH Versus UC: Impacts on

Housing Stability at 37 Months .......................F2

Exhibit F-19. CBRR+PBTH Versus SUB: Impacts on

Housing Stability at 37 Months .....................F14

Exhibit F-2. SUB+CBRR+PBTH Versus UC: Impacts on

Family Preservation at 37 Months ...................F2

Exhibit F-20. CBRR+PBTH Versus SUB: Impacts on

Family Preservation at 37 Months .................F14

Exhibit F-3. SUB+CBRR+PBTH Versus UC: Impacts on

Adult Well-Being at 37 Months .......................F3

Exhibit F-21. CBRR+PBTH Versus SUB: Impacts on Adult

Well-Being at 37 Monthss .............................F15

Exhibit F-4. SUB+CBRR+PBTH Versus UC: Impacts on

Child Well-Being Across Age Groups at 37

Exhibit F-22. CBRR+PBTH Versus SUB: Impacts on Child

Months ............................................................F3

Well-Being Across Age Groups at 37 Months ...F15

Exhibit F-5. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-23. CBRR+PBTH Versus SUB: Impacts on Child

Child Well-Being Developmental Outcomes

Well-Being Developmental Outcomes by

by Age Group at 37 Months ............................F4

Age Group at 37 Months ...............................F16

Exhibit F-6. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-24. CBRR+PBTH Versus SUB: Impacts on

Self-Sufficiency at 37 Months ..........................F5

Self-Sufficiency at 37 Months ........................F17

3-Year Impacts of Housing and Services Interventions for Homeless Families

xiii

FAMILY OPTIONS STUDY

Table of Contents

Exhibit G-1. Number of Programs From Which Cost

Exhibit H-4. SUB Versus CBRR: Program Use From 19th

Data Were Collected and Presence of SUB

Month After RA Until 37-Month Survey

Intervention by Study Site .............................. G2

Response Month ............................................. H3

Exhibit G-2. Cost Data Collection Categories and

Exhibit H-5. SUB Versus PBTH: Program Use From 19th

Associated Item Prompts ................................ G2

Month After RA Until 37-Month Survey

Response Month ............................................. H3

Exhibit G-3. Determining Inclusion of External Services .... G3

Exhibit H-6. CBRR Versus PBTH: Program Use From

Exhibit G-4. Valuing In-Kind Services ................................ G4

19th Month After RA Until 37-Month

Exhibit G-5. Program Level Average per Family Cost

Survey Response Month ................................. H4

Calculations.................................................... G5

Exhibit H-7. Housing Stability Outcomes for the SUB

Exhibit H-1. SUB Versus UC: Program Use From 19th

RA Group by Use of the Offered Permanent

Month After RA Until 37-Month Survey

Subsidy .......................................................... H4

Response Month ............................................. H1

Exhibit H-8. Housing Stability Outcomes for the CBRR

Exhibit H-2. CBRR Versus UC: Program Use From 19th

RA Group by Use of Rapid Re-Housing .......... H5

Month After RA Until 37-Month Survey

Exhibit H-9. Housing Stability Outcomes for the PBTH

Response Month ............................................. H2

RA Group by Use of Transitional Housing ...... H5

Exhibit H-3. PBTH Versus UC: Program Use From 19th

Exhibit I-1. Impacts on Length of Baseline Stay in

Month After RA Until 37-Month Survey

Emergency Shelter ............................................I1

Response Month ............................................. H2

3-Year Impacts of Housing and Services Interventions for Homeless Families

xiv

FAMILY OPTIONS STUDY

LIST OF ABBREVIATIONS AND ACRONYMS

AFCARS Adoption and Foster Care Analysis and Reporting

PBTH

priority access to project-based transitional housing

System

PHA

public housing agency

CBRR

priority access to community-based rapid re-housing

PIC

Public and Indian Housing Information Center

CoC

Continuum of Care (program); continuum of care

PTSD

post-traumatic stress disorder

(generic)

RA

random assignment

FMR

Fair Market Rent

SHP

Supportive Housing Program

HAP

Housing Assistance Payment

SNAP

Supplemental Nutrition Assistance Program

HCV

Housing Choice Voucher program

SUB

priority access to permanent housing subsidy

HMIS

Homeless Management Information System

TANF

Temporary Assistance for Needy Families

HPRP

Homelessness Prevention and Rapid Re-Housing

TRACS Tenant Rental Assistance Certification System

Program

UC

usual care

HTKS

Head, Toes, Knees, Shoulders

WIC

Special Supplemental Nutrition Program for

ITT

intention-to-treat

Women, Infants, and Children

3-Year Impacts of Housing and Services Interventions for Homeless Families

xv

FAMILY OPTIONS STUDY

EXECUTIVE SUMMARY

The U.S. Department of Housing and Urban Develop- and Services Interventions for Homeless Families; hereafter, the ment undertook the Family Options Study to gather

Short-Term Impacts report; Gubits et al., 2015); however, 20

evidence about which types of housing and services

months is not a long enough period to evaluate the effects of

programs work best for homeless families. The study examines

temporary programs that can last up to 18 months (as with CBRR

the effects of three types of programs—permanent housing sub-

programs) or 24 months (as with PBTH programs). Some

sidies, community-based rapid re-housing, and project-based

families may not have received a full dose of the temporary

transitional housing—compared with one another and with

assistance by the time of the short-term analysis at 20 months,

the usual care available to homeless families. The three types of

so different findings may emerge during a longer period. In

programs are distinguished from one another by the duration

addition, some families were still in these temporary programs

of housing assistance and the type and intensity of social

at 20 months, so they may have different outcomes after leaving.

services provided to families. Usual care consists of emergency

This report presents the analysis of the 3-year impacts of the

shelter and housing or services that families can access without

three interventions in five domains related to family well-being:

immediate referral to a program that would provide them with

(1) housing stability, (2) family preservation, (3) adult well-

a place to live.

being, (4) child well-being, and (5) self-sufficiency. The report

From September 2010 through January 2012, 2,282 families

also describes the relative costs of the interventions based on

enrolled in the Family Options Study across 12 communities1

program use during the 3-year followup period.

after spending at least 7 days in emergency shelter. After

The 3-year analysis addresses three primary questions.

providing informed consent and completing a baseline survey,

the families were randomly assigned to one of four groups:

1. What programs do families who experience homelessness

(1) SUB, in which families have priority access to a permanent

use during a 3-year period, and how does assignment to an

housing subsidy; (2) CBRR, in which families have priority

intervention that offers priority access to a particular kind of

access to community-based rapid re-housing; (3) PBTH, in

program affect this program use?

which families have priority access to project-based transitional

housing; or (4) UC, in which families have access to usual

2. At 3 years after random assignment, what are the relative

care homeless and housing assistance but do not have priority

effects of the three active interventions compared with usual

access to any particular program. Random assignment yielded

care and of the active interventions compared with each other?

groups of families with no systematic differences in baseline

3. What are the cumulative costs of the interventions during

characteristics. Families were free to take up the program

the 3 years following random assignment?

to which they were offered priority access or to make other

arrangements, so each group used a mix of programs. None-

theless, patterns of program use among the groups of families

Study Interventions

contrasted substantially. The study therefore provides a strong

The study examines four interventions, to which families are

basis for conclusions about the relative impacts of the interven-

randomly assigned:

tions on several aspects of family well-being.2

1. The SUB intervention offers families a permanent housing

At 20 months after random assignment, the Family Options

subsidy, usually a housing choice voucher, which could include

Study produced important and, in some cases, surprising find-

assistance to find housing but no other supportive services.

ings (see Family Options Study: Short-Term Impacts of Housing

1 The 12 communities participating in the study are Alameda County, California; Atlanta, Georgia; Baltimore, Maryland; Boston, Massachusetts; Bridgeport and New Haven, Connecticut; Denver, Colorado; Honolulu, Hawaii; Kansas City, Missouri; Louisville, Kentucky; Minneapolis, Minnesota; Phoenix, Arizona; and Salt Lake City, Utah.

2 Two previous reports provide information about the Family Options Study: the Interim Report: Family Options Study (Gubits et al., 2013), which documented study implementation findings and baseline characteristics of the research sample, and the Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015), which presents findings from the 20-month impact analysis.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xvi

FAMILY OPTIONS STUDY

Executive Summary

2. The CBRR intervention offers families temporary rental

Random Assignment Design

assistance, potentially renewable for up to 18 months, paired

with limited, housing-focused services to help families find

To be eligible for the study, families had to include at least one

and rent conventional, private-market housing.

child age 15 or younger and had to have resided in emergency

shelter for 7 or more days. The study team excluded families

3. The PBTH intervention offers families temporary housing

who left shelter in fewer than 7 days because the three inter-

for up to 24 months in agency-controlled buildings or

ventions examined may not be necessary for families who can

apartment units, paired with intensive supportive services.

resolve a housing crisis quickly. As soon as was feasible after

4. The UC intervention did not offer priority access to any

the 7-day mark, the study team randomly assigned families to

type of homeless or housing assistance. Families assigned

the SUB, CBRR, PBTH, or UC group.

to the UC intervention could use any housing or services

In the original random assignment design, each family was to

in the community that a family could access in the absence

have had a chance of being assigned to all four groups (SUB,

of immediate referral to the other interventions. The UC

CBRR, PBTH, and UC). A number of factors prevented the

intervention typically included at least some additional stay

study from being implemented exactly as planned. First, all

in the emergency shelter from which families were enrolled.

four interventions were offered in only nine sites. Two sites

The study team analyzed all six possible contrasts among these

(Atlanta and Baltimore) did not offer the SUB intervention and

four interventions, as shown in Exhibit ES-1. The order of

one site (Boston) did not offer the PBTH intervention. Second,

the presentation of findings for the pairwise comparisons is

interventions were available to families only in cases in which

reflected in the alphabetic ordering of the arrows (for example,

at least one provider of the intervention type had an available

discussion begins with “Contrast A” between the SUB interven-

slot. Third, some service providers had unique eligibility

tion and the UC intervention).

requirements for families. Before random assignment, the study

team screened families for eligibility for the providers that had

Exhibit ES-1. Six Pairwise Comparisons Among the Four Interventions

3-Year Impacts of Housing and Services Interventions for Homeless Families

xvii

FAMILY OPTIONS STUDY

Executive Summary

available slots using criteria specified by the providers. The

process will be similar to each other in all respects except

purpose of this screening was to minimize the likelihood of

for their priority access to a particular type of homeless or

assigning families to interventions they would not be eligible to

housing assistance program. This assurance means that that

receive. As a result, for an intervention option to be available

subsequent differences in outcomes (beyond the bounds of

to a family undergoing random assignment, at least one slot

chance sampling variability) reflect the relative impact of those

had to be open at an intervention provider for which the family

interventions.

met provider-specific eligibility requirements. All families were

The Family Options Study tests for the impacts of three different

eligible for usual care by definition.

potential emphases in federal or local assistance policy to home less

These factors cumulatively led to situations in which most

families: (1) What impact would priority access to project-based

study families did not have all four options available to them

transitional housing (offered to the PBTH group) have on

at the time of random assignment. Of the 2,282 families

families in shelter who are not able to resolve their episodes of

enrolled in the study, 264 families had two randomization

homelessness quickly? (2) How does this policy compare with

options, 1,544 families had three randomization options, and

providing access to community-based rapid re-housing (offered

474 had all four randomization options available. All analyses

to the CBRR group)? (3) How does it compare to permanent

were conducted pairwise, contrasting an active intervention

housing subsidies (offered to the SUB group)? In each case,

with another active intervention or with the usual care. Only

the corresponding policy question is, “What impact would this

families who were eligible for both interventions in a pairwise

policy emphasis have on the outcomes of families in shelter

comparison (for example, the SUB and CBRR interventions)

relative to usual care or another policy emphasis?”

and were randomized to one of them were included in each

The 3-year followup data for study participants tell what would

comparison. Hence, each comparison can be thought of as an

happen if each of these ways of targeting offers and access were

experiment between two well-matched groups that differ only

pursued as federal or local policy—for the families actually

in the intervention to which they were assigned.

studied in the target communities. The pairwise comparisons

Exhibit ES-2 shows the total number of families assigned to

between active interventions show the impact of offering fami-

each intervention. The exhibit also shows the number of fami-

lies priority access to one type of program rather than another.

lies who responded to the followup survey conducted a median

The data also allow for the comparison of each option with the

of 37 months after random assignment; this set of families is

mix of programs that the homeless assistance systems provided

included in the impact analyses in this report.3 Altogether,

at the time of the study (that is, the programs available to the

1,621 families (71 percent of the sample) completed both the

UC group). The pairwise comparisons between active interven-

20- and 37-month surveys.4

tions and usual care show the impact of referring a family to a

specific type of program compared with the impact of letting

Meaning of Intention-to-Treat (ITT)

families pursue any available assistance on their own.

Impact Estimates

The analysis in this report measures the impact of having been

offered a particular type of program regardless of whether the

The inherent strength of the experimental research design

family involved actually received the program assistance. The

employed in the Family Options Study is the assurance that

findings reflect the real way in which the homeless assistance

the groups that are created through the random assignment

system interacts with families, in that families are offered

Exhibit ES-2. Total Number of Families Assigned to Each Intervention and Number of Followup Survey Respondents Intervention

Families Assigned

Families Responding to the

37-Month Followup Survey

Response Rate (%)

Permanent housing subsidy (SUB)

599

501

83.6

Community-based rapid re-housing (CBRR)

569

434

76.3

Project-based transitional housing (PBTH)

368

293

79.6

Usual care (UC)

746

556

74.5

Total

2,282

1,784

78.2

Sources: Random assignment records; Family Options Study 37-month followup survey

3 The median time from random assignment to the followup survey was 37 months. The minimum time was 32 months and the maximum was 50 months.

4 Gubits et al. (2015) analyzed short-term impacts of the interventions. The study team attempted to contact families for the study’s first followup survey beginning in the 18th month after random assignment. The median time from random assignment to the followup survey was 20 months. The followup period reported in Gubits et al.

(2015) is thus 20 months, but the followup survey is sometimes referred to as the 18-month survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xviii

FAMILY OPTIONS STUDY

Executive Summary

assistance rather than mandated to accept the assistance being

reveals the range of programs used when no priority access

offered. Whether families participate in a program to which

is provided. The programs (including the programs offered

they have gained priority access through their randomly

by the interventions examined in this study) that UC families

assigned intervention reflects the relative desirability and

used exist in communities and would each continue to exist

accessibility of the programs for families within the context of

even with a stronger federal or local push for only one of them.

the other options they may choose to pursue on their own.

Thus, the full-sample comparisons between randomly assigned

interventions—known as “intention to treat,” or ITT, impact

As the report shows, in the 3 years after random assignment,

estimates—provide the best guide to policymakers in a messy,

a substantial number of families did not use the program to

complex world and are reported here as the main study findings.

which they were given priority access, and some used other

programs. The full experimental sample for a given intervention

collectively shows how different forms of housing assistance are

Data Sources

used when families are given priority access to one particular

program type while simultaneously having the freedom to use

The bulk of the impact findings documented in this report is

other forms of assistance available in their communities. Includ ing

drawn from the 1,784 families who responded to the 37-month

all the families randomly assigned to the UC group similarly

followup survey and is based on data from several sources

described in Exhibit ES-3.5

Exhibit ES-3. Data Sources Used in the Analysis of 3-Year Impacts

Study implementation records

Random assignment enrollment data Random assignment enrollment data contain identifiers for enrolled families, responses to eligibility screening questions, information about intervention availability at the time of random assignment, and random assignment result.

Study families

Baseline survey

The baseline survey conducted immediately before random assignment provides information about the adult respondent and the family. The study team defined covariates from these data and included them in impact models to improve the precision of impacts estimates.

Tracking surveys

Tracking surveys conducted 6, 12, and 27 months after random assignment contain updated contact information and details about family composition and housing status.

20- and 37-month followup surveys

The 20- and 37-month followup surveys with adult respondents measure family outcomes. Adults reported on themselves and up to two children, called focal children, who were part of the family at the time of study enrollment.

Focal children were randomly selected within specified age groups.

37-month child assessments

Child assessments, which were conducted with focal children ages 3 years, 6 months to 7 years, 11 months in conjunction with the adult followup survey, measure child well-being outcomes.

37-month child survey

The child survey, conducted with focal children ages 8 to 17 years in conjunction with the adult followup survey, measures child well-being outcomes.

Study intervention providers

Enrollment verification data

Enrollment verification data collected from program providers measure use of the assigned intervention for each family.

Program information

Program information about the housing and services offered during the study period collected from intervention providers describes the interventions.

Cost information

Cost information collected from intervention providers measures costs of overhead, rental assistance, facility operations, supportive services, and capital costs.

Administrative data systems

HMIS records

Homeless Management Information System (HMIS) records, collected from the communities where families enrolled, provide indicators of study families’ participation in homeless assistance programs.

HUD’s PIC records

HUD’s Public and Indian Housing Information Center (PIC) records measure receipt of housing assistance from the Housing Choice Voucher program, public housing programs, and project-based voucher programs.

HUD’s TRACS records

HUD’s Tenant Rental Assistance Certification System (TRACS) records measure receipt of housing assistance through project-based Section 8 programs.

Child welfare records

State and local child welfare agency records (using Adoption and Foster Care Analysis and Reporting System

[AFCARS] reporting definitions) on formal foster care placements and adoptions measure family separations and unifications.

Quarterly wage records

National Directory of New Hires data on quarterly wages measure employment and earnings.

5 Impact findings from administrative earnings records are drawn from the full study sample of 2,282 families. Impact findings from administrative child welfare records are drawn from the full study samples of 5 of the 12 sites (951 families).

3-Year Impacts of Housing and Services Interventions for Homeless Families

xix

FAMILY OPTIONS STUDY

Executive Summary

Hypothesized Effects of the

late 1980s. Resource constraints mean that, outside the context

Interventions

of this study, a permanent housing subsidy is rarely accessible

by families at the outset of an episode of homelessness unless

The study team developed hypotheses about the potential

they already have a place near the top of a waiting list. By contrast,

effects of the interventions based on the conceptual framework

community-based rapid re-housing was developed specifically

underlying the SUB, CBRR, and PBTH interventions. Chapter 3

as a response to homelessness. Unless resources devoted to

of the Short-Term Impacts report (Gubits et al., 2015) discusses

permanent housing subsidies are vastly increased, this assistance

the theoretical framework in detail. The interventions reflect

is unlikely to become widely available to families at the time

different implicit theories about the nature of family homeless-

they are experiencing homelessness. Thus, proponents of rapid

ness and the approaches best suited to address the problem.

re-housing argue that limited resources dedicated to home-

These implicit theories arise from different understandings of

less ness could be stretched to create the best outcomes for

the origins of homelessness, the needs of homeless families, the

the most people by making subsidies temporary (Culhane,

effect of family challenges on achieving residential stability, and

Metraux, and Byrne, 2011).

the appropriate role of the homeless assistance system.

Proponents of rapid re-housing emphasize restoring families to

Some theories posit that household challenges—for example,

conventional housing as swiftly as possible (the “rapid” in rapid

trauma, substance use problems, mental health issues, lack of

re-housing), thereby reducing time staying in shelter and on

job skills—must be addressed first for families to succeed in

the street, which they see as harmful. In addition, they focus on

housing. Others posit that progress on these issues is likely

preventing returns to homelessness. Proponents of permanent

to be achieved only after families are stabilized in permanent

housing subsidies focus more on long-term stability and ques-

housing.

tion whether the short-term subsidies that rapid re-housing

provides are sufficient to foster such stability. Proponents of

These different theories and different perceptions of the

rapid re-housing argue that a temporary subsidy may induce

homeless assistance system’s role result in different emphases

families to strive to become economically self-sufficient sooner.

among three central goals of interventions for homeless

Unlike permanent housing subsidies, rapid re-housing offers

families: (1) ending the immediate episode of homelessness

some services focused on housing and self-sufficiency.

and preventing returns to shelter; (2) fostering longer-term

residential stability; and (3) promoting nonhousing outcomes,

Advocates of both types of subsidies acknowledge that home-

including self- sufficiency, family preservation, and adult and

less families, like other poor families, must contend with a

child well-being.

variety of challenges, but these advocates believe that such

challenges are better addressed by mainstream community

Conceptual Framework for the SUB and

agencies rather than by specialized homeless services. Pro-

CBRR Interventions

ponents of both types of subsidies argue that stable housing

provides a platform from which families can address other

It is appropriate to consider the conceptual rationales for the

problems on their own using community resources, if they

SUB and CBRR interventions together because proponents of

need to and choose to do so, while reserving scarce housing

both permanent housing subsidies and community-based rapid

dollars for housing. Thus, the stability that either a short-term

re-housing believe that the key goal of homeless interventions

or permanent housing subsidy provides may have radiating

should be ending homelessness swiftly, reducing the number of

effects on other aspects of family well-being.

families who return to shelter, and restoring families to housing

stability. This position follows from the view that family home-

For comparisons involving the SUB and CBRR interventions

lessness is largely a consequence of housing costs that outstrip

and usual care, the study team developed four hypotheses that

the incomes of poor families, a problem that housing subsidies

derive from this conceptual framework.

can solve. Subsidies—whether the permanent housing subsidies

of the SUB intervention or the temporary housing subsidies of

Hypotheses for Comparisons Involving the

the CBRR intervention—can help families obtain and maintain

SUB and CBRR Interventions and Usual Care

stable housing.

SUB Versus UC

Permanent housing subsidies were not created as a response

Relative to the UC intervention, the SUB intervention will

to homelessness. Instead, they already existed as an element of

reduce shelter use and improve housing stability and may

the broader social safety net at the time the homeless assistance

improve family preservation, adult well-being, and child

system began to expand into its current configuration in the

well-being.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xx

FAMILY OPTIONS STUDY

Executive Summary

CBRR Versus UC

and child outcomes are usually seen as more distal outcomes.

Relative to the UC intervention, the CBRR intervention will

Given this conceptual framework for project-based transitional

reduce shelter use and may improve housing stability, employ-

housing, the study team defined five hypotheses about the

ment and earnings, family preservation, adult well-being, and

potential effects of the PBTH intervention when compared with

child well-being. It will reduce the length of the shelter stay at

usual care and with the SUB and CBRR interventions.

the time of study entry and may be less costly.

Hypotheses for Comparisons Involving the

SUB Versus CBRR

PBTH Intervention

Relative to the CBRR intervention, the SUB intervention will

reduce shelter use and improve housing stability and may

PBTH Versus UC

improve family preservation and adult and child well-being.

Relative to the UC intervention, the PBTH intervention will

reduce shelter use and improve housing stability, employment,

Relative to the SUB intervention, the CBRR intervention will

earnings, education, and adult well-being and may improve

reduce the length of the shelter stay at the time of study entry

family preservation and child well-being.

and will be less costly. It may improve employment and earnings.

PBTH Versus SUB

Conceptual Framework for the PBTH

From the perspective of transitional housing proponents: Relative

Intervention

to the SUB intervention, the PBTH intervention will improve

Proponents of transitional housing have a different understand-

employment, earnings, education, and adult well-being and

ing of the origins of family homelessness and the appropriate

may improve long-term housing stability, family preservation,

role of the homeless assistance system than do proponents of

and child well-being.

permanent housing subsidies and rapid re-housing. Although

From the perspective of permanent housing subsidy propo-

the housing market is difficult for poor families, most families

nents: Relative to the PBTH intervention, the SUB intervention

do not experience homelessness. Proponents of transitional

will reduce shelter use and improve housing stability and

housing emphasize that many families who do become home-

may improve family preservation, adult well-being, and child

less have barriers in addition to poverty that make it difficult

well-being.

for them to secure and maintain housing. Thus, housing sub-

sidies alone may be insufficient to ensure housing stability and

PBTH Versus CBRR

other desirable outcomes, particularly for families who have

From the perspective of transitional housing proponents:

been in shelter for at least 7 days (for example, Bassuk and

Relative to the CBRR intervention, the PBTH intervention will

Geller, 2006). Family needs may arise from poverty, health,

improve employment, earnings, education, and adult well-

disability, or other problems that led to homelessness to begin

being and may improve long-term stability, family preservation,

with or from the disruptive effects of homelessness on parents

and child well-being.

and children.

From the perspective of rapid re-housing proponents: Relative

The premise for project-based transitional housing is that, by

to the PBTH intervention, the CBRR intervention will reduce

addressing these barriers and needs in a supervised residential

shelter use and may improve housing stability, family preser-

setting, PBTH programs lay the best foundation for ongoing

vation, adult well-being, child well-being, employment, and

stability. Basing their work on family needs, case managers co-

earnings. It will reduce the length of time families spend in

ordinate the services (on site or by referral) to lay the essential

places not meant for human habitation and in shelters, which

groundwork for later independence.

are costly.

Different PBTH programs focus on different issues, but all

Even if the longer housing subsidies of the SUB intervention or

provide supportive services designed to reduce barriers to

the more extensive social services of the PBTH intervention are

housing, enhance adults’ well-being, and bolster adults’ ability

important for some families, an important question is whether

to manage in ordinary housing after they leave programs (Burt,

all families need such intensive involvement in the homeless

2010). Practitioners’ goals for project-based transitional hous-

assistance system. Thus, the study team also developed hypoth-

ing, as documented in the literature (for example, Burt, 2006),

eses that the more intensive interventions would have larger

thus extend beyond housing stability to adult well-being and

effects on outcomes for families who faced more psychosocial

aspects of family self-sufficiency. Although some PBTH pro-

challenges and housing barriers.

grams provide services directly to children, family preservation

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxi

FAMILY OPTIONS STUDY

Executive Summary

Baseline Characteristics of the

intervention but had slightly more restrictive eligibility

Study Sample

requirements. Thus, it was more difficult for families to meet

the eligibility criteria of the programs that are ordinarily part of

At the time of random assignment, characteristics of the 2,282

the homeless assistance system (PBTH and CBRR) than for the

families who enrolled in the Family Options Study were similar

programs that are not (SUB programs).

to characteristics of families who experience homelessness

For a family to use the type of program offered to the group to

nationwide. This observation is true even though the sites were

which it had been assigned, it had to (1) pass an eligibility de-

not a randomly selected sample of communities.

termination conducted by the specific program to which it was

The typical family in the study consisted of an adult woman, a

referred and (2) choose to take up the program. The Interim

median of 29 years old, living with one or two of her children

Report (Gubits et al., 2013) found that some of the families who

in an emergency shelter. At baseline, 30 percent of families

passed the initial screening by the study were later deemed

had more than one adult present. Nearly all families who had

ineligible by the programs to which they were referred and

two adults present were headed by couples, and 10 percent

thus were denied services in the program to which they were

of families had a partner living elsewhere. As in other studies

given priority access. Other fully eligible families chose not to

of homelessness among families, members of minority groups

take up the program to which they were given priority access.

were overrepresented, even in proportion to the poverty

Compared with the CBRR and SUB interventions, the PBTH

population.

intervention had both the highest proportion of families found

ineligible after random assignment and the highest proportion

A plurality of families (43 percent) had only one child with

of families who chose not to take up the program to which they

them in the shelter, and one-half of the families were with a

were given priority access. Considering both initial screening

child younger than age 3. Most families in the study (79 per-

by the study and later eligibility screening by programs, the

cent) were not homeless immediately before entering the shel-

SUB intervention was the most accessible to families and the

ter from which they were recruited into the study. About 63

PBTH intervention was the least accessible.

percent of family heads in the study, however, had experienced

homelessness at some other point in their lifetime, with 16

The Interim Report (Gubits et al., 2013) concluded that home-

percent of adult respondents having experienced homelessness

less assistance programs in the study communities imposed

as a child. An even greater proportion (85 percent) indicated

eligibility criteria that hampered their ability to serve families

they lived doubled up at some point as an adult, defined in the

in shelter who needed the assistance. Even when programs had

survey as “staying with family or friends because you couldn’t

space available, the programs often screened out families in

find or afford a place of your own.”

shelter based on eligibility criteria such as insufficient income,

substance abuse, criminal histories, and other factors that pre-

Most family heads were not working at the time of random

sumably contributed to the families’ homelessness. Moreover,

assignment (83 percent), and more than one-half had not

families who are homeless do not always pursue the programs

worked for pay in the previous 6 months. The median annual

offered to them, which suggests that some programs deliver

household income of all families in the study at baseline was

assistance that some families perceive as less valuable to them

$7,410. Many reported they either had a poor rental history

than other alternatives available in their communities.

(26 percent had been evicted) or had never been a leaseholder

(35 percent). Nearly one in three reported either post-traumatic

stress disorder or serious psychological distress, and 21 percent

Study Findings Over the 3-Year

reported a disability that prevents or limits work.

Followup Period

Intervention Eligibility Screening

The Short-Term Impacts report (Gubits et al., 2015) presented

findings about the study families and impacts of the interven-

and Family Decisions

tions at 20 months after random assignment. The body of the

current report presents findings about the families and inter-

The Interim Report: Family Options Study (hereafter the Interim

ventions 17 months later, at 37 months after random assign-

Report; Gubits et al., 2013) examined intervention availability

ment. The results from both time points are important. Some

and family eligibility at randomization for the 2,282 families

impacts that were detected at 20 months are not detected in the

in the Family Options Study. Both availability of interventions

37-month analysis. Other impacts are detected at 37 months

and family eligibility, according to screening before random

but were not apparent at the earlier point. Impacts found at

assignment, were most constrained for the PBTH intervention.

either point in time hold importance when considering the

The CBRR intervention was more available than the SUB

relative benefits of the interventions during 3 years of study.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxii

FAMILY OPTIONS STUDY

Executive Summary

Usual Care Group During the Followup Period

mainstream housing programs and permanent supportive

housing; (2) rapid re-housing rental assistance; (3) transitional

Program Use of the Usual Care Group

housing; and (4) emergency shelter.

Inasmuch as the experiences of families in the 12 study com-

mun ities reflect families’ experiences in the nation more

For some families assigned to the UC group, the emergency

broadly, the experiences of families assigned to the UC group

shelter stay was the only interaction with the homeless or housing

inform policymakers about what typically happens to families

assistance systems. As Exhibit ES-4 shows, however, most

who have been in shelter for at least 7 days and who do not

UC families ultimately found their way to other types of

receive priority access to any type of designated assistance. On

assistance. Of the UC families who responded to the 3-month

average, families assigned to the UC group spent 4 months

followup survey, 20 percent received rapid re-housing and 30

in emergency shelter during the 37-month followup period.

percent received transitional housing at some time during the

Exhibit ES-4 shows the proportions of UC families who used

3-year followup period. Considering all forms of permanent

different types of programs in each month during the first

subsidy (housing choice vouchers, public housing, permanent

32 months after random assignment (the longest period for

supportive housing, a project-based voucher, or assistance in a

which the study team has data for all families). It breaks out

Section 8 project), even without priority access, 37 percent of UC

four categories: (1) any permanent housing subsidy, including

families used some form of permanent subsidy for an average of

19 months at some point during the 3-year followup period.6

Exhibit ES-4. Program Use of UC Families for 32 Months After RA

100

90

80

70

Unknown/no program use

60

50

40

Any permanent housing subsidy

30

Rapid re-housing

20

Percent of UC families using program type in month (percent)

Transitional housing

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

UC = usual care.

RA = random assignment.

Notes: This exhibit shows program use for all families assigned to UC who responded to the 37-month survey. Complete Program Usage Data for 37-month respondent families are available only through month 32 after random assignment. Families who have more than one type of program use in a calendar month are counted fractionally in each type.

Source: Family Options Study Program Usage Data

6 In the entire study, 746 families were randomly assigned to the UC group. Of these families, 556 responded to the followup survey and are the families shown in Exhibit ES-4. Different subsets of these 556 families form the comparison groups for SUB, CBRR, and PBTH.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxiii

FAMILY OPTIONS STUDY

Executive Summary

This percentage continued to grow over time, whereas use of

had been in shelter in the past 12 months, 18 percent reported

other housing programs declined. By the time of the followup

being homeless in the 6 months before the 37-month survey,

survey (not shown in Exhibit ES-4) more than one-half of the

and 5 percent were in emergency shelter in the survey month.

UC families (60 percent) were not using any type of assistance

A little more than two-thirds (69 percent) of families were liv-

measured, nearly one-third (30 percent) were using some type

ing in their own house or apartment (up from 58 percent at 20

of permanent housing, and the remainder were using emergency

months). Physical health, substance use, and intimate partner

shelter, rapid re-housing, or transitional housing.

violence were at levels similar to those at 20 months. Only

37 percent of family heads were working for pay in the week

Outcomes of the Usual Care Group

before the survey. Median annual cash income from all sources

UC families were not faring well 20 months after enrollment

for the previous calendar year was about $12,000, and nearly

in the study. One-half had spent at least 1 night homeless or

one-half of the families (47 percent) were food insecure.

doubled up in the 6 months before the 20-month survey or

had been in shelter in the past 12 months. In the 6 months

Program Use for Pairwise Comparisons

before the survey, 15 percent of families had been separated

from a child who was with the family in shelter at study outset,

The three panels of Exhibit ES-5 show the use of programs

and one-fourth of these separated children were in foster care.

by the SUB, CBRR, and PBTH groups in each of the first 32

At the time of the survey, 32 percent of family heads reported

months of the followup period (the period for which complete

fair or poor health, 15 percent reported alcohol dependence or

data are available). Program use for each group is initially

substance abuse, and 12 percent had experienced intimate part ner

dominated by the program to which families were offered pri-

violence in the past 6 months. Less than one-third of UC family

ority access. The SUB group was offered permanent assistance,

heads (31 percent) were working for pay and previous-year

and most families were still using the permanent subsidy they

total family income was only about $9,000 per year—much

were offered by the end of the followup period. The CBRR and

less than what is needed to rent market-rate housing for a

PBTH groups were offered temporary assistance, and, by the

family in most communities.

end of the followup period, most families had ceased using the

temporary assistance they were offered (earlier for the CBRR

At 37 months after random assignment, UC families still faced

group than for the PBTH group, as expected, because of typical

substantial challenges, although, on average, their circumstanc-

program lengths). In both the CBRR and PBTH groups, the

es had improved somewhat since the 20-month survey. Of the

proportions of families using permanent housing subsidies

whole group, 39 percent had spent at least 1 night homeless

grew steadily during the followup period.

or doubled up in the 6 months before the 37-month survey or

Exhibit ES-5. SUB, CBRR, and PBTH: Program Use for 32 Months After RA

100

Unknown/no program use

90

80

70

60

50

Any permanent housing subsidy

40

30

Rapid re-housing

20

Percent of SUB families using program type in month

10

Transitional housing

Emergency shelter

0

0

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxiv

FAMILY OPTIONS STUDY

Executive Summary

Exhibit ES-5. SUB, CBRR, and PBTH: Program Use for 32 Months After RA (continued) 100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

20

Rapid re-housing

Percent of CBRR families using program type in month

Transitional housing

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

Rapid re-housing

20

Transitional housing

10

Percent of PBTH families using program type in month (percent)

Emergency shelter

0

0

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

RA = random assignment

Notes: This exhibit shows program use for all SUB, CBRR, and PBTH families who responded to the 37-month survey. Complete Program Usage Data for 37-month respondent families are available only through month 32 after random assignment. Families who have more than one type of program use in a calendar month are counted fractionally in each type.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxv

FAMILY OPTIONS STUDY

Executive Summary

Exhibit ES-6 documents the program use of the study families

6. Project-based vouchers and units in Section 8 projects in

who responded to the 37-month followup survey—the sample

places other than Bridgeport.

for the longer-term impact analysis (and same sample shown

7. Any permanent subsidy (that is, any of the programs that

in Exhibit ES-5). The exhibit shows the percentages of families

comprise the SUB intervention in this study, permanent

who ever participated in several types of housing assistance

supportive housing, public housing, or project-based

programs between random assignment and the followup survey

vouchers and units in Section 8 projects).

response. The columns in Exhibit ES-6 are organized by pair-

wise comparison. The exhibit displays the number of families

8. Emergency shelter.

included in each comparison (see row 10) and details their

program use during the followup period (see rows 1 to 9). It

The experimental contrasts in use of these programs are

accounts for eight types of programs:

depicted in the exhibit. The exhibit also shows (see row nine)

the proportion of families who used no homeless or housing

1. Permanent housing subsidies offered to the SUB group

assistance and no emergency shelter after the first 6 months

(housing choice vouchers, public housing in Honolulu, and

after random assignment.

project-based vouchers in Bridgeport).

Exhibit ES-6 shows that the intervention assignments created

2. Rapid re-housing (that is, the temporary subsidies offered to

substantial contrasts in program use, particularly in the use of

the CBRR group).

programs that reflect the intended contrast (the shaded boxes).

For example, in the SUB-versus-UC comparison, 83 percent of

3. Transitional housing.7

families assigned to the SUB intervention used the permanent

4. Permanent supportive housing.

housing subsidies they were offered, whereas only 13 percent

of families assigned to the UC group used these permanent

5. Public housing in places other than Honolulu.

Exhibit ES-6. Program Use Since RA for All Pairwise Impact Comparisons

Percent of Families Who Ever Used Program Type From RA to 37-Month Followup Surveya

Type of Homeless or Housing Assistance

SUB

UC

CBRR

UC

PBTH

UC

SUB CBRR SUB PBTH CBRR PBTH

Permanent housing subsidies offered to the SUB groupb

83.2

12.7

9.8

12.2

9.6

10.6

82.3

9.8

82.3

7.2

6.7

8.7

Rapid re-housingc

11.4

22.9

58.5

22.5

13.7

17.8

14.7

60.5

5.7

12.0

54.6

15.8

Transitional housingd

7.4

28.9

23.2

27.5

53.2

34.6

7.4

19.8

9.4

49.5

30.7

53.4

Permanent supportive housing

3.0

10.7

9.8

11.7

11.0

11.6

3.9

11.6

2.1

11.6

9.2

10.7

Public housing

1.6

10.3

10.7

9.8

8.3

8.5

1.1

10.9

1.6

8.2

11.5

9.1

Project-based vouchers/Section 8 projects

1.5

6.1

5.6

6.3

6.2

7.2

1.1

7.0

1.7

7.2

4.8

6.1

Any permanent housing subsidye

88.4

37.9

35.4

37.9

33.0

34.7

87.4

38.4

86.6

30.9

32.3

33.3

Emergency shelterf

84.4

89.5

90.7

90.0

83.6

89.7

84.9

90.4

87.9

83.0

88.1

86.5

No use of homeless or housing programsg

5.1

24.3

9.2

24.4

18.1

26.9

4.5

9.5

7.1

20.5

14.3

16.8

N

501

395

434

434

293

259

362

290

215

201

180

184

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Source: Family Options Study Program Usage Data

7 Some transitional housing programs are based in projects or facilities that families leave after exiting the program. These programs are studied here, hence the term project-based transitional housing. Other transitional housing programs use residential units in the community so that families can “transition in place” to unassisted housing without having to move after supports are no longer needed. Transition-in-place programs of this sort share many of the same characteristics of CBRR programs, so they were not included as programs to which PBTH families could be directed after random assignment. The team made this decision to provide a stronger contrast between the PBTH and CBRR interventions studied. Some PBTH programs to which families were offered priority access provided units in the community (called “scattered-site” units) without the opportunity to transition in place. The Homeless Management Information System records, an important data source for observing program use, unfortunately do not distinguish between project-based and transition-in-place transitional housing. Therefore, some of the transitional housing use shown in Exhibit ES-6 may have been in transition-in-place units.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxvi

FAMILY OPTIONS STUDY

Executive Summary

housing subsidies. The durations of assistance were also longer

are organized into five panels corresponding to each outcome

for the assigned interventions (not shown in the exhibit). In

domain. The exhibit columns show the mean value of each

the SUB-versus-UC comparison, families assigned to the SUB

outcome for the entire UC group, followed by impact estimates

intervention who used the offered permanent housing subsidies

for each outcome in each of the six pairwise comparisons. Aster-

did so for an average of 31 months, whereas UC families who

isks to the right of the impact estimates denote the statistical

used these subsidies without priority access used them for only

significance of the differences between the two groups being

21 months, on average. These findings generally reflect the

compared, with more asterisks indicating higher levels of

longer time it took UC families to obtain access to permanent

statistical significance.

housing subsidies, if they did so at all. Considering all forms of

Within each domain, Exhibit ES-7 presents impacts on three

permanent subsidy, Exhibit ES-6 shows that 38 percent of UC

or four outcomes. For the first four outcome domains (housing

families in the SUB-versus-UC comparison used any form of

stability, family preservation, adult well-being, and child well-

permanent subsidy during the followup period compared with

being), the outcomes are specified so that lower values indicate

88 percent of families assigned to the SUB intervention.

improvements. That is, for these domains, impact estimates with

In the CBRR-versus-UC comparison, the contrast in use of rapid

negative values indicate reductions in unfavorable outcomes

re-housing rental assistance is smaller than for the SUB-versus-

or improvements for families. For the self-sufficiency domain,

UC comparison. Of families assigned to the CBRR group, 59

the goals of the interventions are to achieve higher values for

percent used the offered assistance during the followup period

each outcome. Thus, positive values for self-sufficiency impact

compared with 23 percent of their counterparts in the UC group

estimates indicate improvements. Detailed definitions for the

for this comparison. Families assigned to the CBRR group

full set of outcomes appear in Chapter 2 and Appendix B of the

who used rapid re-housing programs did so for an average of

report.

8 months, the same length of time that UC families without

The results of the pairwise comparisons appear in the following

priority access to CBRR used rapid re-housing programs. Similar

sections.

proportions of CBRR and UC families used any form of perma-

nent subsidy (35 and 38 percent, respectively).

SUB Versus UC

In the PBTH-versus-UC comparison, 53 percent of families

The most notable effect of the SUB intervention relative to the

assigned to the PBTH group used transitional housing during

UC intervention was its reduction in homelessness and doubling

the followup period and 35 percent of UC families also used

up. Assignment to the SUB intervention reduced the proportion

transitional housing. PBTH families who used transitional

of families who had a shelter stay in months 7 to 18 by almost

housing of any kind did so for an average of 15 months,

one-half and in months 21 to 32 by more than three-fourths. At

whereas the UC families who used transitional housing without

both the 20-month and 37-month followup points, assignment

priority access did so for 11 months, on average. Altogether, 33

to the SUB intervention reduced by more than one-half the pro-

percent of families assigned to the PBTH group used any form

por tion of families who reported having spent at least 1 night

of permanent subsidy compared with 35 percent of UC families

in shelter or in places not meant for human habitation in the

in the PBTH-versus-UC comparison.

past 6 months, increased the proportion of families living in

their own place by 15 percentage points, and reduced the

Impact Estimates for Pairwise Comparisons

number of places the families lived in the past 6 months.

Before seeing the results of the impact analysis, the study team

Additional effects occurred for some family preservation and

prespecified impacts on 18 key outcomes in the six pairwise

child and adult well-being indicators. At 20 months, assignment

comparisons to present in this executive summary. That step

to the SUB intervention reduced the proportion of families

was taken to prevent the selective presentation of statistically

separated from a child present at baseline (10 percent in SUB

significant results in the executive summary from among the

families compared with 17 percent in UC families). This effect

84 outcomes examined for each comparison in the body of the

was not detected in the 3-year analysis. At 37 months (but not

report (534 impact estimates). The outcomes deemed most

found earlier), assignment to the SUB intervention increased

central to the study and those anticipated a priori to be most

separations from the spouse or partner present at baseline

likely to be affected by the interventions were selected for this

(48 percent in SUB families compared with 34 percent in UC

executive summary presentation. Impacts on the full set of

families). Assignment to the SUB intervention also improved

outcomes are presented in Chapters 3 through 6 of the report.

some of the measures of adult well-being preselected for the

executive summary presentation. At 20 months, assignment

Exhibit ES-7 reports estimated 3-year impacts for the 18 prespec-

to the SUB group reduced psychological distress, reduced

ified outcomes for each pairwise comparison. The exhibit rows

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxvii

FAMILY OPTIONS STUDY

Executive Summary

Exhibit ES-7. Summary of Impacts for Six Policy Comparisons

Mean

ITT Impact Estimates

Outcome

All UC

SUB

CBRR

PBTH

SUB

SUB

CBRR

Group

vs. UC

vs. UC

vs. UC

vs. CBRR

vs. PBTH vs. PBTH

Housing stability (intervention goal: lower values)

At least 1 night homelessa or doubled up in past 6 months or in shelter

38.6

– 21.1***

1.9

0.3

– 20.4***

– 24.4***

– 3.1

in past 12 monthsb (%) [confirmatory]

At least 1 night homelessa or doubled up in past 6 months (%)

34.9

– 18.2***

1.8

0.2

– 15.5***

– 22.1***

– 2.8

Number of places lived in past 6 months

1.59

– 0.25***

0.03

– 0.03

– 0.14**

– 0.31***

0.09

Any stay in emergency shelter in months 7 to 18 after RA (%)

17.3

– 14.3***

– 2.5

– 6.0**

– 12.8***

– 5.5**

1.6

Family preservation (intervention goal: lower values)

Family has at least one child separated in past 6 monthsc (%)

16.7

– 3.3

– 0.5

2.0

– 2.2

– 8.1**

– 3.2

Spouse/partner separated in past 6 months, of those with spouse/partner 38.1

13.4**

8.6

11.7

– 6.6

0.3

0.5

present at RAd (%) [limited base]

Family has no child reunified, of those families with at least one child

66.3

– 6.8

– 3.4

4.1

– 9.8

– 21.0

– 0.1

absent at RAe (%) [limited base]

Adult well-being (intervention goal: lower values)

Health in past 30 days was poor or fair (%)

31.4

3.1

1.3

– 0.7

0.1

– 2.2

– 1.0

Psychological distressf

0.00

– 0.11*

– 0.02

– 0.01

0.01

– 0.11

– 0.20**

Alcohol dependence or drug abuse in past 6 monthsg (%)

11.3

– 2.1

– 1.7

2.8

1.8

– 0.6

– 7.1**

Experienced intimate partner violence in past 6 months (%)

10.5

– 4.0*

– 1.6

– 1.1

– 1.8

– 1.2

0.2

Child well-being (intervention goal: lower values)

Number of schools attended since RAh

2.10

– 0.15**

0.02

0.06

– 0.26***

– 0.22*

0.01

School absences in past month (ages 5 to 17 years)i

1.06

– 0.08

– 0.19

– 0.19

0.09

– 0.15

– 0.17

Poor or fair health (%)

5.9

2.0

– 0.6

– 0.0

1.0

– 2.9

– 2.2

Behavior problemsj

0.59

– 0.23**

– 0.20**

– 0.06

0.06

– 0.25*

– 0.23*

Self-sufficiency (intervention goal: higher values)

Work for pay in week before survey (%)

37.0

– 0.9

0.5

0.6

– 0.6

– 2.0

– 3.8

Total family income ($)

12,099

– 883

– 505

– 191

– 901

– 1,230

– 1,745

Household is food secure (%)

53.2

9.6***

4.2

1.9

2.6

12.8***

11.6**

Number of families

556

895

868

551

652

414

363

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test (not adjusted for multiple comparisons).

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing. Additional impacts on the use of transitional housing are provided in Appendix E.

b After adjustment for multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-UC, SUB-versus-CBRR, and SUB-versus-PBTH comparisons.

c Percentage of families in which a child who was with the family at baseline became separated from the family in the 6 months before the 37-month survey.

d Percentage of families in which a spouse or partner who was with the family at baseline became separated from the family in the 6 months before the 37-month survey.

e Percentage of families in which at least one child was separated from the family at baseline where no child was reunited with the family at the time of the 37-month survey.

f Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress. Impacts shown as standardized effect sizes. Effect sizes were standardized by dividing impacts by standard deviation for the UC group.

g Measures evidence of alcohol dependence or drug abuse using responses to the Rapid Alcohol Problems Screen (RAPS-4) and six items from the Drug Abuse Screening Test (DAST-10).

h Number of schools outcome is topcoded at 4 or more schools.

i Absences outcome is defined as 0 = no absences in past month; 1 = one to two absences; 2 = three to five absences; 3 = six or more absences. This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

j Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire, or SDQ.

Notes: Impact estimates are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

evidence of alcohol and drug problems, and halved intimate

random assignment (evident at both time points). Assignment

partner violence compared with usual care. In the 3-year analysis,

to the SUB group also reduced the number of school absences

reductions in psychological distress and intimate partner

(at 20 months only) and reduced the number of behavior

violence were still evident.

problems reported by parents (at 37 months only).

Assignment to the SUB group also caused improvements in a

At 20 months after random assignment, assignment to the

few of the child well-being measures shown in Exhibit ES-7.

SUB group, when compared with usual care, reduced the

Relative to usual care, assignment to the SUB intervention

proportion of family heads working in the week before the

reduced the number of schools that children attended since

survey by 6 per centage points. This result is consistent with

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxviii

FAMILY OPTIONS STUDY

Executive Summary

economic theory, given that housing subsidies lessened the

The lack of impacts on adult well-being and family self-sufficiency

need for disposable income. A reduction was not evident at the

are particularly noteworthy here, given the emphasis placed

time of the 37-month survey, although reports of work activity

by PBTH programs on delivering supportive services in these

between the followup points showed that the proportion of

areas. None of the eight indicators examined for results in these

family heads with any work effort between the surveys was

domains showed any impact from assignment to the PBTH

reduced by 6 percentage points (not shown in the exhibit).

intervention compared with usual care, nor did assignment to

the PBTH intervention provide better family preservation or

Assignment to the SUB intervention caused improvements in

child well-being outcomes than usual care. Overall, 3 years after

food security, increasing the percentage of households classified

assignment, the study did not find evidence that the goals of this

as food secure by 10 percentage points relative to usual care at

distinctive approach to assisting families facing unstable hous-

both followup points.

ing situations were achieved relative to leaving families to find

CBRR Versus UC

their way out of shelter without priority access to any program.

Almost no evidence of assignment to the CBRR intervention

SUB Versus CBRR

differentially affecting outcomes compared with assignment to

The most noteworthy effect of assignment to the SUB group

the UC intervention appears at either 20 or 37 months of the

relative to assignment to the CBRR group was improved hous-

followup period across the domains of housing stability, family

ing stability. Relative to assignment to the CBRR intervention,

preservation, and adult and child well-being. It is most striking

assignment to the SUB intervention reduced the proportion of

that, relative to usual care, priority access to the temporary rental

families who had a shelter stay or an experience of homelessness

assistance offered in the CBRR intervention does not show impacts

or doubling up—producing effects, at both followup points,

on subsequent stays in shelter or places not meant for human

nearly as large as those from the SUB-versus-UC comparison

habitation during the 3-year followup period. Indications about

covered previously. By the time of the 37-month survey, the

consequences for children are limited, with assignment to the

improvement in housing stability seems largely driven by a

CBRR intervention leading to a reduction in school or childcare

reduction in doubling up, rather than in shelter stays or in places

absences compared with usual care (at 20 months only) and in

not meant for human habitation (not shown in the exhibit). At

parent-reported behavior problems (at 37 months only).

both followup points, the greater stability afforded by assign-

At 20 months, relative to usual care, assignment to the CBRR

ment to the SUB intervention was also evidenced in a reduction

intervention improved food security and family income (with

in the number of places families lived in the past 6 months

previous-year income for CBRR families about $1,100 more

relative to assignment to the CBRR intervention.

than for UC families). Neither of these effects was evident at

At 20 months, other scattered effects shown in the SUB-versus-

37 months.

CBRR comparison mostly suggest more favorable outcomes

PBTH Versus UC

for families assigned to the SUB intervention. In particular,

Compared with the UC intervention, assignment to the PBTH

assignment to the SUB intervention relative to assignment to

intervention improved housing stability but showed no evidence

the CBRR group reduced separations of spouses and partners,

of impact in other domains. Assignment to the PBTH intervention

intimate partner violence, and the number of schools attended

reduced the proportion of families who had a stay in emergen-

by children; however, it also reduced total prior-year income

cy shelter in months 7 to 18 after random assignment from 27

for the family.

to 19 percent (a much smaller impact than that of assignment

At the later followup point, the study found few statistically

to the SUB intervention relative to usual care). Assignment to

significant differences outside the housing stability domain. Of

the PBTH intervention also reduced the proportion of families

the impacts detected at 20 months in this comparison, only the

who had any stay in emergency shelter in months 21 to 32

reduction in the number of schools attended was evident at 37

after random assignment. This time period corresponds to the

months.

last full year observed for all sample members but overlaps a

portion of the 24-month period that families could have stayed

SUB Versus PBTH

in transitional housing. For outcomes occurring further along

In most respects, the measured effects of assignment to the

in the followup period, no evidence indicates that assignment

SUB intervention compared with assignment to the PBTH

to the PBTH intervention is superior to usual care regarding

intervention mirror those found in the previous SUB-versus-

housing stability. Besides housing stability, the study yields

UC comparison. At both of the study’s followup points, the

no evidence of effects of assignment to the PBTH intervention

most noteworthy effect of assignment to the SUB intervention

relative to usual care in other domains at either point in time.

relative to the PBTH intervention is its greater prevention of

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxix

FAMILY OPTIONS STUDY

Executive Summary

stays in shelter or places not meant for human habitation in

Do Certain Interventions Work Better

each 6-month window before the followup surveys and in

When Offered to Families Facing Greater

its reduction in doubled-up housing situations. The greater

Difficulties?

stability afforded by assignment to the SUB intervention was

evidenced in a reduction in the number of places families lived

A central question motivating the Family Options Study is

in the past 6 months compared with assignment to the PBTH

whether some interventions work better than other interventions

intervention.

for families who had particular characteristics. The previously

described study findings show that, on average, the SUB inter-

At 20 months, the SUB-versus-PBTH comparison showed

vention had substantial impacts relative to the other interventions,

effects on family preservation, adult well-being, and child

not only for housing stability but also for outcomes in other

well-being. The most notable effects of assignment to the SUB

domains. Do all families who experience homelessness need

group relative to assignment to the PBTH group were a reduc-

a deep permanent housing subsidy, however, or might some

tion in the proportion of families who had a child separation

do as well on their own, in usual care, or with the shorter and

in the past 6 months, a decrease in the psychological distress

often shallower subsidies of offered in the CBRR intervention?

reported by family heads, and a reduction in the number of

Conversely, although, on average, assignment to the PBTH

schools that children attended since random assignment. The

intervention had few impacts relative to other interventions,

3-year analysis found two of these three effects: (1) reductions

might some families who face more challenges benefit more

in child separation and in the number of schools attended and

from its intensive social services? The more general form of

(2) an additional reduction in child behavior problems. No ef-

these questions is whether the relative benefits of the longer

fects on adult well-being were found at the later followup point.

term or more intensive interventions (SUB and PBTH) might

In the self-sufficiency domain at 20 months, the study found

increase as families’ reported difficulties increase. Because of

that assignment to the SUB group relative to assignment to the

the number of family characteristics that could lead to differ-

PBTH group caused a reduction in work effort. Assignment

ential effects of interventions, the study team confined analyses

to the SUB intervention reduced the proportion of family

of impact variations to two broad categories of family char-

heads who worked at 20 months (from 36 to 25 percent) and

acteristics, summarized in indices of psychosocial challenges

reduced average prior-year income from $10,500 to $9,000.

and barriers to housing. The study team examined whether the

These effects were not found at 37 months. At both followup

impact of the interventions relative to each other and to usual

points, the additional resources represented by the permanent

care increased as families’ scores on these indices increase.

housing subsidy served to increase food security relative to

It is clear that families in this study experience high levels of

assignment to the PBTH intervention.

both psychosocial challenges and barriers to housing, which

was by design: the study enrolled families only after they had

CBRR Versus PBTH

spent at least 7 days in shelter. The examination of these po-

The CBRR-versus-PBTH comparison does not yield a strong

tential moderator effects at 20 and 37 months does not provide

pattern of effects across the study domains. In particular, no

appreciable evidence that any of the interventions studied work

evidence indicates an effect on housing stability or family

comparatively better for families who have more psychosocial

pre servation. The CBRR-versus-PBTH comparison, however,

challenges or housing barriers than for families who face fewer

yields statistically significant effects for four of the outcomes

difficulties. As a result, the study’s clearest guidance for policy

selected for the executive summary presentation in the adult

for all types of families at the 3-year followup point consists of

well-being, child well-being, and self-sufficiency domains.

the main study results on overall impacts.

All these significant results favored assignment to the CBRR

intervention. In the adult well-being domain, at both followup

Intervention Costs

points, assignment to the CBRR intervention reduced the

proportion of adult respondents reporting alcohol dependence

The Family Options Study interventions were intended to vary

or drug abuse in the past 6 months and lowered the amount of

in both intensity and duration. The permanent housing subsi-

psychological distress for family heads relative to assignment to

dies offered to the SUB group provided a deep rental subsidy,

the PBTH intervention. Assignment to the CBRR intervention

limiting families’ contributions to rent and utilities to about 30

also reduced the proportion of family heads reporting poor

percent of monthly adjusted income. SUB programs did not

or fair health (at 20 months only), reduced parent-reported

provide supportive services, but the rental subsidy was for an

child behavior problems (at 37 months only), and reduced the

indefinite duration. PBTH programs provided intensive housing

proportion of families who are food insecure (at both followup

and services support for a relatively long duration. CBRR pro-

points).

grams provided a short-term rental subsidy with more limited

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxx

FAMILY OPTIONS STUDY

Executive Summary

supportive services, while emergency shelter programs often

$1,400. Across the 10 sites where permanent housing subsidies

offered intensive supportive services and housing for a limited

were offered to families assigned to the SUB group, the average

time. The study team compared the costs of the interventions

per-family per-month cost of subsidy program participation

using two measures of cost:

ranged from $770 to $2,100, largely reflecting differences in

the local cost of rental housing.

1. Per-family monthly cost of using a particular type of

program (for example, permanent housing subsidies of the

Total Costs of Programs Used During the 3-Year

type offered to SUB families).

Followup Period

2. Per-family cost of all programs used by those assigned to

Exhibits ES-4, ES-5, and ES-6 show how families assigned to

an intervention (that is, the SUB, CBRR, PBTH, and UC

the four interventions used different mixes of homeless and

interventions) during the 3-year followup period.

housing assistance programs during the 3 years studied. The

study team combined information about program use with

The first measure provides information about the relative costs

per-family monthly program costs to estimate the total costs of

of funding different types of programs. The second measure

programs used by families in each intervention group in each of

provides context for interpreting the impacts of the contrasting

the six pairwise comparisons.

results of random assignment—that is, the impact of assign-

ment to a particular intervention presented in the pairwise

At 20 months after random assignment, the study found that

comparisons in Chapters 3 through 6. This latter measure re-

the per-family cost of all program use by families assigned usual

flects the combined cost of all homeless and housing assistance

care was about $30,000. The cost of all program use of all types

program types that families in each pairwise comparison used

for families assigned to the SUB intervention was about the

during the 3-year followup period, those offered by the study

same as that for families assigned to usual care, slightly more

and other types as well.

than that for families assigned to the CBRR intervention, and

clearly less than that for PBTH families. The near-equivalent

Per-Family Monthly Program Cost

cost of program use for families assigned to the SUB intervention

Emergency shelter programs had the highest average per-family

compared with usual care at the 20-month followup survey

monthly program cost—about $4,800—of all the types of

resulted from the higher use of permanent housing subsidies

programs examined. Supportive services made up 63 percent

being offset by the decreased use of relatively expensive emer-

of emergency shelter costs, the highest share among the four

gency shelter and transitional housing programs by families

program types considered. PBTH programs had an average

assigned to the SUB intervention. In a similar way, the costs of

cost per family per month of participation of about $2,700,

total program use for families assigned to the SUB and CBRR

with supportive services constituting, on average, 42 percent

interventions were nearly equivalent, because the greater use

of those costs. Permanent subsidy programs cost, on average,

of permanent subsidy programs by SUB families was offset by

about $1,200 per month per participating family. The cost of

the greater use of transitional housing, emergency shelter, and

SUB programs consisted wholly of the cost of housing, because

other programs by CBRR families. At 20 months after random

permanent subsidy programs do not provide supportive

assignment, the cost of all program use for families assigned to

services. CBRR programs had the lowest per-family per-month

the CBRR intervention was clearly less than that for families in

cost, averaging about $900. Housing costs made up, on aver-

either the UC group or the PBTH group.

age, 72 percent of CBRR program costs.

Cost results at 37 months after random assignment appear in

Costs of the individual local programs that made up these

Exhibit ES-8. The exhibit shows that total program use cost

averages varied considerably. PBTH programs and emergency

about $41,000 for families assigned to usual care. The cost of

shelters had the greatest variation, driven largely by variation

all program use during 37 months for families assigned to the

in supportive services costs across local providers but also

SUB intervention was about $3,800 (9 percent) more than for

by variation in capital costs and administrative expenses. For

comparable families assigned usual care. This difference arises

the 24 PBTH programs examined in the cost analysis, average

out of a growing differential between the average monthly costs

per-family monthly program cost ranged from $1,300 to $6,300.

of all program use in the SUB and UC groups. At the time of

Average per-family monthly program cost for the 45 emergency

the 20-month survey, this monthly differential was only $20

shelter programs examined ranged from $1,900 to $9,200.

(the difference between $1,086 for the SUB group and $1,066

for the UC group). At the time of the 37-month survey, the

Variation in rapid re-housing and permanent subsidy costs

monthly differential had reached $136 ($978 for the SUB group

from one local program to another was driven largely by hous-

compared with $842 for the UC group). In other comparisons

ing costs. For the 12 CBRR programs in the cost analysis, the

involving the SUB intervention, families assigned to the SUB

average per-family monthly program cost ranged from $550 to

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxxi

FAMILY OPTIONS STUDY

Executive Summary

Exhibit ES-8. Cost of Program Use Since RA for Each Intervention Contrast

Other

SUB

CBRR

PBTH

ES

Panel A

SUB vs. UC

CBRR vs. UC

PBTH vs. UC

$50,000

$45,902

$45,000

$42,134

$42,167

$40,130

$40,000

$38,722

$38,144

$35,000

$30,000

$25,000

$20,000

$15,000

Cost of program use since RA

$10,000

$5,000

$0

SUB

UC

CBRR

UC

PBTH

UC

N = 501

N = 395

N = 434

N = 434

N = 293

N = 259

Assigned intervention and pairwise comparison

Panel B

SUB vs. CBRR

SUB vs. PBTH

CBRR vs. PBTH

$50,000

$45,668

$44,895

$45,000

$41,743

$40,793

$40,269

$40,000

$35,000

$30,479

$30,000

$25,000

$20,000

$15,000

Cost of program use since RA

$10,000

$5,000

$0

SUB

CBRR

SUB

PBTH

CBRR

PBTH

N = 362

N = 290

N = 215

N = 201

N = 180

N = 184

Assigned intervention and pairwise comparison

CBRR = priority access to community-based rapid re-housing. ES = emergency shelter. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

Notes: Averages are for all 37-month survey respondents in each pairwise comparison and are weighted for survey nonresponse to represent full comparison sample. Cost estimates assume a site-specific average cost per month based on the Family Options Study cost data and HUD administrative data. The other category refers to other permanent housing subsidies and includes permanent supportive housing, public housing, and project-based assistance (project-based vouchers or Section 8 projects).

Sources: Family Options Study cost data; U.S. Department of Housing and Urban Development, Public and Indian Housing Information Center, Tenant Rental Assistance Certification System, and Financial Data Schedule records (SUB); Family Options Study Program Usage Data group have about 9 percent higher average costs during the

In other comparisons involving the PBTH group, the high monthly

first 37 months than families assigned to the CBRR group and

cost of transitional housing programs results in a higher aver-

10 percent higher than families assigned to the PBTH group.

age cost of all programs used for families assigned to the PBTH

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxxii

FAMILY OPTIONS STUDY

Executive Summary

group compared with families assigned to either the UC or

group more than halved most forms of residential instability,

CBRR group. On the other hand, in each of the three compari-

improved multiple measures of adult and child well-being, and

sons involving the CBRR intervention, families assigned to the

reduced food insecurity.

CBRR group have the lowest average cost for all programs used.

The 3-year evidence shows that families randomly assigned to

The largest difference is for the CBRR group compared with the

the CBRR group do about as well as families assigned to the UC

PBTH group, in which the high monthly cost and greater use

group but at substantially lower cost, mainly because assign-

of transitional housing programs for the latter group results in

ment to the CBRR group lowers the rate at which families use

a nearly $10,000 difference in average cost of all program use

costly transitional housing programs. Assignment to the PBTH

for comparable families. For CBRR-versus-UC and SUB-versus-

group has few advantages over other types of assistance. In

CBRR, CBRR families have an average cost of all programs

addition, the study does not provide appreciable evidence that

used that is $4,000 (roughly 10 percent) less than the cost of

intervention impacts differ according to families’ psychosocial

program use for those assigned to the other interventions.

challenges or housing barriers at baseline, whatever form of

active assistance is prioritized.

Conclusions

The 3-year findings lend support for the underlying theoretical

The Family Options Study’s random assignment design for

model for permanent housing subsidies. The striking impacts

measuring intervention impacts is a stronger design than that of

of assignment to the SUB group in reducing subsequent stays

other studies of programs for homeless families. Evidence from

in shelter and places not meant for human habitation provide

the study’s 3-year analysis provides important new information

support for the view that, for most families, homelessness is a

about what happens to families who experience homelessness

housing affordability problem that can be remedied with per-

in the absence of any special offers of assistance. It also

manent housing subsidies without specialized homeless-specific

provides information about the impact of assignment to three

psychosocial services. The findings also provide support for

active interventions: SUB, CBRR, and PBTH.

the more tentative theoretical proposition that resolving home-

lessness would have a radiating impact, given the short-term

A clear finding from the study is that homelessness is expensive

impacts found for assignment to the SUB intervention on family

for families and communities. Even without priority access

preservation, adult well-being, and school stability compared

to assistance, families in 12 communities used housing and

with the impacts of usual care and the continuing effects on

services programs costing about $41,000, on average, during

adult and child well-being and self-sufficiency after 3 years.

a period of a little more than 3 years. Despite this considerable

public (and in some cases private) investment, many families

The short-term findings provided less support for the theoret-

who had been in shelter for at least a week at the outset of

ical model underlying the PBTH intervention. PBTH programs

the study were still not faring well 3 years later. Well over

are intended to address the root causes of homelessness by

one-third had been homeless or doubled up recently, nearly

providing social services packaged with housing assistance. The

one-half were food insecure, and incomes averaged less than

short-term analysis did not provide evidence that assignment

two-thirds of the poverty threshold. The high cost of homeless

to the PBTH intervention achieved that goal. Assignment to the

services suggests that prevention efforts with low per-family

PBTH intervention led to modest reductions in homelessness

costs—if they were effective—would not need to be tightly

when compared with usual care during the period that some

targeted to just the families who would otherwise experience

families were still in transitional housing, but effects on hous-

homelessness in order to save resources.

ing stability did not last until 3 years, and assignment to the

PBTH intervention did not produce effects in other aspects of

The longer-term evidence from the Family Options Study

family well-being during the longer term.

indicates that having priority access to deep permanent housing

subsidies produces substantial benefits for families. More than

The Family Options Study suggests that families who expe-

one-third of families assigned to any intervention found their

rience homelessness can successfully use and retain housing

way to permanent housing subsidies, but families given priority

vouchers, and that having priority access to deep permanent

access to that assistance obtained subsidies sooner. Providing

housing subsidies has considerable benefits at some additional

priority access to permanent housing subsidies cost 9 percent

cost. The homeless assistance system does not currently

more than not giving families any priority offer during a 3-year

provide immediate access to such subsidies for most families in

followup period, and it suppressed work effort by about 6 per-

shelter, although more than one-third of families without pri-

centage points during the second half of that period; however,

ority access nevertheless obtained permanent housing subsidies

it had substantial benefits. Assignment to the SUB intervention

during a 3-year followup period.

3-Year Impacts of Housing and Services Interventions for Homeless Families

xxxiii

FAMILY OPTIONS STUDY

CHAPTER 1.

INTRODUCTION

As part of its mission, the U.S. Department of Housing To be specific, the study randomly assigned a sample of 2,282

and Urban Development (HUD) has supported a

families in 12 sites to one of three active interventions or to usual

range of programs to provide shelter and services for

care. The Family Options Study: Short-Term Impacts of Housing

families experiencing homelessness. The Department has also

and Services Interventions for Homeless Families (hereafter, the

engaged in partnerships with other federal agencies to focus

Short-Term Impacts report; Gubits et al., 2015) presented 20-month

resources on eradicating homelessness. Opening Doors: Federal

impact estimates, documenting how families were faring roughly

Strategic Plan to Prevent and End Homelessness, released in 2010

1 1/2 years after random assignment. This report presents esti-

by the U.S. Interagency Council on Homelessness, articulates

mates of longer-term impacts at the point of 3 years after random

this collective commitment and lists four goals, one of which

assignment to the interventions studied.9 This report also

is to “prevent and end homelessness for families, youth, and

presents information about the relative costs of the three active

children by 2020” (USICH, 2010, 2015).

interventions and emergency shelter. The analysis presented in

this report uses survey and administrative data for the sample

During a 12-month period ending September 2015, nearly

of 1,784 families and 2,665 children from the 12 sites who

155,000 families with children in the United States (502,521

responded to a followup survey approximately 37 months after

people) stayed in emergency shelters or transitional housing

random assignment (henceforth referred to as the 37-month

programs (HUD, 2015).8 Although most homeless individuals

followup survey).

are single men, about one-third (34 percent) of the sheltered

homeless population are members of families (12-month estimate).

The balance of this chapter characterizes study families and

interventions, describes the design of the study, summarizes

To develop evidence on which to base policy decisions, HUD

findings from the enrollment phase, and recaps the short-term

launched the Family Options Study in 2008, awarding a

impacts of the interventions for families in the first 20 months

contract to Abt Associates, Vanderbilt University, and several

after random assignment. The chapter closes with a discussion

other partners. The Department intends the results of this study

of the questions that motivate this longer-term analysis.

to support its efforts to help families leave homelessness and to

create housing stability and other positive outcomes for families

who have experienced homelessness, including family preser-

1.1. The Research Sample

vation, adult well-being, child well-being, and self-sufficiency.

The study team recruited 12 sites (displayed in Exhibit 1-1) to

The Family Options Study examines the effects of three types

conduct the study. The 12 study sites represent a diverse range

of programs—permanent housing subsidies, community-based

of geographic locations, size, population, and housing and

rapid re-housing, and project-based transitional housing—

labor market characteristics. Although not a randomly selected

compared with one another and with the usual care available

sample of communities, the sites are varied in geography and

to homeless families. The three types of programs are distin-

conditions that are related to homelessness. The sites are located

guished from one another by the duration of housing assistance

in all four of the Census Bureau-designated regions in the country.10

and the type and intensity of social services provided to

families. Usual care consists of emergency shelter and housing

At study entry, characteristics of the 2,282 families who enrolled

or services that families can access without immediate referral

in the Family Options Study were similar to characteristics

to a program that would provide them with a place to live.

of families who experience homelessness nationwide. This

8 AHAR 2015 Part 2, 2015.

9 Gubits et al. (2015) report the short-term impacts of the Family Options Study in Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015). Two other reports provide information about the Family Options Study: the Revised Data Collection and Analysis Plan: Family Options Study (Gubits et al., 2012), which describes the research design and analysis plan, and the Interim Report: Family Options Study (Gubits et al., 2013), which documents study implementation findings and baseline characteristics of the research sample.

10 For more detail on the study sites and baseline characteristics of the research sample, see the Interim Report: Family Options Study (Gubits et al., 2013).

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Chapter 1. Introduction

Exhibit 1-1. Location of Study Sites

Minneapolis, MN

Boston, MA

Salt Lake City, UT

Bridgeport and

Denver, CO

New Haven, CT

Alameda County, CA

Kansas City, MO

Baltimore, MD

Louisville, KY

Phoenix, AZ

Atlanta, GA

Honolulu, HI

observation is true even though the sites were not a randomly

A plurality of families (43 percent) had only one child with

selected sample of communities. This section briefly summariz-

them in the shelter, and one-half of the families were with a

es the study sample (Exhibit 1-2). Additional details about the

child younger than age 3. Most families in the study (79 per-

baseline characteristics of the research sample are provided in

cent) were not homeless immediately before entering the shel-

the Interim Report: Family Options Study (hereafter, the Interim

ter from which they were recruited into the study. About 63

Report; Gubits et al., 2013) and the Short-Term Impacts report

percent of family heads in the study, however, had experienced

(Gubits et al., 2015).

homelessness at some other point in their lifetime, with 16

percent of adult respondents having experienced homelessness

The typical family in the study consisted of an adult woman,

as a child. An even greater proportion (85 percent) indicated

who was a median of 29 years old and lived with one or two

they lived doubled up at some point as an adult, defined in the

of her children in an emergency shelter. In about one-fourth

survey as “staying with family or friends because you couldn’t

of the families at least one child was separated from the family

find or afford a place of your own.”

at the time of enrollment and was living with other relatives,

with friends, or in foster care. At baseline, 30 percent of families

Most family heads were not working at the time of random

had more than one adult present. Nearly all families who had

assignment (83 percent), and more than one-half had not

two adults present were headed by couples, and 10 percent

worked for pay in the previous 6 months. The median annual

of families had a partner living elsewhere. As in other studies

household income of all families in the study at baseline was

of homelessness among families, members of minority groups

$7,410. Many reported they either had a poor rental history

were overrepresented, even in proportion to the poverty

(26 percent had been evicted) or had never been a leaseholder

population.

(35 percent). Nearly one in three reported either post-traumatic

stress disorder or serious psychological distress.

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Chapter 1. Introduction

Exhibit 1-2. Family Characteristics

Family Characteristic

Percent of Adult Respondents/ Percent of Families/Years

Family composition

Adults

Adult respondent is female

91.8

Median age of adult respondent

29.0 years

Children

1 child present in shelter

43.2

2 children present in shelter

30.2

3 children present in shelter

15.3

4 or more children present in shelter

11.2

At least 1 child present in shelter younger than age 3

50.4

Mother is pregnant

9.8

At least 1 child not present in shelter

23.9

Housing instability and history of homelessness

Homelessness history

Previous episode of homelessness

62.9

Total homelessness in life

Median: 6 months

Doubled up history

Ever doubled up as adult because could not pay rent

84.7

Time doubled up in past 5 yearsa

Median: 1 year

Income stability and disability

Employment history of adult respondentsb

No work past 1 week

82.9

No work past 6 months

57.1

No work past 1 year

45.0

No work past 2 years

30.3

Family income during the past year

20th percentile

$2,880

50th percentile (median)

$7,410

80th percentile

$15,000

Barriers to increasing income or finding housing

Exposure to violence and mental health

Domestic violence by spouse or partner as an adult

49.0

PTSD symptoms

21.6

Psychological distress

22.1

Previous housing history—problems finding housing

History of evictionc

25.9 big or small problem

Never a leaseholderc

34.8 big or small problem

PTSD = post-traumatic stress disorder.

a Time doubled up in past 5 years or time doubled up since age 18 for respondents ages 18 to 22 years.

b Rows are not mutually exclusive.

c Information was collected on history of eviction and never having been a leaseholder only if the respondent thought these factors presented a problem in finding a place to live.

Note: Sample size = 2,282.

Source: Family Options Study baseline survey

1.2. Interventions Studied and

market housing. Families did not receive any other dedicated

Rationales

services but were free to obtain whatever social services

might be available in their communities. The rationale for

The Family Options Study compares three active interventions

permanent housing subsidies is that homelessness is an eco-

with one another and with usual care in the communities

nomic problem that subsidies can address. Once stabilized

where families were recruited:

in housing, families can address other nonhousing needs,

such as employment services or connections to benefits,

1. Assignment to the SUB intervention provided priority access

with community-based providers. Families could continue

to a permanent housing subsidy, usually a housing choice

receiving housing assistance as long as they remained eligible

voucher that paid the difference between 30 percent of fami-

and followed program rules, such as paying their share of

lies’ incomes and their housing costs in conventional private

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Chapter 1. Introduction

the rent and living in housing that passes a housing-quality

average ratio of 20 families to each case manager, almost

inspection. Families in homeless shelters do not ordinarily

twice the intensity of community-based rapid re-housing),

have immediate access to housing choice vouchers, which

but services could also be provided by other agency staff

are not typically part of the homeless assistance system.

or through dedicated linkages with other agencies.

2. Assignment to the CBRR intervention provided priority

4. Assignment to the UC intervention provided no priority

access to temporary rental assistance to help families find

access to programs but consisted of whatever housing or

and rent conventional, private-market housing, paired with

services a family accessed in the absence of immediate

limited services focused on housing and self-sufficiency.

referral to the programs offered to families assigned to the

Rental assistance, which had to be renewed every 3 months

other interventions. Because all families were recruited

and could last up to 18 months (median stay for study

from emergency shelter, usual care typically consisted

families was 8 months), was structured differently in

of continued stays in the emergency shelter from which

different communities. All programs assessed family needs

families were enrolled, until families were able to make other

at the outset and developed formal services plans focused

arrangements on their own or with the assistance of service

on housing and self-sufficiency. Case managers adjusted

providers. Families in shelters also received case manage-

these services plans based on quarterly reassessments

ment and services similar to those received by families in the

conducted to determine whether to renew the rental

PBTH group (with an average ratio of 16 families per case

assistance. Case management ratios varied, but averaged

manager, although this varied considerably from shelter to

36 families per case manager, with most families meeting

shelter). Families in the UC group found their way into a

with the case manager monthly. Although rapid re-housing

variety of housing and services programs (see Chapter 2).

was an emerging practice at the time families enrolled in

For more information about the interventions studied in the

the study, the programs studied were consistent with the

Family Options Study see Chapter 2 and Appendix B in the

core components in guidelines HUD issued in 2012.11 The

Interim Report (Gubits et al., 2013), Chapters 6 to 8 in the

rationale for community-based rapid re-housing is that,

Short-Term Impacts report (Gubits et al., 2015), and short

although homelessness is largely an economic problem,

papers on the SUB and CBRR interventions.13

the role of the homeless assistance system should be to get

families into conventional housing as rapidly as possible

with the minimum intervention necessary to prevent returns

1.3. Evaluation Design

to shelter.

In each study site, the study team recruited families who had

3. Assignment to the PBTH intervention provided priority

stayed in emergency homeless shelters for at least 7 days. The

access to temporary housing for up to 24 months (the

study team excluded families who left shelter in fewer than

median stay for study families lasted 13 months), coupled

7 days, because the more intensive interventions considered in

with more intensive social services in supervised programs.12

this study may not be necessary for families who can resolve a

The rationale for project-based transitional housing is

housing crisis quickly.

that families who experience homelessness have a variety

of additional challenges (mental health or substance

In the original design of the study, each family was to have a

issues, lack of job experience or skills, domestic violence,

chance of being assigned to all four groups (SUB, CBRR, PBTH,

parenting challenges) that must be addressed before families

and UC). A number of factors prevented the study from being

can succeed in independent housing. Families received

implemented as planned. First, all four intervention were

comprehensive assessments at the outset of the program

offered in only nine sites. Two sites (Atlanta and Baltimore)

and most programs provided housing placement assistance,

did not offer the SUB intervention, and one site (Boston) did

self- sufficiency services, employment or training assistance,

not offer the PBTH intervention. Second, interventions were

life skills, mental and physical health care, parenting,

available to families only in cases when at least one provider of

child advocacy and substance abuse services, as needed.

the intervention type had an available slot. Third, some service

Most services were provided by case managers (with an

providers had unique eligibility requirements for families.

11 See http://usich.gov/population/families/core-components-rrh/. See also Wood (forthcoming).

12 To distinguish transitional housing from community-based rapid re-housing, the Family Options Study examined project-based transitional housing (PBTH), rather than scattered-site transitional housing programs that sometimes allow families to assume the lease or “transition in place” at the end of the program. Some PBTH programs included in the study provided units in the community (called “scattered site” units) without the opportunity to transition in place when assistance ended.

13 Solari and Khadduri (forthcoming), Wood (forthcoming).

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Chapter 1. Introduction

Before random assignment, the study team screened families

preexisting differences among the families. Following random

for eligibility for the providers that had available slots based

assignment, the study team analyzed the baseline characteristics

on criteria specified by the providers. The purpose of this

of the sample to confirm that the different sets of families as-

screening was to minimize the likelihood of assigning families

signed to each intervention were indeed equivalent. The Interim

to interventions they would not be eligible to receive.14 As a

Report (Gubits et al., 2013) describes these analyses.

result, for an intervention option to be available to a family

Assignment to an intervention meant that families were given

undergoing random assignment, at least one slot had to be

priority access to specific programs that had a place reserved

open at an intervention provider for which the family met

for them and expected them to enroll. Families were not

provider-specific eligibility requirements. All families were

required to enroll in the programs. If families did not take up

eligible for usual care by definition.

the offered program, they stayed at the shelter from which they

Families were randomly assigned among available interventions

were enrolled until they were able to make other arrangements,

and all analyses compare families who were eligible for both

either on their own or with the assistance of social service

interventions in a comparison and randomized to one of them.

providers at the shelter or elsewhere. Families assigned to each

Thus, for example, all families who had no PBTH program

intervention actually used a variety of programs, but they were

available at the time of random assignment in their site or who

more likely to use a program when they got a priority offer to

were ineligible for all available PBTH programs were excluded

it than when they did not, as described in the analysis of the

from comparisons involving the PBTH intervention and either

pairwise comparisons in Chapters 3 through 6. Exhibit 1-3

usual care or one of the other interventions. This random

illustrates the six pairwise comparisons. To investigate the im-

assignment design assures that comparisons of interventions

pact of offers of priority access to programs, the impact analysis

involve well-matched groups and that any (statistically

includes all families who received priority offers, regardless of

significant) observed differences in outcomes can be attributed

whether they used them.15

to the differential assignment families received and not to any

Exhibit 1-3. Six Pairwise Comparisons Among the Four Interventions

14 See the Interim Report (Gubits et al., 2013) for a detailed description of the eligibility screening conducted before random assignment.

15 See Appendix C for details of the methods used to estimate impacts.

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Chapter 1. Introduction

1.4. Findings From the Enrollment

access to community-based rapid re-housing and project-based

Phase

transitional housing and with usual care reduced homelessness,

doubling up, and residential mobility across multiple measures.

The Interim Report (Gubits et al., 2013) concluded that home-

Impacts of assignment to the SUB intervention radiated beyond

less assistance programs in the study communities imposed

housing. Compared with assignment to the UC group, assign-

eligibility criteria that hampered their ability to serve families

ment to the SUB group reduced the number of separations

in shelter who needed the assistance. Even when programs had

of children from parents and adult drug abuse, psychological

space available, the programs often screened out families in

distress, and domestic violence, and it improved food security.

shelter based on eligibility criteria such as insufficient income,

Assignment to the SUB group also reduced the number of schools

substance abuse, criminal histories, and other factors that pre-

children attended and their absences from school, although

sumably contributed to the families’ homelessness. Screening

effects on other child outcomes were more sparse. The salutary

out was most common for PBTH programs and least common

effects of assignment to the SUB intervention extended across

for SUB programs. Moreover, families who are homeless do

all five of the outcome domains when compared with assign-

not always choose to pursue the programs offered to them,

ment to the UC group; however, heads of these families exerted

which suggests that some programs deliver assistance that some

less work effort compared with families assigned to the UC

families perceive as less valuable to them than other alternatives

group. In the week before the 20-month survey, 30 percent of

available in their communities.

UC families were working for pay compared with 24 percent of

families assigned to the SUB group.

1.5. Short-Term Findings (20 Months The striking impacts of assignment to the SUB intervention in After Enrollment)

reducing subsequent stays in shelter and places not meant for

human habitation provided support for the view that, for most

The Short-Term Impacts report (Gubits et al., 2015) examined

families, homelessness is a housing affordability problem that

20-month impacts of intervention assignment for 73 outcomes

can be remedied with permanent housing subsidies without

in five domains: (1) housing stability, (2) family preservation,

specialized homeless-specific psychosocial services. The

(3) adult well-being, (4) child well-being, and (5) self-sufficiency.

findings also provided support for the theoretical proposition

that resolving homelessness would have a radiating impact,

In the absence of priority access to a specific program, families

given the short-term impacts found for assignment to the SUB

assigned to the UC group were not faring well 20 months after

intervention on family preservation, adult well-being, school

enrollment in the study. One-half had spent at least 1 night

stability, and food security compared with usual care.

homeless or doubled up in the 6 months before the 20-month

survey16 or had been in shelter in the past 12 months. In the

Considering costs through 20 months (all that was possible as

6 months before the survey, 15 percent of families had been

of that analysis), the benefits of having priority access to perma-

separated from a child who was with the family in shelter at

nent housing subsidies were achieved at comparable cost with

study outset, and 4 percent had children in foster care. At the

that of usual care, slightly higher costs than priority access to

time of the survey, 15 percent reported alcohol dependence

community-based rapid re-housing, and at substantially lower

or substance abuse, and 12 percent had experienced intimate

cost than priority access to project-based transitional housing.

partner violence in the past 6 months. Children had attended

two schools, on average, in the past 20 months and were

CBRR families had housing outcomes similar to those of

absent, on average, about 1 day per month. Less than one-third

families assigned to the UC group. Relative to families assigned

of UC families (31 percent) were working for pay, and total

to the UC group, however, the cost of all program use for

family income was only $9,067 per year—much less than what

families assigned to the CBRR group was lower than for families

is needed to rent market-rate housing for a family in most

assigned to the UC group, mainly because assignment to the

communities.

CBRR intervention lowered the rate at which families used cost-

ly transitional housing programs in the first 20 months after

Relative to assignment to the UC group, assignment to the SUB

random assignment. At the time of the short-term analysis, the

group caused striking improvements in families’ outcomes in

three major advantages of assignment to the CBRR intervention

the first 20 months after random assignment. Having priority ac-

over other interventions were (1) the comparatively low cost of

cess to permanent housing subsidies—compared with priority

the CBRR intervention, (2) the greater income observed among

16 Gubits et al. (2015) analyzed short-term impacts of the interventions. The study team attempted to contact families for the study’s first followup survey beginning in the 18th month after random assignment. The median time from random assignment to the followup survey was 20 months. The followup period reported in Gubits et al.

(2015) is thus 20 months, but the followup survey is sometimes referred to as the 18-month survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

Chapter 1. Introduction

families assigned to the CBRR group compared with income

temporary intervention by the time of the 20-month followup

among families assigned to the UC group, and (3) greater work

analysis, and anxieties about the impending end of a program

effort compared with families assigned to the SUB group.17

or disruption from having recently moved at the time of the

20-month survey could have depressed families’ outcomes in

Assignment to the PBTH group led to modest reductions in

the CBRR and PBTH interventions. To the extent that assign-

homelessness when compared with assignment to the UC

ment to interventions strengthen families or set the foundation

group (during a period when many families were still in the

for later success, as theorized by proponents of project-based

PBTH programs to which they received priority access), but it

transitional housing, new findings may emerge at 3 years. In

did not produce effects in other aspects of family well-being.

a similar way, increases in incomes observed at 20 months

The short-term findings provided little support for the theo-

for CBRR families may set families on a positive trajectory of

retical model that project-based transitional housing would

sustained benefit from that intervention. On the other hand, to

produce better results by addressing families’ psychosocial

the extent that the reduction in homelessness seen for PBTH

issues in supervised settings. Finally, although the statistical

families at 20 months was a temporary consequence of still

power for these tests was low, the short-term results showed

being in PBTH programs at that time, impacts may fade. This

no evidence that intervention impacts differed according to

3-year analysis enables the study team to examine outcomes

families’ psychosocial challenges or housing barriers whatever

well after families reach the time limits for these temporary

type of programs were offered to families. Overall, considering

programs. Longer-term analysis is also important to measure

the cost of all program use in the first 20 months for families

impacts that may take longer than 20 months to emerge, such

assigned to the PBTH group, the PBTH intervention was more

as those on child well-being outcomes.

costly and, at the 20-month followup point, had few advantag-

es over other programs.

The Short-Term Impacts report (Gubits et al., 2015) found that

families assigned to all interventions used a variety of programs.

1.6. Questions for the 3-Year

The 3-year analysis updates information about the ways that

having priority access to particular interventions affected pat-

Analysis

terns of use. In the case of permanent housing subsidies, which

The 3-year analysis addresses one primary question about

can last as long as families comply with program requirements,

program use and two primary questions about impacts:

it was not clear at 20 months whether families would success-

fully renew leases and sustain tenancies. The 3-year analysis

1. What programs do families who experience homelessness

addresses whether families assigned to the SUB group are

use during a 3-year period, and how does assignment to an

able to retain the assistance. Emergency shelters, transitional

intervention that offers priority access to a particular kind of

housing programs, and rapid re-housing programs frequently

program affect this program use?

attempt to enroll their families on waiting lists for permanent

housing. Longer-term followup can also show whether these

2. At 3 years after random assignment, what are the relative

programs serve as way stations to more permanent housing

effects of the three active interventions compared with usual

subsidies.

care and of the active interventions compared with each

other?

Finally, the Short-Term Impacts report (Gubits et al., 2015)

found that that, after 20 months, the cost of all the programs

3. What are the cumulative costs of the interventions during

used by families assigned to the SUB group was about the same

the 3 years following random assignment?

as for families assigned to the UC group. Because the subsidies

Longer-term followup is often desirable in assessing the effects

tested in the SUB intervention are permanent, whereas the CBRR

of social policy interventions and is especially so in the case of

and PBTH interventions are time limited, there is good reason

the Family Options Study. To evaluate the effects of temporary

to expect that the relative costs of interventions may change

assistance that can last up to 18 months (CBRR programs) or

when analyzed over a longer followup period. The current

24 months (PBTH programs), 20 months is not a long enough

report examines these costs cumulatively for a full 3 years after

period. Some families may not have received a full dose of a

random assignment.

17 The Short-Term Impacts report (Gubits et al., 2015) also presented results showing that assignment to the CBRR group led to more rapid departures from emergency shelter than assignment to the UC group by about 2 weeks, but those departures were not more rapid than for families assigned to the SUB group or to the PBTH group.

Revised analysis using updated Program Usage Data on length of emergency shelter stays, however, has resulted in changes to the findings about length of initial shelter stay. For the full study sample (not limited to 20-month survey respondents), families assigned to the CBRR group left shelter, on average, 1 week faster than families assigned to the UC group. This difference is not statistically significant. The results of the revised analysis are in Appendix I.

3-Year Impacts of Housing and Services Interventions for Homeless Families

7

FAMILY OPTIONS STUDY

Chapter 1. Introduction

In sum, the analyses presented in the report give evidence

presents findings from the comparison of assignment to the

about the relative impacts and costs of the interventions 3 years

CBRR intervention with usual care in the five domains, and

after random assignment. The report describes what housing

Chapter 5 does so for the comparison of the PBTH intervention

and services interventions families in the 12 communities en-

with usual care. Chapter 6 turns to the other pairwise compari-

gage in when they receive no special priority offer of assistance,

sons of the three active interventions, reporting impacts of SUB

and how those UC families are faring 3 years after random as-

compared with CBRR, SUB compared with PBTH, and CBRR

signment. It also examines the relative impacts of priority offers

compared with PBTH. Chapter 7 discusses results about the

of permanent housing subsidies, temporary rapid re-housing

relative impacts of groups of interventions to illuminate other

assistance, and project-based transitional housing compared

policy questions. Chapter 8 explores the variability of impacts

with usual care and with each other at that same time point.

across types of families, using indices related to psychosocial

challenges and housing barriers constructed for each family.

1.7. Organization of the Report

Chapter 9 describes the relative costs of the interventions.

Chapter 10 discusses study conclusions.

The balance of this report is organized as follows. Chapter 2

Several technical appendixes support the report. Appendix A

describes the experiences of the UC group. It also defines the

provides details about the data sources and data set construc-

outcomes derived from participant surveys and administrative

tion. Appendix B discusses the construction of adult and child

data that are used to estimate intervention effects. Chapters 3

well-being outcomes. Appendix C presents technical details re-

through 6 then present findings about the relative impacts of

garding the samples and analysis methods. Survey nonresponse

the four interventions, organized by six pairwise comparisons.

analysis is documented in Appendix D. Appendix E presents

In particular, Chapter 3 provides impact measures for assign-

supplemental tables showing use of transitional housing during

ment to the SUB intervention compared with usual care, for

the followup period. Appendix F presents exhibits showing

the five domains of housing stability, family preservation, adult

the results of the pooled comparisons. Appendix G presents

well-being, child well-being, and self-sufficiency. Chapter 4

technical details about the cost data collection and analysis.

3-Year Impacts of Housing and Services Interventions for Homeless Families

8

FAMILY OPTIONS STUDY

CHAPTER 2.

DESCRIPTION OF USUAL CARE (UC) AND

OUTCOMES MEASURED IN THE STUDY

Evidence from the Family Options Study informs in emergency shelter of at least 7 days, without special referral policymakers not only about the effects of assignment

to one of the study’s active interventions. All families were

to the active interventions—permanent housing

recruited for the study from emergency shelters.18

subsidy (SUB), community-based rapid re-housing (CBRR),

Because families assigned to the UC group were not explicitly

and project-based transitional housing (PBTH)—but also about

provided other assistance, all UC families remained in emer-

the experiences of families assigned to remain in usual care

gency shelter until they navigated their way out or until they

(UC) after 7 days in emergency shelter. The types of homeless

reached the shelters’ length-of-stay limits. Emergency shelter

and housing assistance programs that the UC families use and

staff may have provided some assistance to UC families in leav-

indicators of their well-being 3 years after study enrollment are

ing shelter, but the study team asked shelter staff not to actively

instructive. These findings show how families in the 12 study

help UC families enter SUB, CBRR, or PBTH programs.

communities navigate the housing and homeless assistance

systems and how they fare if they do not receive priority access

Across all 12 study sites, 746 families were randomly assigned

to one of the active interventions.

to the UC group from 56 emergency shelters.19 Of these 746

families, 556 (75 percent) responded to the 37-month followup

This chapter describes the types of programs—both emer-

survey and are therefore included in the impact analysis in this

gency shelters and other housing and homeless assistance

report. We refer to these 556 families as the UC respondent

programs—that UC families used during the followup period

families.

and the timing of their use of those programs. The chapter

also introduces the outcomes examined in the impact analysis

(reported in Chapters 3 through 6) and presents levels of these

2.1.1. Length of Time UC Families Spent in

outcomes for UC families, the benchmarks against which the

Emergency Shelters

impacts of other interventions are measured.

The Program Usage Data collected for the study from various

sources indicate the time that study families spent in emergency

2.1. The Emergency Shelter Experi-

shelter during the followup period.20 These data show that UC

ence of Usual Care (UC) Families

respondent families stayed in emergency shelter for a mean

of 4 months and a median of 3 months during the followup

Usual care for this study consisted of whatever program services

period.21 Exhibit 2-1 shows the distribution of total month

UC families were able to access on their own following a stay

durations in emergency shelter across all shelter stays during

18 A description of the UC program environment, including profiles of the housing and supportive services provided by the emergency shelters from which study participants were drawn, is provided in Chapter 5 of the Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015).

19 One family was enrolled into the study from a 57th shelter, but this family was not assigned to the UC group.

20 See Appendix A for details of how the sources of program usage were combined into a single data set. Missing data on emergency shelter stays bias the counts of total time spent in emergency shelter somewhat downward. The baseline stay in emergency shelter does not appear in the Program Usage Data (largely based on data from the Homeless Management Information Systems) for 16.7 percent of UC respondent families.

21 The mean and median are computed with weights to adjust for survey nonresponse, so that the respondent families represent all 746 families assigned to the UC group.

The length of the followup period is from the month of random assignment to the month of the followup survey response (median 38 calendar months included in followup period). Most families had only one spell in emergency shelter, wherein spells are separated by at least 1 calendar month with no emergency shelter stay. The weighted percentage of families with zero spells (that is, missing data on the spell when they were recruited from shelter at baseline and also no subsequent spell recorded in the Homeless Management Information System Program Usage Data) is 10.2 percent, with one spell is 63.7 percent, with two spells is 18.7 percent, with three spells is 5.0

percent, and with four or more spells is 2.4 percent.

3-Year Impacts of Housing and Services Interventions for Homeless Families

9

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-1. Length of Time Spent in Emergency Shelters

the followup period. About 60 percent of UC families stayed in

by UC Families

emergency shelter for fewer than 3 months during the followup

Number of Months

period. Another 19 percent stayed between 3 and 6 months

(with at least 1 night stay)

Percent of

during the followup period, and 21 percent stayed 6 or more

in Emergency Shelter

UC Respondent Familiesb

During Followup Period

(N = 556)

months. Only 2 percent stayed in emergency shelter for 18 or

a

0.00c

11.1

more months.

Less than 1 month

18.9

1 to 1.99

15.4

Exhibit 2-2 shows the percentage of UC families who have at

2 to 2.99

14.1

least 1 night in emergency shelter during the month for each

3 to 3.99

8.6

month after random assignment. A second line shows the

4 to 4.99

4.8

percentage of UC families who have not left the initial stay in

5 to 5.99

5.7

shelter each month. The exhibit shows that most families had

6 to 6.99

4.1

7 to 7.99

2.8

left emergency shelter by the 3rd or 4th month after random

8 to 8.99

2.1

assignment. The exhibit shows that only about 17 percent of

9 to 11.99

4.6

UC families are in any emergency shelter by the 6th month

12 to 14.99

3.4

after random assignment. Past this point, the percentage drops

15 to 17.99

2.3

18 to 23.99

0.9

slowly and, by 32 months after random assignment (the longest

24 and above

1.3

period for which the study team has data for all families),

UC = usual care.

about 4 percent of UC families are in emergency shelter. From

a Total time spent in emergency shelter is calculated from entry and exit dates in

Exhibit 2-2, which also shows almost no UC families still in

Homeless Management Information System (HMIS) and survey data. Missing data on

emergency shelter stays bias the counts of total months spent in emergency shelter

their baseline emergency shelter stay by the 32nd month, it is

somewhat downward. The baseline stay in emergency shelter does not appear in the

possible to deduce that almost all the families in shelter in the

Program Usage Data for 16.7 percent of UC respondent families.

b Percentages are weighted for survey nonresponse to represent all 746 UC families.

32nd month after random assignment had returned to shelter

c Of UC respondent families, 11.1 percent do not have any emergency shelter stay (at

after a departure. A third line in the exhibit shows the percent-

baseline or after baseline) in the HMIS Program Usage Data, even though all families

were recruited from emergency shelters. These families are among the 16.7 percent

age of UC families using any form of permanent subsidy each

of UC respondent families whose baseline stay in emergency shelter does not appear in the data. The other 5.6 percent of UC respondent families (16.7–11.1=5.6) whose

month after random assignment. The exhibit illustrates that the

baseline stay in emergency shelter does not appear in the data have at least one stay

proportion of families assigned to the UC group who use shel-

in emergency shelter after baseline and so are included in the rows below the 0.00

months row.

ter declines as the proportion who use any form of permanent

Source: Family Options Study Program Usage Data

Exhibit 2-2. Percent of UC Families With at Least 1-Night Stay in Emergency Shelter During Month, by Number of Months After RA

100

90

80

70

60

50

All UC families in emergency shelter

40

Percent of families

UC families with any form of permanent housing subsidy

30

UC families

who have

20 not left shelter

since RA

10

0 0

5

10

15

20

25

30

35

Month after RA

UC = usual care.

RA = random assignment.

Notes: Percentages are weighted for survey nonresponse to represent all UC families in the study. Missing data on emergency shelter stays bias the percentages somewhat downward. The baseline stay in emergency shelter does not appear in the data for 16.7 percent of UC respondent families. The missing data rate for subsequent stays in emergency shelter is unknown.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

10

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

subsidy increases. By 32 months after random assignment,

• Public housing in places other than Honolulu.

nearly 30 percent of UC families were receiving some form of

• Project-based vouchers or Section 8 projects in places other

permanent subsidy.

than Bridgeport.

2.2. Use of Other Homeless and

• Any form of permanent housing subsidy (that is, any of

Housing Assistance Programs by

the programs that comprise the SUB intervention in this

study, permanent supportive housing, public housing, or

Usual Care (UC) Families

project-based vouchers and Section 8 projects).

For some families assigned to the UC group, emergency shelters

• Emergency shelter.

were the families’ only interaction with the homeless assistance

or housing subsidy system. Other UC families found their way

All families enrolled in the study while staying in emergency

into permanent housing subsidies, rapid re-housing, and transi-

shelter. By the 7th month after random assignment, most UC

tional housing programs on their own. Exhibit 2-3 shows the use

families had departed from emergency shelter. Exhibit 2-3 shows

of homeless and housing programs by UC families during the

information on the proportion of UC families who did not use

followup period. The exhibit shows eight types of programs.

emergency shelter again and did not use any of the other programs

during the entire followup period. For one-fourth of the UC families,

• Subsidy (that is, the programs comprising the SUB

the initial stay in emergency shelter is the only use of housing

intervention in this study: housing choice vouchers; public

or homeless assistance observed during the followup period.

housing in Honolulu, Hawaii; and project-based vouchers in

Bridgeport, Connecticut).

The first column of Exhibit 2-3 shows the percentage of

UC families who ever used a type of program between the

• Rapid re-housing (that is, programs offered to families

month of random assignment and the month of the followup

assigned to the CBRR intervention).

survey response. This column shows that, during this entire

period, 20 percent of UC families received rapid re-housing

• Transitional housing (both non-“transition-in-place” and

assistance (at least once) and 30 percent of UC families received

“transition-in-place”).

transitional housing (at least once). Altogether, more than

• Permanent supportive housing.

one-third (37 percent) of UC families received some sort of

Exhibit 2-3. Program Use Since RA for UC Families

Percent Ever Used

Number of Months Used From RA to

Percent Used in

From RA to 37-Month 37-Month Followup Survey, if Ever Used

Month of Followup

Type of Homeless or Housing Assistance

Followup Surveya

Type of Housing Assistance

Survey Response

UC

UC

UC

Mean

Median

Permanent housing subsidies offered to the SUB group

11.7

19.0

19.5

9.7

Rapid re-housing

20.4

7.6

6.0

1.5

Transitional housing

29.8

12.0

10.0

4.2

Permanent supportive housing

11.0

16.9

15.5

7.6

Public housing

9.8

18.3

17.5

7.6

Project-based vouchers or Section 8 projects

6.5

18.9

17.5

5.2

Any permanent housing subsidyc

36.6

19.4

19.5

29.9

Emergency shelterd

89.8

4.4

2.8

5.2

No use of homeless or housing programse

25.3

—

—

60.0

N

556

—

—

556

UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of the 37-month followup survey response (median period duration: 37 months). Percentages do not add to 100 because some families use more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at random assignment. The percentage less than 100 is because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

11

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

permanent subsidy during the followup period—accessing the

assistance shown in the exhibit. Nearly one-third of the families

subsidy, public housing, permanent supportive housing, or

(30 percent) were receiving some form of permanent subsidy,

project-based housing assistance programs.

5 percent were using emergency shelter assistance, 4 percent

were using transitional housing, and 2 percent were using rapid

The second and third columns of Exhibit 2-3 show the mean

re-housing. Altogether, 82 percent of families who ever used a

and median number of months of program usage for those

form of permanent subsidy were still doing so at the time of the

families who used the program. The number of months of use of

survey. By contrast, 7 percent who ever used rapid re-housing

rapid re-housing (median 6 months) and transitional housing

were still using it at the time of the survey and 14 percent

(median 10 months) during the study period are consistent

who ever used transitional housing assistance were still using

with the expected durations of these program types. Also

the assistance at the time of the survey. Thus, most families

consistent with expectations are higher median use for all types

who ever used either rapid re-housing or transitional housing

of permanent housing subsidies compared with other types

assistance were no longer using it at the time of the followup

of programs—20 months for the subsidy intervention defined

survey, whereas most UC families who accessed permanent

for the study, 18 months for public housing and project-based

assistance were still receiving it.

vouchers or Section 8 projects, and 16 months for permanent

supportive housing, during the study period.

Exhibit 2-4 provides a more detailed characterization of the

timing of program use by the families in the UC group. This

The fourth column of Exhibit 2-3 shows the percentage of families

exhibit shows the proportions of families within the UC group

who used a program type in the month of the followup survey

who received different types of assistance during each calendar

response. Although the study team expects that many outcomes

month for the first 32 months after random assignment.22

in the report will be influenced by assistance received at any

As shown here, the UC group used a mix of program types

point during the entire followup period, some outcomes will

coming out of emergency shelter. The UC families’ use of rapid

be most strongly influenced by assistance that is received at the

re-housing had greatly diminished by month 20 (the followup

time of the followup survey response. By the time of the followup

period for the Family Options Study: Short-Term Impacts of

survey, 60 percent of UC families were not receiving any of the

Exhibit 2-4. Program Use of UC Families for 32 Months After RA

100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

Rapid re-housing

20

Transitional housing

Percent of CBRR families using program type in month

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

UC = usual care.

RA = random assignment.

Notes: This exhibit shows program use for all families assigned to UC who responded to the 37-month survey. Complete Program Usage Data for 37-month respondent families are available only through month 32 after random assignment. Families who have more than one type of program use in a calendar month are counted fractionally in each type.

Source: Family Options Study Program Usage Data

22 Month 32 is the latest month for which the study team has data for all the families who responded to the 37-month survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

12

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Housing and Services Interventions for Homeless Families report;

2.3.1. Measures of Housing Stability

Gubits et al. [2015]). Their use of transitional housing con-

The study team defined several outcomes related to home-

tinued past this point, only beginning to decrease in the third

lessness and housing stability and used information from the

year. Use of permanent housing subsidies steadily increased

followup survey and Program Usage Data to measure these

throughout the followup period. Slightly more than 20 percent

indicators. The study team developed eight measures related

of UC families were receiving some form of permanent subsidy

to housing stability and homelessness experienced during the

20 months after random assignment and nearly 30 percent

followup period. The following bullet points list the concepts

were receiving some form of permanent subsidy 32 months

of interest, how the study team operationalized the concept,

after random assignment. Some UC families may have been

during what period it is defined, and the data sources used to

on waiting lists for permanent housing subsidies at the time of

measure the indicator.

random assignment and others may have entered waiting lists

after random assignment.23

• At least 1 night homeless in the past 6 months (percent

of families). This outcome measures the percentage of

2.3. Outcomes for Families Randomly

families who reported having spent at least 1 night in a

shelter or a place not meant for human habitation in the

Assigned to Usual Care (UC)

6 months before the 37-month followup survey.25 This

measure is based on responses to the followup survey.

Having just discussed the patterns of program use for the UC

respondent families during the 37-month followup period, the

• At least 1 night doubled up in the past 6 months (per-

chapter now turns to a discussion of the outcomes examined in

cent of families). This outcome measures the percentage

the Family Options Study impact analysis. This section serves

of families who reported having spent at least 1 night in the

the dual purpose of (1) introducing the outcomes examined in

6 months before the 37-month survey living with a friend

the impact analysis and (2) describing the outcomes of families

or relative because they could not find or afford a place of their

assigned to the UC group who responded to the 37-month

own. This outcome is intended to measure episodes in which

followup survey. The section is organized according to the five

families reported the doubled-up living situations that may

outcome domains: (1) housing stability, (2) family preserva-

precede or follow a stay in emergency shelter. This outcome

tion, (3) adult well-being, (4) child well-being, and (5) self-

is measured from responses to the followup survey.

sufficiency. Each outcome domain first includes the specific

outcomes con sidered (including how they are measured) and

• At least 1 night homeless or doubled up in the past

then presents the outcomes for the UC respondent families.

6 months (percent of families). This outcome measures

Changes in measured outcomes between the study’s earlier

the percentage of study families who reported either of the

20-month followup point and the current 37-month window

previous two outcomes.

occasionally are noted, along with their statistical significance.

• Any stay in emergency shelter in the past 6 months

This information about the status of UC respondent families

(percent of families). This measure is the percentage of

3 years after study entry is important for policymakers. The

families who spent at least 1 night in emergency shelter in

data from the Family Options Study show how families in the

the 6 months before the survey. The measure is based on

12 communities fare after an episode of emergency shelter

the Program Usage Data (see Appendix A) and is primarily

use when they do not receive an immediate offer of assistance

taken from the Homeless Management Information System

through one of the active interventions. Appendix B provides

(HMIS), but it also uses survey data.

additional technical details regarding the construction of the

outcome measures from survey and administrative data.24

23 Note that the exhibit does not indicate the two-way flows of families moving into and out of these program types from month to month. Instead, it reflects only the overall usage level in a given month. Though most programs showed positive participation in all 32 reported months, no individual family necessarily remained in the same program for the entire followup period.

24 The Revised Data Collection and Analysis Plan: Family Options Study (Gubits et al., 2012) provides additional details about the selection of outcome domains. Chapter 3

of the Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015) addresses the theoretical framework for the interventions and hypothesized effects on the outcome domains.

25 See 24 CFR 91.5. HUD defines homelessness as living in a supervised publicly or privately operated shelter designated to provide temporary living arrangement (including congregate shelters, transitional housing, and hotels and motels paid for by charitable organizations or by federal, state, or local government programs for low-income individuals). In the followup survey, the study team asked survey respondents if, during the 6 months before the survey, they had spent at least 1 night staying in shelters, institutions, or places not typically used for sleeping, such as on the street, in a car, in an abandoned building, or in a bus or train station. The survey question excluded stays in transitional housing from the question because project-based transitional housing is one of the study interventions.

3-Year Impacts of Housing and Services Interventions for Homeless Families

13

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

• Any stay in emergency shelter in months 21 to 32 after

or rented by a “boyfriend/girlfriend, fiancé or significant

random assignment (percent of families). This measure

other” is not counted as living in the respondent’s own house

is the percentage of families who spent at least 1 night in

or apartment). This outcome measures the percentage of

emergency shelter during the period 21 to 32 months after

families who reported living in their own house or apart-

random assignment. The measure is based on the Program

ment, either with or without housing assistance.

Usage Data. This time period is the latest 1-year period for

• Living in own house or apartment at the time of the

which data are available for all families.

survey with no housing assistance (percent of families).

• Number of days homeless in the past 6 months. This

This outcome measures the percentage of families who

outcome measures the average number of days spent in

reported living in their own house or apartment at the time

shelters or places not meant for human habitation in the

of the survey and who were not receiving housing assistance.

6 months before the survey. It is measured from survey data.

• Living in own house or apartment at the time of the

• Number of days doubled up in the past 6 months. This

survey with housing assistance (percent of families).

outcome measures the average number of days spent living

This outcome measures the percentage of families who

with friends or relatives in the 6 months before the survey. It

reported living in their own house or apartment at the time

is measured from survey data.

of the survey and who were receiving housing assistance to

help pay the rent.

• Number of days homeless or doubled up in the past

6 months. This outcome sums the previous two outcomes.

• Number of places families lived in the past 6 months.

This outcome measures the number of places the family

Because housing stability is the central outcome domain for the

lived in the 6 months before the survey. The number of

study, the study team designated a small set of impact com-

places families lived is top-coded at six places.

parisons and hypothesis tests related to housing stability as the

confirmatory set.26 For this purpose, the analysts constructed

• Persons per room. This outcome measures housing crowding

a single composite outcome for the 37-month followup point.

using information collected from the adult respondent about

This binary outcome is defined as “at least one return to home-

the number of rooms in the housing unit (not counting

lessness” measured from both the followup survey and Program

kitchens, hallways, and bathrooms) and the number of

Usage Data.

people living in the housing unit. Housing situations with

more than one person per room are considered crowded.

• Confirmatory outcome. At least 1 night spent staying

in a shelter or a place not meant for human habitation or

• Housing is of fair or poor quality (percent of families).

doubled up during the past 6 months at the time of the

This outcome measures the percentage of families reporting

survey (measured from survey data) OR any stay in emer-

that the condition of their housing at the time of the survey

gency shelter in the 12 months before the date of the survey

was fair or poor.27

(measured from Program Usage Data).

2.3.2. Housing Stability of the UC Group

The analysts also used data from the followup survey to con-

struct outcomes pertaining to the type of living arrangements at

Exhibit 2-5 shows the values of these indicators for the UC

the time of the followup survey, the number of places families

group. The exhibit displays the average value of each outcome

lived, and housing quality. All these measures are from the

measured for the 556 families assigned to the UC group who

followup survey.

responded to the 37-month followup survey.28 The UC group

displays substantial housing instability during the followup

• Living in own house or apartment at the time of the

period. Based on responses to the survey, 18 percent of UC

survey (percent of families). Survey respondents are con-

families reported having been homeless in the 6 months before

sidered to have independent housing if they rented or owned

the survey, with homeless defined as spending at least 1 night

their own housing at the time of the survey. (Housing owned

in shelter or in places not meant for human habitation. Based

26 The motivation for designating a confirmatory set of outcomes is based on considerations of multiple comparisons; that is, the problem of interpreting individual statistical tests when a large number of tests are conducted. See Appendix C for a discussion of the study’s approach to the multiple comparisons problem, the role of the confirmatory outcome in this approach, and the details of the adjustment procedure.

27 The housing-quality outcome is measured with self-reported assessments of housing condition. This outcome should not be interpreted as representing housing quality as determined by third-party inspections, such as those conducted as part of HUD’s Housing Quality Standards process.

28 Outcome values are weighted for survey nonresponse so the responses represent all families randomly assigned to the UC group.

3-Year Impacts of Housing and Services Interventions for Homeless Families

14

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-5. Housing Stability Outcomes for UC Families

UC Group

Outcome

Mean Value

(Standard Deviation)

Homeless or doubled up during followup period

At least 1 night homelessa or doubled up in past 6 months or in shelter in past 12 months (%) 38.6

(56.5)

At least 1 night homeless or doubled up in past 6 months (%)

34.9

(55.2)

At least 1 night homeless in past 6 months (%)

18.1

(44.7)

At least 1 night doubled up in past 6 months (%)

28.5

(52.3)

Any stay in emergency shelter in past 6 months (%)

8.6

(32.6)

Any stay in emergency shelter in months 21 to 32 after RA (%)

17.3

(43.8)

Number of days homeless or doubled up in past 6 months

47.7

(86.1)

Number of days homeless in past 6 months

18.5

(55.8)

Number of days doubled up in past 6 months

33.3

(73.0)

Housing independence

Living in own house or apartment at followup (%)

69.0

(53.6)

Living in own house or apartment with no housing assistance (%)

41.1

(57.1)

Living in own house or apartment with housing assistance (%)

27.9

(52.0)

Number of places lived

Number of places lived in past 6 monthsb

1.6

(1.2)

Housing quality

Persons per room

1.6

(1.3)

Housing quality is poor or fair (%)

31.5

(53.8)

UC = usual care.

RA = random assignment.

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

b The number of places lived in the past 6 months is topcoded at 6 places.

Notes: N = 556. See Appendix B for details on outcome specifications and values. Means and standard deviations are weighted to adjust for survey nonresponse.

Sources: Family Options Study 37-month followup survey; Program Usage Data for three measures including emergency shelter on Program Usage Data, 9 percent of UC families had stayed in

in emergency shelter in the 12 months before the survey. Tak-

emergency shelter at some point during the same 6-month time

en together, these indicators show that families assigned to the

period. Looking at a longer time period, about 17 percent of

UC group were experiencing a substantial degree of housing

UC families had stayed in emergency shelter during months 21

instability 37 months after study enrollment.

to 32 after random assignment.29

Exhibit 2-6 shows the mean values for three key outcomes for

UC families also reported other experiences that indicate

the UC respondents to the 20-month and 37-month surveys.

housing instability 37 months after random assignment. More

The proportion of UC families who reported being homeless in

than one-fourth of UC families (28 percent) said they spent at

the past 6 months declined by a statistically significant amount,

least 1 night living with friends or relatives because they could

from 24 percent to 18 percent. The proportion who reported

not find or pay for housing in the 6 months before the survey.

being doubled up in the past 6 months did not change signifi-

More than one-third (35 percent) of UC families reported

cantly between the two time points; it equaled approximately

spending at least 1 night either homeless or doubled up in the

30 percent at each time point. The UC families had improved

6 months before the survey.

housing stability over time as measured by the third outcome,

living in own house or apartment. At 20 months, 58 percent

Measuring housing stability using the confirmatory outcome

of the families were living in their own place. This proportion

revealed that 39 percent of UC families spent at least 1 night

increased significantly, to 69 percent of families, at the later

homeless or doubled up in the 6 months before the survey or

followup point.30

29 The proportion of families using emergency shelter during the past 6 months is only about one-half of those reporting being homeless in the past 6 months. This discrepancy is likely due to two factors: (1) some of the homelessness captured in the survey measure is for stays in places not meant for human habitation rather than in emergency shelter, and (2) measurement error in the survey (recall bias) and Program Usage Data (lack of coverage of all local shelters in HMIS records and lack of coverage for families who moved away from the community where they enrolled in the study). The relative importance of these factors is unknown.

30 These results are based on a partial paired t-test. See Appendix C for additional details about the test.

3-Year Impacts of Housing and Services Interventions for Homeless Families

15

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-6. Homeless, Doubled Up, and Living in Own

the percentage of families for whom a child who had been

Place in the UC Group at 20 and 37 Months After RA

with the family in shelter was separated from the family at

any time in the 6 months before the 37-month followup

80

***

survey. This outcome includes both formal (that is, by

a child welfare agency) and informal separations from

69%

70

the family and both ongoing and new separations in this

period. To exclude children who had reached the legal

58%

60

age of adulthood before the separation, the outcome is

based on children who are less than 18 years, 6 months at

50

the time of the followup survey. The time period for this

40

outcome, 6 months before the survey, thus included at

Percent

***

least some time before the children reached adulthood.

31% 29%

30

• The family has at least one foster care placement in

24%

18%

the past 6 months (percent of families). This outcome

20

measures the percentage of families who reported that a

child was in a formal foster care placement or was adopted

10

by another family at any time in the 6 months before the

0

37-month survey. It includes both new and ongoing foster care

At least

At least

Living in own house

placements. This outcome excludes informal arrangements

1 night homeless

1 night doubled up

or apartment

in past 6 months (%)

in past 6 months (%)

at time of survey (%)

in which a child may have stayed with friends or family

20-month survey

37-month survey

members, but it includes adoptions of children by another

family.

UC = usual care.

RA = random assignment.

• Spouse or partner separated in the past 6 months, of

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

those with a spouse or partner present at the time of

Notes: See Appendix B for details on outcome specifications and values. Means are

random assignment (percent of families). This outcome

weighted to adjust for survey nonresponse.

Sources: Family Options Study 20-month followup survey and 37-month followup survey.

measures the percentage of families in which a spouse or

partner who had been with the family in shelter at baseline

was separated from the family at any time in the 6 months

2.3.3. Measures of Family Preservation

before the 37-month survey. The outcome is measured only

for families in which a spouse or partner was present at

Family preservation refers to separation and reunification of

baseline (151 families) and includes both new and ongoing

family members. The study team collected detailed information

separations.

about changes in family composition during the followup

period. The study team collected names and ages of family

The team also constructed the following family reunification

members with the adult respondent in shelter at the time of

outcomes that measure the return of family members who had

random assignment. Interviewers also collected information

been separated from the family at baseline.

about family members the adult respondent considered part

• The family has at least one child reunified, of those

of the family but who were separated from the family at the

families who had at least one child absent at the time of

time of random assignment. Then, during the 37-month

random assignment (percent of families). This outcome

followup survey, the study team collected information on the

measures the percentage of families in which a child who

whereabouts of all family members reported at baseline. The

had been living apart from the family at baseline had

team also collected information about new family members

rejoined the family at the time of the 37-month followup

who joined the family since the previous survey. The family

survey. This outcome is measured only for families in which

preservation analysis examines impacts on five outcomes.

a child was separated from the family at the time of random

The study team used information on changes in family compo-

assignment (107 families).

sition to construct outcomes measuring recent separations of

• Spouse or partner reunified (percent of families). This

family members who were present at baseline.

outcome measures the percentage of families in which a

• The family has at least one child separated in the past

spouse or partner who was separated from the family at

6 months (percent of families). This outcome measures

baseline had rejoined the family at the time of the 37-month

3-Year Impacts of Housing and Services Interventions for Homeless Families

16

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

followup survey. This outcome is measured only for families

separated from children who were with the family at the

in which a spouse or partner was separated from the family

time of random assignment and children who were not with

at the time of random assignment (55 families).

the family at the time of random assignment.

In addition to constructing the outcomes measured from the

2.3.4. Family Preservation in the UC Group

37-month followup survey, the study team also estimated

impacts on two family preservation outcomes measured from

Exhibit 2-7 presents the values of the family preservation

child welfare administrative data that use Adoption and Foster

outcomes for the UC group. Across the UC group, 17 percent

Care Analysis and Reporting System (AFCARS) reporting

of families had a child with the family in shelter at the time

definitions. The team was able to negotiate data use agreements

of the baseline survey who was separated from the family at

in five of the sites (Alameda County, Baltimore, Kansas City,

any time during the 6 months before the 37-month followup

Minneapolis, and Phoenix) to obtain data on out-of-home

survey. Only 3 percent of the UC families reported formal

placements for children in the study sample. The administrative

placements in foster care. Among UC families in which a child

data measure formal out-of-home placements that family heads

was separated at the time of random assignment, 34 percent

may have been reluctant to acknowledge to the interviewer. The

reported that the child had rejoined the family at the time of

child welfare administrative records also permitted the team to

the 37-month survey.

measure the duration of child separations, including those that

Of families who reported a spouse or partner present at

may have started or ended between surveys. These admin istrative

baseline, 38 percent experienced separation from that spouse

data thus provide an important supplement to the self-reported

or partner at any time in the 6 months before the 37-month

family preservation measures collected in the survey.

followup survey. Of the families who had a spouse or partner

The study team constructed two outcome measures for the five

separated at baseline, 21 percent reported that the spouse or

sites that provided child welfare administrative data.

partner had rejoined the family.

• Any formal separations from a child (percent of fam-

Exhibit 2-8 shows the mean values of outcomes measured in

ilies). This outcome measures the percentage of families

the child welfare agency records collected from 5 of the 12

who experienced any formal, out-of-home placements of

study sites. Of the UC family heads in these sites, 12 percent

any children after random assignment. The length of the

had a formal child separation that began after random assign-

followup period was approximately 3 years but varied by

ment. This proportion is notably higher than the survey-reported

site, ranging from 1,046 days to 1,123 days after random

rate of formal separations in the 6 months before the 37-month

assignment.

followup survey. The average total number of days during the

followup period that UC family heads were formally separated

• Total number of days during the followup period that

from at least one child was 83 days. For those UC family

the family was separated from at least one child. This

heads with any formal separation (14.0 percent, not shown in

outcome counts the total number of days the family was

exhibit), the average number of days separated from at least

separated from at least one child. The outcome includes days

one child was 594 days, or about 20 months out of the 3-year

followup period.

Exhibit 2-7. Family Preservation Outcomes in the UC Group

UC Group

Outcome

Mean Value (%)

(Standard Deviation)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%) (N = 545)

16.7

(43.3)

Family has at least one foster care placement in past 6 monthsa (%) (N = 550)

3.4

(20.9)

Spouse/partner separated in past 6 months, of those with spouse/partner present at RA (%) (N = 151) 38.1

(57.7)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at least one child absent at RA (%) (N = 107) 33.7

(55.4)

Spouse/partner reunified, of those with spouse/partner absent at RA (%) (N = 55)

20.7

(46.3)

UC = usual care.

RA = random assignment.

a Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: N = 556. See Appendix B for details on outcome specifications and values. Means and standard deviations are weighted to adjust for survey nonresponse.

Source: Family Options Study 37-month followup survey

3-Year Impacts of Housing and Services Interventions for Homeless Families

17

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-8. Child Welfare Outcomes in the UC Group

UC Group

Outcome

Mean Value (%)

(Standard Deviation)

Had a formal child separation that began after RA (%)

12.3

(32.9)

Total days during followup separated from at least one childa

83.4

(244.2)

UC = usual care.

RA = random assignment.

a Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days.

Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: N = 292. See Appendix B for details on outcome specifications and values.

Source: State child welfare agency records

2.3.5. Measures of Adult Well-Being

adult respondent displayed evidence of alcohol dependence

based on self-reported behavior in the past 6 months. Adult

The study team included outcomes measuring several aspects

respondents were asked to report on the four items in the

of well-being for the adult respondent in the study families.

Rapid Alcohol Problems Screen, or RAPS4 (Cherpitel, 2000).

The outcomes address physical health, mental health, symptoms

An affirmative answer to any of the four items indicates

of trauma, substance use, and experience with domestic violence.

dependence.

All these outcomes are measured in the followup survey.

• Drug abuse (percent of family heads). This outcome

Adult Physical Health

measures the percentage of families in which the adult

Health in the past 30 days was poor or fair (percent of

respondent showed evidence of drug abuse based on

family heads). Adult respondents provided self-reported

self-reported behavior in the past 6 months. Evidence of a

assessments of their physical health in the 37-month followup

drug problem was measured using six items from the Drug

survey. The outcome measures the percentage of families in

Abuse Screening Test, or DAST-10 (Skinner, 1982). An

which the adult respondent reported poor or fair health (rather

affirmative answer to any of the six items indicates abuse.

than good, very good, or excellent health) in the 30 days before

the survey.

• Alcohol dependence or drug abuse (percent of family

heads). This outcome measures the percentage of families

Adult Mental Health

in which the adult respondent displayed evidence of alcohol

The team measured two outcomes related to adult mental health.

dependence or drug abuse.

• Goal-oriented thinking. This outcome is measured with

Experience of Intimate Partner Violence

a modified version of the State Hope Scale (Snyder et al.,

• Experienced intimate partner violence in the past 6 months.

1996). Scores range from 1 to 6, with higher scores repre-

(percent of family heads). The outcome measures the

senting higher levels of positive, goal-oriented thinking.

per centage of adult respondents reporting experience of

• Psychological distress. The study team used the Kessler

inti mate partner violence in the 6 months before the survey.

Psychological Distress Scale (K6) to measure nonspecific

The survey asks whether the adult respondent had been

psychological distress in the 30 days before the survey

physically abused or threatened with violence by a person

(Kessler et al., 2003). The scale ranges from 0 to 24, with

he or she was romantically involved with, such as a spouse,

higher scores indicating greater psychological distress.

boyfriend or girlfriend, or partner in the 6 months before the

survey.

Adult Trauma Symptoms

• Post-traumatic stress disorder (PTSD) symptoms

2.3.6. Adult Well-Being in the UC Group

(percent of family heads). This outcome measures the

Exhibit 2-9 shows the mean values of the adult well-being

percentage of families in which the adult respondent

outcomes for the UC group measured in the 37-month

experienced symptoms of PTSD in the 30 days before the

survey. Nearly one-third (31 percent) of the adult respondents

survey. The study team used responses to the 17 items about

described their health as fair or poor. Across the UC group, 17

PTSD symptoms from the Posttraumatic Stress Diagnostic

percent of adult respondents in UC families experienced psy-

Scale, or PDS, to make this determination.

chological distress, and 23 percent gave survey responses that

Adult Substance Use

indicate symptoms of PTSD. These rates of PTSD and serious

psychological distress for homeless families are substantially

• Alcohol dependence (percent of family heads). This

higher than national rates of PTSD (5.2 percent for women and

outcome measures the percentage of families in which the

3-Year Impacts of Housing and Services Interventions for Homeless Families

18

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-9. Adult Well-Being Outcomes in the UC Group

UC Group

Outcome

Mean Value

(Standard Deviation)

Maternal physical health

Health in past 30 days was poor or fair (%)

31.4

(53.8)

Maternal mental health

Goal-oriented thinkinga

4.50

(1.14)

Psychological distress scoreb

7.20

(6.73)

Symptoms of serious psychological distress (%)

17.33

(43.89)

Maternal trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

22.9

(48.7)

Maternal substance use

Alcohol dependence or drug abusec (%)

11.3

(36.7)

Alcohol dependencec (%)

8.3

(31.9)

Drug abusec (%)

5.1

(25.5)

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

10.5

(35.5)

UC = usual care.

a Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

b Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

c Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: N = 556. See Appendix B for details on outcome specifications and values. Means and standard deviations are weighted to adjust for survey nonresponse.

Source: Family Options Study 37-month followup survey

1.8 percent for men)31 and serious psychological distress (3.9

Regarding incidence of intimate partner violence, 11 percent

percent for women and 2.9 percent for men) (CDC, 2012a).32

of the adult respondents in the UC group reported having

experienced intimate partner violence in the 6 months before

Of UC families, 8 percent of adult respondents gave survey

the 37-month survey. This percentage is substantially lower

responses that indicate alcohol abuse in the 6 months before

than figures reported in other studies, but other studies report

the survey, and 5 percent of respondents gave survey responses

on experience of such violence during a longer period of time

that indicate a history of drug abuse during the same period.

(and often use more detailed measures). At enrollment, 49

These rates are substantially lower than those reported (using

percent of the Family Options Study sample reported having

different measures) by homeless adults in families to the U.S.

experienced violence during their entire adulthood.

Census Bureau’s National Survey of Homeless Assistance

Providers and Clients, or NSHAPC (18 percent for alcohol use

2.3.7. Measures of Child Well-Being

problems and 38 percent for drug use problems within the past

year; Burt et al., 2001). In another study of homeless families,

The study team collected several types of data to measure out-

Rog and Buckner (2007) reported that 12 percent of adult

comes associated with child well-being. For all focal children,

respondents had used illicit drugs in the past year. Both studies

parents reported on children’s school or childcare enrollment;

measured substance use at the time families were in shelter,

attendance; grades; grade completion; experiences at childcare,

using different measures. Rates for Family Options Study

preschool, or school; behavior at school or childcare; and

families during the baseline shelter stay were more comparable

attitudes about school or childcare. Parents also reported on

to the Rog and Buckner (2007) figure. At the Family Options

prosocial behaviors and emotional and behavioral problems

Study baseline, 11 percent indicated alcohol abuse and 14

of children with the Strengths and Difficulties Questionnaires

percent of family heads reported drug use in the past year.33

(SDQ; Goodman, 1997) and also on children’s health, access to

31 The statistic for PTSD is the national 12-month prevalence rate as measured in the National Comorbidity Survey Replication (NCS-R), which was fielded in 2001 and 2002. The NCS-R used a different instrument to measure PTSD than did the Family Options Study. See Harvard Medical School (2012).

32 The statistic for national rate of serious psychological distress is from the 2011 National Health Interview Survey. This survey used the same measure of psychological distress as used in the Family Options Study (CDC, 2012a).

33 See the Interim Report: Family Options Study (Gubits et al., 2013) for additional detail about the study sample at the time of random assignment.

3-Year Impacts of Housing and Services Interventions for Homeless Families

19

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

health care, and sleep disruptions, which are associated with

• Grade completion—not held back (percent of focal

a variety of emotional and behavioral disorders (Dahl and

children). This outcome is measured from the parent survey

Harvey, 2007). Additional instruments were tuned to children’s

and defined for children who have been in school at any

developmental stage. Children 12 to 66 months of age were

time since random assignment and who are less than age 18

assessed with the Ages and Stages Questionnaires (ASQ-3)—a

at the time of the 37-month survey. The outcome measures

family of questionnaires used to measure gross and fine motor

the percentage of these children who have not repeated a

skills, social development, communication, and problemsolv-

grade level and have not been prevented from moving on to

ing, as observed by parents (Squires et al., 2009). The adult

the next grade level since random assignment.

respondent completed the ASQ-3 form. Study staff assessed

• School grades. This outcome measures the parent’s

children from 3 years, 6 months to 7 years, 11 months of age

assessment of the child’s school grades for the most recently

with the Woodcock-Johnson III (WJ III) letter-word identi-

completed term. The outcome uses a 4-point scale ranging

fication and applied problems scales (Woodcock, McGrew,

from 1 (mostly Ds and Fs) to 4 (mostly As).

and Mather, 2001), which are early indicators of verbal and

quantitative/analytic skills, respectively. Children in this age

Health

group also completed the Head Toes Knees Shoulders (HTKS)

task. HTKS assesses self-regulation, in which children must

• Poor or fair health (percent of focal children). This

remember rules and inhibit incorrect responses (for example,

outcome measures the parent’s assessment of the child’s

by following instructions to touch their head when the inter-

health at the time of the 37-month followup survey. The

viewer says “touch your toes”).

outcome measures the percentage of children with poor or

fair health (rather than good, very good, or excellent health),

Finally, the study team conducted surveys with children from

based on the parent’s assessment.

8 years to 17 years, 11 months of age measuring anxiety, fears,

and substance use. This array of measures, along with the paren-

• Well-child checkup in the past year (percent of focal

tal report, captured the most likely mental health consequences

children). This outcome measures the percentage of focal

of homelessness and behavioral responses thereto. Parental

children who received a physical examination or well-child

reports of behavior for this age group included arrests or police

checkup in the year before the 37-month survey, based on

involvement. Youth reported on school effort to supplement

the parental report.

parental reports of functioning in the key developmental domain

• Child has a regular source of health care (percent of

of school. Youth also completed the Children’s Hope Scale

focal children). This outcome measures the percentage of

(Snyder et al., 1997), a measure of goal-oriented thinking.

focal children who had a regular provider of health care at

the time of the 37-month followup survey, based on the

Child Well-Being Measures From Parental Report

parental report.

Across Age Groups

The study team used the parent-reported information on focal

• Sleep problems. This outcome measures the parental report

children to construct the following child well-being outcomes

on the frequency of two indicators of sleep problems—

that are measured for children across age groups.34 All these

tiredness on waking and tiredness during the day. The out-

outcomes are measured in the 37-month adult followup survey.

come is measured using a score of 1 to 5, with higher values

indicating a greater frequency of these sleep problems.

Child education

• Number of schools attended since random assignment.

Behavioral challenges and strengths

This outcome is measured from parental reports on the

• Behavior problems. The followup survey included the

37-month survey. The outcome indicates the total number of

SDQ, a battery of items about the behavioral strengths and

schools a child attended since random assignment. (Change

difficulties of children. This outcome is measured as the

in school could be because of grade completion, residential

nationally standardized score from the SDQ. The parent

move, or another reason.) The outcome is measured using

reported on focal children’s behavior using the SDQ. The

a count of 1 to 4. Number of schools is top coded at four or

total problem score measures emotional symptoms, conduct

more schools.

problems, hyperactivity, and peer problems. The reported

34 The study team analyzes child outcomes in this report somewhat differently than for the Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015). For example, school absences in the past month, positive school experiences, positive school attitudes, and school conduct problems were collected for only the first 38 percent of parents surveyed due to an error in data collection. Parallel questions for younger, preschool aged children were collected from all parents. These outcomes are thus analyzed and presented separately by child age group. In addition, preschool and school enrollment are analyzed and presented by age group. See Appendix B for additional information about the child outcomes.

3-Year Impacts of Housing and Services Interventions for Homeless Families

20

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

scores are standardized by age and gender so that children

Health

can be compared with their peers in a national sample.

Parents reported that 6 percent of children in the UC group

Higher scores indicate greater behavior problems.

were in fair or poor health 37 months after random assignment.

This percentage is similar to 5 percent of poor children but is

• Prosocial behavior. Parents reported on focal children’s

higher than 1 percent of nonpoor children younger than age 18

prosocial behavior. Prosocial behavior refers to positive

in the National Health Interview Survey in 2012. The propor-

actions that benefit others. Prosocial behavior is measured

tion of study children who had a regular source of medical care

for the study using the parental report to the SDQ prosocial

was similar to the national rate for poor children (92 percent

domain and is a nationally standardized score. Higher scores

compared with the national rate of 95 percent; Bloom, Jones,

indicate greater prosocial behavior.

and Freeman, 2013). Only 91 percent of children in the UC

group had received a well-child checkup in the year before

2.3.8. Characteristics of Focal Children in

the 37-month survey. Parents reported that children rarely

the UC Group

had trouble waking up or were tired during the day (2.18 on a

Exhibit 2-10 displays the values of the just-described outcomes

5-point scale, wherein 2 indicates rarely).

for focal children in the UC group.

Behavioral Challenges and Strengths

School and Childcare

Children in the UC group scored markedly higher than nation-

Children had attended 2.1 schools, on average, since random

al norms on behavioral problems (0.59 standard deviations

assignment, which means one school change, on average, in

in the national data) and somewhat lower (0.24 standard

3 years. School mobility is associated with lower levels of aca-

deviations) on prosocial behavior.

demic achievement.35 Furthermore, only 91 percent of children

had completed all the grade levels in which they were enrolled

Child Well-Being Outcomes for Specific Age Groups

by the end of the school year; that is, 9 percent had been held

The study team constructed the following child well-being

back since random assignment. The grades that children earned

outcomes measured only for specific age groups.36

in school, as reported by parents, averaged 3.06 on a scale

wherein 3 is mostly Bs and 4 is mostly As.

Exhibit 2-10. Child Well-Being Outcomes in the UC Group Measured for Children Across Age Groups UC Group

Outcome

Mean Value

(Standard Deviation)

Child education

Number of schools attended since random assignmenta

2.10

(1.18)

Grade completion—not held back (%)

91.0

(38.2)

School gradesb

3.06

(1.14)

Child physical health

Poor or fair health (%)

5.9

(31.0)

Well-child checkup in past year (%)

90.6

(38.4)

Child has regular source of health care (%)

91.7

(36.4)

Sleep problemsc

2.18

(1.46)

Child behavioral strengths and challenges

Behavior problemsd

0.59

(1.69)

Prosocial behaviore

– 0.24

(1.43)

UC = usual care.

a Number of schools outcome is topcoded at 4 or more schools.

b School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, and 4 = mostly As.

c Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness on waking and during the day.

d Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

e Prosocial behavior is measured as the standardized prosocial domain score from the Strengths and Difficulties Questionnaire (SDQ).

Notes: Sample sizes vary by outcome. See Appendix B for details on outcome specifications and values. Means and standard deviations are weighted to adjust for survey nonresponse.

Source: Family Options Study 37-month followup survey (parent report)

35 National Research Council and Institute of Medicine (2010).

36 Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015) also examined low birth weight for randomly selected focal children born since random assignment. The study team did not reanalyze this outcome in the 37-month analysis because only an additional 10 focal children born since random assignment were newly selected at the time of the 37-month survey. Therefore, results would have been almost identical to those previously reported.

3-Year Impacts of Housing and Services Interventions for Homeless Families

21

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Ages 2 years to 5 years37

• Math ability. This outcome is measured as the nationally

• Preschool or Head Start enrollment (percent of focal

standardized score from the WJ III applied problems

children). This parent-reported outcome measures the

test. The study team administered the assessment to focal

percentage of children in the age range who were enrolled in

children ages 3 years, 6 months to 7 years.

preschool, center-based childcare, or Head Start at the time

• Executive functioning (self-regulation). This outcome is

of the 37-month followup survey.38

measured using the HTKS developmental assessment, meas-

• Childcare or preschool absences in the past month. This

uring inhibitory control, attention, and working memory.

outcome is measured from parental reports of the number

The study team administered the assessment to children ages

of absences from childcare or preschool in the month before

3 years, 6 months to 7 years.

the survey. The outcome is measured using a scale of 0 to 3,

Ages 5 years to 17 years40

with 0 indicating no absences, 1 indicating 1 to 2 absences,

2 indicating 3 to 5 absences, and 3 indicating 6 or more

• School enrollment (percent of focal children). This

absences.

outcome is measured for children ages 5 to 17 years, using

the parental report. It measures the percentage of children

• Positive childcare or preschool experiences. This outcome

ages 5 to 17 who were enrolled in school at the time of the

measures the parent’s assessment of the child’s childcare or

37-month followup survey.

preschool experiences, rating them as mostly positive (=1),

both positive and negative (=0), or mostly negative (=-1).

• School absences in the past month. This outcome is meas-

ured from the parental report of the number of absences

• Positive childcare or preschool attitudes. This outcome

from school in the month before the survey. The outcome

measures the parent’s assessment of the child’s attitude toward

is measured using a scale of 0 to 3, with 0 indicating no

childcare or preschool. The parent was asked to rate how

absences, 1 indicating 1 to 2 absences, 2 indicating 3 to 5

much the child likes childcare or preschool. The outcome

absences, and 3 indicating 6 or more absences.

uses a 5-point scale ranging from 1 (not at all) to 5 (very much).

• Positive school experiences. This outcome measures the

• Childcare or preschool conduct problems (percent of

parent’s assessment of the child’s school experiences, rating

focal children). This outcome measures whether the parent

them as mostly positive (=1), both positive and negative

reports having been contacted by the child’s childcare or

(=0), or mostly negative (=-1).

preschool provider regarding the child’s conduct problems

or if the child was suspended or expelled.

• Positive school attitudes. This outcome measures the parent’s

assessment of the child’s attitude toward school. The parent

Ages 2 years to 5 years, 6 months

was asked to rate how much the child likes school or child-

• Met developmental milestones (percent of focal

care. The outcome uses a 5-point scale ranging from 1 (not

children). This outcome is defined as the percentage of

at all) to 5 (very much).

focal children who scored above the typical developmental

• School conduct problems (percent of focal children).

thresholds on the five domains measured in the parent-

This outcome measures whether the parental reports having

reported ASQ-3.39

been contacted by the child’s school or childcare provider

regarding the child’s conduct problems or if the child was

Ages 3 years, 6 months to 7 years

suspended or expelled.

• Verbal ability. This outcome is measured as the nationally

standardized score from the WJ III letter-word identification

In this age group, the last four outcomes about school-aged

test. The study team administered the assessment to focal

children (school absences, school experiences, school attitudes,

children ages 3 years, 6 months to 7 years.

and school conduct problems) were collected from only the

37 This age group includes focal children who were ages 4 years or younger on the September 1st before the 37-month survey. These children had thus not reached 5 years of age (typical school age) in the school year corresponding to the 37-month survey.

38 Though this enrollment outcome is named differently than the school enrollment outcome for children ages 5 to 17 years, the two outcomes are defined identically. For this younger group, enrollment is overwhelmingly in preschool or center-based childcare. For the older group, enrollment is overwhelmingly in school.

39 In the 20-month data collection, the ASQ-3 was collected for focal children who were less than 3 years, 6 months of age. To capture the same set of children at the later followup point, the outcome was collected for focal children who were up to 5 years, 6 months of age.

40 This age group includes focal children who were ages 5 to 17 years on the September 1st before the 37-month parent survey and who were no older than 17 years at the time of the survey. These children were thus typical school age in the school year corresponding to the 37-month survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

22

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

first 38 percent of parents surveyed due to an error in data

2.3.9. Characteristics of Focal Children by

collection. The parallel outcomes for younger, preschool-aged

Age-Specific Outcomes in the UC Group

children were collected from all parents. Because of the dis-

Exhibit 2-11 displays the values of the outcomes described

crepancy in data collection between the two age groups, these

previously for focal children in the UC group.

outcomes are analyzed separately according to age group.

Ages 2 years to 5 years

Ages 8 years to 17 years

Among children ages 2 years to 5 years in the UC group, 39

Older children were surveyed about a broader array of develop-

percent were enrolled in center-based care, preschool, or Head

mental outcomes. For focal children between the ages of 8 and

Start. Children in this age group scored 0.8, on average, for

17 at the time of the 37-month followup data collection, the

absences from preschool or childcare in the past month, where-

study team measured five outcomes from the child survey and

in 0 indicates no absences and 3 indicates 6 or more absences.

one from the parental report.

(During the summer, parents reported on the most recent

• Anxiety. This outcome is measured using the A-Trait scale

month of enrollment.) Parents reported that children had

from the State-Trait Anxiety Inventory for Children, or

mostly positive experiences in preschool or childcare (0.8 on

STAIC (Spielberger et al., 1973). Scores range from 20 to 60,

a 3-point scale wherein 0 indicates both positive and negative

with higher scores indicating greater anxiety.

experiences, 1 indicates mostly positive experiences and -1 in-

dicates mostly negative experiences). Parents also reported that

• Fears. This outcome is measured using the Children’s Fear

children liked preschool or childcare, averaging 4.5 on a scale

Scale (Ramirez, Masten, and Samsa, 1991). Scores range

wherein 4 is pretty much and 5 is very much. Parents reported

from 33 to 99, with higher scores indicating more fear.

that about 5 percent of the children had conduct problems

Children were asked to indicate the extent to which they

necessitating some contact from preschool or childcare.

had 33 different fears.

• Substance use. This outcome, which combines data using

Ages 2 years to 5 years

23 items from the Centers for Disease Control and Prevention

By parental report, only 72 percent of children met the devel-

(CDC, 2012c) Youth Risk Behavior Surveillance—United

opmental cutoff score in all five domains of ASQ-3. This rate

States, 2011, measures whether the child had used tobacco,

was somewhat less than the national rate of 77 to 86 percent

alcohol, or marijuana in the past 30 days or had ever used

used by the creators of the ASQ-3 instrument.41 Children were

other substances—cocaine, inhalants, or steroids (ages 8

least likely to meet age standards for fine motor development

to 17) or ecstasy, methamphetamine, heroin, controlled

and most likely to meet standards for gross motor develop-

prescription drugs, or injected drugs (ages 13 to 17 only).

ment, with performance in the communication, problemsolv-

ing, and personal-social domains falling in between.

• Goal-oriented thinking. This outcome is measured with

a modified version of the Children’s Hope Scale (Snyder et

Ages 3 years, 6 months to 7 years

al., 1997), which measures positive, goal-oriented thinking.

Children ages 3 years, 6 months to 7 years were assessed

Scores range from 6 to 30, with higher scores indicating

directly with two subscales of the WJ III test of cognitive

greater hope.

abilities, with scores compared with national age norms. Given

• School effort in the past month. On the child survey,

the large association of family income with reading and math

respondents were asked to report on how hard they worked

ability (for example, Miller, Votruba-Drzal, and Setodji, 2013),

in the month before the survey during the school day and

it is not surprising that children in the UC group scored about

on homework. The outcome measure ranges from 1 to 4,

one-third of a standard deviation below national norms on

with higher scores indicating greater effort at school and on

both letter-word identification (a measure of verbal ability) and

homework.

applied problems (an early measure of math ability).

• Arrests or police involvement in the past 6 months

Children also completed the HTKS test of self-regulation or

(percent of focal children). This outcome is measured

executive functioning, with a mean score of 17.8 out of a

using the parental report about whether the child had any

possible 40. The mean score was substantially lower than in a

problems that involved the police contacting the parent and

normative sample of largely middle-class children in Michigan

whether the child had been arrested in the 6 months before

and Oregon (27.45 out of 40) of the same mean age, although

the followup survey.

both the age range and the variability of scores for children in

41 Squires et al. (2009).

3-Year Impacts of Housing and Services Interventions for Homeless Families

23

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-11. Child Well-Being Developmental Outcomes in the UC Group for Children in Specific Age Groups UC Group

Outcome

Mean Value

(Standard Deviation)

Ages 2 to 5 yearsa

Preschool or Head Start enrollmentb

39.0

(61.7)

Child care or preschool absences in past monthc

0.8

(1.1)

Positive child care or preschool experiencesd

0.8

(0.5)

Positive child care or preschool attitudese

4.5

(0.8)

Child care or preschool conduct problemsf

5.2

(27.5)

Ages 2 years to 5 years, 6 months

Met developmental milestonesg (%)

72.1

(59.2)

Ages 3 years, 6 months to 7 years

Verbal abilityh

– 0.33

(1.49)

Math abilityi

– 0.30

(1.30)

Executive functioningj (self-regulation)

17.8

(23.9)

Ages 5 to 17 yearsk

School enrollmentb

97.9

(19.2)

School absences in past monthc,r

1.1

(1.3)

Positive school experiencesd,r

0.5

(0.8)

Positive school attitudese,r

4.1

(1.5)

School conduct problemsf,r

30.3

(62.5)

Ages 8 to 17 years

Anxietyl

35.5

(10.9)

Fearsm

63.6

(21.0)

Substance usen (%)

7.4

(38.2)

Goal-oriented thinkingo

22.1

(6.8)

School effort in past monthp

2.8

(1.1)

Arrests or police involvement in past 6 monthsq (%)

8.4

(38.2)

UC = usual care.

a Includes focal children who were age 4 years or younger on September 1 before the 37-month parent survey.

b Outcome defined as enrollment in preschool, center-based child care, or school.

c Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

d Positive childcare or school experiences outcome is defined as -1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

e Positive childcare or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

f Childcare or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or childcare center.

g Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

h Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word Identification test.

i Math ability outcome is the nationally standardized score from the WJ III applied problems test.

j Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

k Includes focal children who were ages 5 to 17 years on September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

l Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

m Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

n Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention (CDC) 2011 Youth Risk Behavior Survey.

o Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

p School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

q Arrests or police involvement in past 6 months is from parent report.

r This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

Notes: Sample sizes vay by outcome. See Appendix B for details on outcome specifications and values. Means and standard deviations are weighted to adjust for survey and assessment nonresponse.

Sources: Family Options Study 37-month followup survey; ASQ-3; WJ III; HTKS assessment; Family Options Study 37-month child survey our sample were larger (fall scores from Ponitz et al., 2009).

Ages 5 years to 17 years

Children in our study scored about the same as in the control

Among school-age children ages 5 to 17 years, 97.9 percent

group of an intervention study involving low-income, multi-

were enrolled in school at the time of the 37-month survey.

racial, multiethnic children in San Diego, California (Layzer,

This rate is similar to the national enrollment rate (92.9 to 97.8

2014).

percent for children ages 7 to 17; U.S. Bureau of the Census,

2014). Children in the 5-to-17 age group scored 1.1, on average,

3-Year Impacts of Housing and Services Interventions for Homeless Families

24

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

for absences from school in the past month, wherein 0 indicates

Parents reported that 8 percent of children ages 12 to 17 had

no absences and 3 indicates 6 or more absences. Most children

had a problem that involved the police contacting the parent or

in the UC group (70 percent) had no conduct problems at

had been arrested in the 6 months before the 37-month survey.

school, but the balance (30 percent) did have problems that led

to the school’s contacting the parent or suspending or expelling

2.3.10. Measures of Self-Sufficiency

the child.

The impact analysis examines relative effects of the four inter-

Ages 8 years to 17 years

ventions on several outcomes pertaining to self-sufficiency. The

study team used the 37-month followup survey to construct

Children’s scores on the State-Trait Anxiety Inventory for

five categories of self-sufficiency outcomes: (1) employment,

Children—STAIC measure, a general measure of worries, (mean

(2) sources of income, (3) receipt of education or training,

of 35.5) were somewhat less than those in the normative sample

(4) food security, and (5) economic hardship.

of fourth graders from the test originators (36.3 for males and

38.1 for females; Spielberger, 1970) and further were less than

Employment

those in a large sample of disadvantaged Black children from a

The study team used responses to the adult followup survey to

large metropolitan school district (40.0 for males and 40.26 for

construct six outcomes regarding employment status.

females; Papay and Hedl, 1978). A score of 40 would reflect an

average answer of “sometimes” on a 3-point scale from “hardly

• Work for pay in the week before the survey (percent of

ever” to “often” across all items.

family heads). This outcome measures the percentage of

survey respondents who reported working for pay in the

Across 33 specific fears, rated on a scale from “not at all” to

week before the 37-month followup survey.

“a lot,” children averaged slightly less than “some,” or 63.6, at

the 37-month followup survey (a consistent answer of “some”

• Any work for pay since the 20-month survey (percent

would yield a score of 66). The fears most commonly rated

of family heads). This outcome measures the percentage

“a lot” (by more than one-half of the youth) were “I worry

of survey respondents who reported working for pay at any

about my brothers and sisters,” “I worry about my parents,”

time since the date of their 20-month survey. It is defined

and “I worry about myself.”

only for families who responded to both 20-month and

37-month followup surveys.

Substance use in the UC group was quite low by comparison

with national norms. (Our data are self-reported, but so are the

• Months worked for pay since the 20-month survey

corresponding national norms from CDC (2012c). Only 7 per-

(includes partial months). This outcome is a count of the

cent of children ages 8 to 17 in the UC group reported having

months worked since random assignment, including partial

used tobacco, alcohol, or marijuana in the past 30 days or had

months. It is defined only for families who responded to

ever used more serious drugs. The CDC provides norms for

both 20-month and 37-month followup surveys.

children in grades 9 through 12, to which the 13- to 17-year-

• Any work for pay since random assignment (percent of

old youth in the study are compared.42 Study youth were less

family heads). This outcome measures the percentage of

likely to have smoked (8 versus 22 percent), used alcohol (11

survey respondents who reported working for pay at any

versus 35 percent), or used marijuana (10 versus 23 percent) in

time since random assignment.

the past 30 days.

• Months worked for pay since random assignment

The measure of goal-oriented thinking, a version of the Chil-

(includes partial months). This outcome is a count of the

dren’s Hope Scale (Snyder et al., 1997), measures both belief in

months worked since random assignment, including partial

one’s ability to solve problems and reach goals and belief about

months.

esteem or efficacy. The version in this study used a slightly

different scoring format than the original. The average score of

• Hours of work per week at current main job. For adult

22.1 indicates answers closer to having these beliefs “most of

respondents who had more than one job in the week before

the time,” or 24, than “a lot of the time,” or 18.

the 37-month survey, the main job is defined as the job at

which she or he usually worked the most number of hours

For school effort, children rated how hard they worked on

per week.

homework and during the school day, with an average of 2.8

on a 4-point scale wherein 2 is “could have done a little better”

and 3 is “did about as well as you could.”

42 Kann et al. (2014).

3-Year Impacts of Housing and Services Interventions for Homeless Families

25

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Income Sources and Amounts

• Number of weeks in training programs since random

The study team also constructed outcomes that measure the

assignment.

percentage of families who reported receiving income from each of

• Participated in school lasting 2 weeks or more since

the following sources in the month before the survey. The fam-

random assignment (percent of family heads). This

ily heads reported whether anyone in the family had received

outcome measures the percentage of families in whom the

income from each of the following sources in the past month.

adult respondent reported having participated in school or

• Earnings.

academic training lasting 2 weeks or more since random

• Temporary Assistance for Needy Families (TANF).

assignment. School or academic training is defined as at-

• Social Security Disability Insurance (SSDI).

tending a regular high school directed toward a high school

diploma; preparing for a general educational development,

• Supplemental Security Income (SSI).

or GED, examination; taking courses at a 2-year or 4-year

• Supplemental Nutrition Assistance Program (SNAP).

college; or taking graduate courses.

• Special Supplemental Nutrition Program for Women,

• Participated in basic education lasting 2 weeks or more

Infants, and Children (WIC).

since random assignment (percent of family heads).

The study team also constructed two other outcomes related to

This outcome measures the percentage of families in whom

income.

the adult respondent reported having participated in basic

education lasting 2 or more weeks since random assignment.

• Annualized earnings from the main job. This outcome

Basic education is defined as nonvocational adult education,

measures the annualized value of current earnings from the

literacy training, or English as a second language not

job reported at the time of the 37-month survey. This value

directed toward a degree.

usually represents either the product of the reported hourly

wage and usual hours per week multiplied by 52 weeks or

• Participated in vocational education or training lasting

the reported usual weekly earnings multiplied by 52 weeks.

2 weeks or more since random assignment (percent of

By construction, the measure ignores any seasonality in

family heads). This outcome measures the percentage of

earnings.

families in whom the adult respondent reported having

participated in vocational education or training lasting 2 or

• Total family income. This outcome measures total family

more weeks since random assignment. Vocational education

income from all sources for the calendar year preceding the

or training is defined as business or technical schools,

survey (2013 for all respondents).

employer- or union-provided training, or military training in

vocational skills (not military skills).

Education and Training

The analysts constructed five outcomes pertaining to partici-

Food Security

pation in education and training activities during the followup

The analysis examines impacts of the interventions on food

period. The followup survey asked respondents whether they

security for two outcomes.

had participated in any education or training activities since

random assignment and, if so, how many weeks they spent

• Household is food insecure (percent of families). This

in such programs. For up to six programs reported, sample

outcome measures the percentage of families determined

members reported on the type of program. The study team

“food insecure” at the time of the followup survey according

used this information to construct the following education and

to criteria used by the U.S. Department of Agriculture

training outcomes.

(USDA).43

• Participated in any school or training lasting 2 weeks

• Food insecurity scale. This outcome measures the food

or more since random assignment (percent of family

insecurity level of each family based on responses to the

heads). This outcome measures the percentage of families in

USDA food security questions included on the followup

whom the adult respondent reported having participated in

survey. The food insecurity scale ranges from 0 to 6, with

any type of school or training lasting 2 or more weeks since

higher values indicating greater food insecurity.44

random assignment.

43 See Nord, Andrews, and Carlsen (2005). The assessment of food insecurity is based on two USDA short-form metrics, which are scores assigned to a household based on answers to six survey questions.

44 See Appendix B for further details on the construction of this outcome.

3-Year Impacts of Housing and Services Interventions for Homeless Families

26

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Economic Hardship

• Any employment in quarters 11 to 14 after random

The study team also measured the economic hardship reported

assignment (percent of families). This outcome measures

by each family at the time of the 37-month followup survey

the percentage of family heads with any employment in

on a measure derived from Pearlin and Schooler (1978). The

quarters 11 to 14 after the quarter of random assignment, as

outcome, expressed as an economic stress scale, measures the

indicated by the presence of any earnings during the quarter.

extent of hardship using responses about the frequency with

• Number of quarters employed in quarters 11 to 14 after

which the family said they experienced an inability to afford

random assignment. This outcome counts the total number

medical care the family needed, clothing the family needed, lei-

of quarters that the family head was employed during

sure activities the family wanted, or rent. The economic stress

quarters 11 to 14 after the quarter of random assignment.

scale also takes into account the adult respondent’s assessment

The team measured this outcome using the number of

of the family’s monthly finances; that is, whether they usually

quarters in which the NDNH data indicated that the family

have some money left over at the end of the month, barely

head had earnings.

enough to make ends meet, or not enough to make ends meet.

Earnings and Employment From Administrative Data

2.3.11. Self-Sufficiency Indicators for the

In addition to the self-sufficiency outcomes constructed from

UC Group

survey data, the study team measured three employment and

Exhibit 2-12 displays the values of the self-sufficiency outcome

earnings outcomes from administrative records. Under a Mem-

measures for the UC group. More than one-third (37 percent)

orandum of Understanding with the U.S. Department of Health

of the adult respondents in these families reported working for

and Human Services’ Office of Child Support Enforcement

pay in the week before the survey, and 72 percent said they

(OCSE), HUD matched the Family Options Study to quarterly

had worked for pay at some time since random assignment.

wage records maintained by HHS. These records are part of

The adult respondents in the UC group spent an average

the larger National Directory of New Hires (NDNH) database,

of 13.1 months working for pay since random assignment.

administered by the OCSE. To adhere to privacy protections

Including respondents who were not working at all, they

that govern the use of the NDNH data for research purposes,

worked an average of 11.8 hours per week at the current job,

HUD personnel analyzed these data and provided results to the

and annualized earnings from the current job averaged $6,414.

study team.

The 37 percent of UC families who were working at the time

The Family Options Study obtained quarterly earnings for the

of the survey worked an average of 32 hours per week, had

study sample for the period covering the second quarter of

hourly earnings of $10.57,45 and had annualized earnings for

2013 through the third quarter of 2015. This calendar period

the current job of $18,125.

provided data for the full study sample that corresponds

Exhibit 2-13 shows the proportion of families working for

with the 11th through 14th quarters after the quarter of

pay and having any work since random assignment to the

random assignment. This 1-year period covered by the NDNH

UC group. A statistically significant higher proportion of UC

earnings data is approximately 32 to 44 months after random

families (37 percent) were working at the 37-month survey

assignment and includes the date of 37-month followup survey

than either at baseline (17 percent) or at the 20-month survey

response for nearly all the survey respondents.

(31 percent). The proportion that had performed any work

The study team constructed three outcomes from the NDNH

for pay since random assignment also significantly increased

quarterly wage data.

between 20 and 37 months, from 61 percent to 72 percent.

The large difference46 between the proportions working for pay

• Total earnings in quarters 11 to 14 after random assign-

at the surveys and those who had worked for pay at any point

ment. This outcome measures the sum of all earnings for

since random assignment shows that a sizable proportion of the

each family head in the 11th through 14th quarters after the

family heads are working intermittently.

quarter of random assignment. The outcome is measured

in 2015, quarter 3 dollars. Because earnings are available

Families in the UC group reported median cash income

for four consecutive quarters for all families, the earnings

from all sources of $12,099 for the calendar year before the

measure is not subject to seasonality bias.

37-month survey (2013 for all families), higher than the

45 The average hourly earnings were calculated for those who reported wages on an hourly, a weekly, or a biweekly basis (representing 89 percent of those working for pay at the time of the followup survey).

46 Results are based on partial paired t-tests. See Appendix C for more details about the test.

3-Year Impacts of Housing and Services Interventions for Homeless Families

27

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-12. Self-Sufficiency Outcomes in the UC Group

UC Group

Outcome

Mean Value

(Standard Deviation)

Employment status

Work for pay in week before survey (%)

37.0

(56.0)

Any work for pay since 20-month surveya (%)

63.6

(48.2)

Months worked for pay since 20-month surveya,b

7.2

(7.9)

Any work for pay since RA (%)

72.3

(51.9)

Months worked for pay since RAb

13.1

(15.5)

Hours of work per week at current main jobc

11.8

(19.3)

Income sources and amounts

Annualized current earnings ($)

6,414

(12,250)

Total family income ($)

12,099

(13,102)

Anyone in family had earnings in past month (%)

50.1

(58.0)

Anyone in family received TANF in past month (%)

22.9

(48.7)

Anyone in family received SSDI in past month (%)

9.1

(33.3)

Anyone in family received SSI in past month (%)

14.8

(41.2)

Anyone in family received SNAP/food stamps in past month (%)

81.6

(44.9)

Anyone in family received WIC in past month (%)

23.0

(48.8)

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

38.8

(56.5)

Number of weeks in school or training programs since RA

8.5

(20.2)

Participated in 2 weeks or more of school since RA (%)

11.5

(37.0)

Participated in 2 weeks or more of basic education since RA (%)

2.2

(16.9)

Participated in 2 weeks or more of vocational education since RA (%)

14.3

(40.5)

Food security

Household is food insecure (%)

46.8

(57.8)

Food insecurity scaled

1.94

(2.42)

Economic stressors

Economic stress scalee

– 0.11

(0.57)

UC = usual care.

RA = random assignment. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

a Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

b Number of months worked for pay includes partial calendar months.

c Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

d Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

e Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: N = 556. See Appendix B for details on outcome specifications and values. Means and standard deviations are weighted to adjust for survey nonresponse.

Source: Family Options Study 37-month followup survey

amount reported at the 20-month survey ($9,067) and at

More than one-third (39 percent) of the adult respondents in

baseline ($7,410). The 2013 federal poverty threshold for

the UC families said they had participated in 2 or more weeks

three-person households (the median family size for study

of school or training since random assignment. On average,

families) was $18,769 (U.S. Census Bureau, 2013).

families in the UC group said they spent 8.5 weeks in educa-

tion or training activities. Nearly one-half (47 percent) of UC

Regarding sources of income in the month before the 37-month

families met USDA criteria for food insecurity at the time of the

survey, the UC families reported a high rate of SNAP receipt

37-month followup survey.47

(82 percent). Other sources of income reported were earned

income (50 percent of families), TANF (23 percent), WIC (23

Exhibit 2-14 shows the means for the earnings and employ-

percent), SSI (15 percent), and SSDI (9 percent).

ment outcomes measured in the administrative quarterly wage

records. The mean value of earnings during the 11th through

47 The proportion of UC families who met criteria for food insecurity at the time of the 20-month survey was nearly identical, at 45.9 percent. Due to a programming error, this proportion was erroneously reported as 35.5 percent in the Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015; Exhibit 5-13).

3-Year Impacts of Housing and Services Interventions for Homeless Families

28

FAMILY OPTIONS STUDY

Chapter 2. Description of Usual Care (UC) and Outcomes Measured in the Study

Exhibit 2-13. Work for Pay at 20 and 37 Months After RA

emergency shelter. This chapter examined patterns of program

in the UC Group

use and outcomes for families assigned to the UC group who

responded to the 37-month followup survey.

80

***

Emergency shelters in this study were the entry points into

72%

70

homeless assistance in each site. Families randomly assigned

to the UC group typically remained in emergency shelter and

61%

60

sought whatever assistance was available in the community.

UC families stayed in emergency shelter for an average of 4 months

50

during the followup period. For about one-fourth of families

**

assigned to the UC group, the emergency shelter where families

40

37%

were recruited was the families’ only interaction with the home-

Percent

31%

less assistance or housing subsidy system during the 3-year

30

followup period. The remainder found their way to other

types of assistance. In the 3 years after random assignment,

20

17%

20 percent of UC families received rapid re-housing assistance

10

and 30 percent received transitional housing. Altogether, more

than one-third (37 percent) of those in the UC group received

0

some sort of permanent housing subsidy during the followup

Work for pay in week before survey (%)

Any work for pay since RA (%)

period—accessing either the subsidy intervention defined for

Baseline

20-month survey

37-month survey

the study, public housing, permanent supportive housing, or

RA = random assignment.

project-based housing assistance programs. Families in the

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respec-UC group who received permanent housing subsidies tended

tively, using a two-tailed t-test.

Notes. See Appendix B for details on outcome specifications and values. Means are

to retain them. Most UC families (82 percent) who ever used

weighted to adjust for survey nonresponse.

a permanent subsidy in the 3-year followup period were still

Sources: Family Options Study baseline survey, 20-month followup survey, and

37-month followup survey

receiving the assistance at the time of the 37-month survey.

Overall, UC families were not faring that well at 37 months

14th quarters after the quarter of random assignment for UC

after random assignment, although they were somewhat better

family heads was $5,555. About 57 percent of UC family heads

off, on average, than at 20 months. Of the whole group, 18

had any employment during this 1-year period. The average

percent reported being homeless in the 6 months before the

number of quarters with employment during this 1-year period

37-month survey, 5 percent were in emergency shelter in the

for all UC family heads was 1.7 quarters, and the average

survey month, and 69 percent were living in their own house

number of quarters with employment for UC families who had

or apartment. Only 37 percent of family heads had worked for

any employment during the year was 2.9 quarters.

pay in the week before the survey. Median annual cash income

from all sources for the calendar year before the survey was

2.4. Summary

about $12,000.

The 37-month outcomes presented in this chapter for the UC

The experiences of families who did not receive a special offer

families provide the benchmarks against which the study team

of assistance at the time of random assignment tell how families

estimated impacts of the active interventions. The next chapter

in the 12 study communities navigate the housing and home-

presents the impacts of priority access to the permanent hous-

less assistance system and how they fare 3 years after entering

ing subsidy intervention compared with usual care.

Exhibit 2-14. Earnings and Employment in the UC Group

UC Group

Outcome

Mean Value (%)

(Standard Deviation)

Earnings in quarters 11 to 14 after RA (2015Q3$)

5,555

(9,366)

Any employment in quarters 11 to 14 after RA (%)

57.5

(49.5)

Number of quarters employed in quarters 11 to 14 after RA

1.7

(1.7)

UC = usual care.

RA = random assignment.

Notes: N = 746. See Appendix B for details on outcome specifications and values.

Source: Quarterly wage records from the National Directory of New Hires

3-Year Impacts of Housing and Services Interventions for Homeless Families

29

FAMILY OPTIONS STUDY

CHAPTER 3.

IMPACTS OF PERMANENT HOUSING

SUBSIDY (SUB) COMPARED WITH

USUAL CARE (UC)

This chapter presents estimates of the impact of the sites during the course of the followup period. Then, the core permanent housing subsidy (SUB) intervention rela-sections of the chapter present the 3-year effects of being

tive to usual care. The goal of the analyses presented

offered the SUB intervention (compared with usual care) on

in this chapter is to determine the extent to which being offered

outcomes within each of the study domains: housing stability,

a permanent housing subsidy on a priority basis (that is, not

family preservation, adult well-being, child well-being, and

having to enroll in and reach the top of waiting lists for subsidy

self-sufficiency. The final section is a summary of the 3-year

assistance) increases families’ housing stability and improves

impacts of the SUB intervention relative to usual care.

other family outcomes 3 years after receiving the priority offer.

The analysis presented in the Family Options Study: Short-Term

3.1. Permanent Housing Subsidy

Impacts of Housing and Services Interventions for Homeless

(SUB) Intervention

Families (hereafter, the Short-Term Impacts report; Gubits et al.,

2015) showed that assignment to the SUB intervention caused

The SUB intervention provided priority access to rental

striking improvements in housing stability relative to usual care

assistance without a time limit, typically in private-market

in the first 20 months after random assignment. The 20-month

housing.48 The permanent housing subsidies offered to families

impact analysis also found that, compared with usual care,

assigned to the SUB group could include housing placement

the benefits of having priority access to permanent housing

assistance but not ongoing social services. The SUB interven-

subsidies extended beyond housing stability to other outcome

tion was available in 10 of the 12 study sites. The subsidies

domains of family preservation, adult well-being, child

were provided by 18 local and state public housing agencies

well-being, and self-sufficiency. Assignment to the SUB inter-

(PHAs), with some sites having more than 1 participating PHA.

vention reduced child separations, increased adult well- being,

All the housing assistance included in the SUB intervention is

and reduced the number of schools that children attended.

considered permanent—that is, families can continue receiving

Assignment to the SUB intervention also reduced labor market

housing assistance as long as they remain eligible and follow

engagement 20 months after random assignment but improved

program rules, such as paying their share of the rent and living

food security and reduced economic stress. Longer-term

in housing that passes a housing-quality inspection. In all

effects beyond this window hold particular importance for the

sites, recipients of the subsidy had to participate in the annual

SUB-versus-UC comparison, given the long-term support that

recertification of their income that would determine their share

permanent housing subsidies provide.

of the rent and the amount of the housing assistance payment

This chapter begins with a brief description of the assistance

made to the owner of the housing.

offered to the families assigned to the SUB intervention. It

In 8 of the 10 sites, the SUB intervention provided priority

then reviews the extent to which families in both the SUB

access to a tenant-based voucher provided by one or more

and UC groups used permanent housing subsidies and other

PHAs through the Housing Choice Voucher (HCV) program,

housing and services programs available to them in the study

as shown in Exhibit 3-1. One site also offered some permanent

48 Chapter 6 of the Short-Term Impacts report (Gubits et al., 2015) provides additional details about the SUB intervention.contract with a property owner for specified units and for a specified term. Recipients of this type of assistance also pay 30 percent of adjusted monthly income for rent.

3-Year Impacts of Housing and Services Interventions for Homeless Families

30

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

Exhibit 3-1. Subsidy Type Provided by Site

Type of Subsidy

Number of Participating Subsidy Programs

Percent of Families Assigned to

With This Type of Subsidy

Subsidy Intervention of This Type (N = 599)

Tenant-based voucher

16

92

Project-based voucher

1

3

Public housing

1

6

Note: Percentages do not add to 100 because of rounding.

Sources: Program data; random assignment enrollment data

housing subsidies through public housing units,49 and another

community-based rapid re-housing (CBRR) and project-based

offered some project-based vouchers.50 Tenant-based vouchers

transitional housing (PBTH) interventions. In total, 1,139

made up 92 percent of all family referrals.

families took part in the test of the SUB intervention versus

usual care. These families all had the SUB intervention and

The HCV program is the federal government’s largest housing

usual care available to them at the time of random assignment

assistance program, providing rental subsidies to more than

and were assigned to one of these two interventions; 599

2 million households across the country.51 Participants in the

families were assigned to the SUB group, and 540 families were

study who were assigned to the SUB intervention, accepted by

assigned to the UC group.53 Of these 1,139 families, 896 (501

the PHA, and issued a voucher were free to use the voucher to

SUB families and 395 UC families), or 79 percent, responded

rent a housing unit of their choice in the private rental market

to the 37-month followup survey and are included in the SUB-

as long as it met HUD’s Housing Quality Standards and had a

versus-UC impact comparison reported in this chapter. The

rent that the PHA determined to be reasonable when compared

current section describes the extent to which these 501 SUB

with the rents of unassisted units in the same housing market.

families used the SUB intervention and other types of homeless

The voucher assistance subsidized the monthly rent for the

and housing assistance during the followup period. Parallel

unit, and the amount that the subsidy provided was the pay-

information is presented for the 395 included UC families.

ment standard established by the PHA (or the unit’s actual rent,

if lower) minus 30 percent of the family’s adjusted monthly

Exhibit 3-2 shows the use of eight types of homeless and

income.52

housing programs. The first column shows the percentage of

the 501 families assigned to the SUB intervention who ever

3.2. Program Use by Families in the

used each program type during the followup period. The top

row (shaded in the exhibit) shows the takeup of the permanent

Permanent Housing Subsidy (SUB)

housing subsidies offered to the SUB group by the families

Versus Usual Care (UC) Comparison

assigned to that group; 83.2 percent of families assigned to the

SUB group received the permanent subsidy they were offered

Each impact comparison in the study may be thought of as a

at some point during the 37-month followup period—meaning

distinct experiment or test. This chapter addresses only the

they successfully leased a housing unit with a voucher or

comparison between the SUB intervention and usual care,

moved into an assisted housing unit.54 The second column cor-

without reference to the families who were randomized to the

respondingly shows the percentage of the 395 families assigned

49 In Honolulu, Hawaii, the subsidy intervention consisted of 33 units of public housing provided by the state PHA and 10 units of tenant-based rental assistance provided by the City and County of Honolulu Department of Community Services. Public housing units are owned and managed by the PHA. Like voucher program participants, tenants in public housing pay 30 percent of adjusted monthly income for rent. The city’s tenant-based rental assistance program that provided five permanent housing subsidy units for the study operates much like the HCV program.

50 In Bridgeport, Connecticut, which together with New Haven, Norwalk, and Stamford made up one of the study sites, the subsidy intervention was provided through 15

units of project-based vouchers. PHAs can use up to 20 percent of their HCV program funding for project-based assistance, under which a PHA enters into an assistance contract with a property owner for specified units and for a specified term. Recipients of this type of assistance also pay 30 percent of adjusted monthly income for rent.

51 http://portal.hud.gov/hudportal/HUD?src=/program_offices/cfo/reports/2013/main_toc.

52 Payment standards are adjusted for the number of bedrooms in the unit. The total rent payment that the PHA uses to calculate the subsidy includes an estimate of the cost of utilities paid for by the tenant. Details regarding the calculation of housing assistance payments under the HCV program are in 24 CFR Part 982.505.

53 In the entire study, 746 families were randomly assigned to the UC group. Only 540 of these families had the SUB intervention available to them when they were randomized, however. Therefore, only those 540 UC families are part of the SUB-versus-UC comparison sample. All 599 families randomly assigned to the SUB group had usual care available to them when they were randomized, so all are part of the SUB-versus-UC comparison sample.

54 All percentages, means, and medians in the exhibit are weighted to adjust for survey nonresponse and, hence, as best as possible, represent the full experimental sample of 1,139 families. The findings on program use are thus in line with similarly weighted impact estimates provided later in the chapter. Appendix D provides details about the nonresponse analysis and adjustments.

3-Year Impacts of Housing and Services Interventions for Homeless Families

31

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

Exhibit 3-2. SUB Versus UC: Program Use Since RA

Percent Ever Used

Number of Months Used From

Percent Used in

From RA to 37-Month

RA to 37-Month Followup Survey,

Month of Followup

Type of Homeless or Housing Assistance

Followup Surveya

if Ever Used Type of Assistance

Survey Response

SUB

UC

SUB

UC

SUB

UC

Mean

Median

Mean

Median

Permanent housing subsidies offered to the SUB groupb

83.2

12.7

31.0

32.5

20.7

21.5

68.4

10.8

Rapid re-housingc

11.4

22.9

5.6

4.5

7.8

6.5

0.3

1.8

Transitional housingd

7.4

28.9

9.0

6.0

11.4

10.0

0.9

3.7

Permanent supportive housing

3.0

10.7

11.3

9.5

15.1

12.5

2.3

6.9

Public housing

1.6

10.3

22.7

24.5

19.3

18.5

1.4

7.6

Project-based vouchers/Section 8 projects

1.5

6.1

14.3

12.5

18.8

19.5

0.8

5.4

Any permanent housing subsidye

88.4

37.9

30.2

32.5

19.5

19.5

73.0

30.5

Emergency shelterf

84.4

89.5

3.0

2.0

4.6

2.9

1.7

5.4

No use of homeless or housing programsg

5.1

24.3

—

—-

—

—

24.1

59.1

N

501

395

—

—

—

—

501

395

SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

to the UC group who ever used each program type during the

families randomly assigned to the SUB group nor families

followup period.55 The top row of the second column shows

assigned to the UC group had priority access: permanent sup-

that 12.7 percent of the UC families received the permanent

portive housing, public housing, and project-based vouchers/

housing subsidy offered to the SUB group during the followup

Section 8 projects.56 Both SUB and UC families found their way

period, presumably through the regular process of coming off

to these programs. It is not surprising that that finding their

waiting lists and leasing units during the course of 3 years.

way to the programs was more common for the UC families

who did not have priority access to a permanent housing

The next five rows of the exhibit show participation in other

subsidy through the study’s SUB intervention. The seventh

nonshelter types of homeless and housing assistance programs.

row shows the use of any type of permanent housing subsidy,

For all programs other than SUB programs, use is higher for the

which includes the permanent subsidy programs offered to the

UC group than for the SUB group, presumably because the UC

SUB group, permanent supportive housing, public housing,

group did not have the SUB intervention easily available and

and project-based vouchers/Section 8 projects. Nearly two-

so turned to other types of assistance. The second and third

fifths (37.9 percent) of UC families obtained some form of

rows show use of rapid re-housing and transitional housing.

permanent subsidy during the followup period.

Continuing to focus for the moment on the first two columns

of overall participation figures, 23 percent of UC families found

The eighth row shows the use of emergency shelter during

their way to rapid re-housing assistance at some point during

the followup period. All study families started in emergency

the 37-month followup period, and 29 percent found their

shelter; however, the numbers in this row are largely based on

way to transitional housing. The next three rows of the exhibit

administrative data, which was missing for some of the study

show the use of permanent housing subsidies to which neither

families. Nonetheless, the exhibit shows a contrast between

55 The percentages in rows one through six and row eight of these columns are not mutually exclusive, because some families used more than one program type during the followup period. Row seven is a summary of rows one, four, five, and six.

56 Although project-based vouchers and Section 8 projects are distinct programs, the distinction is not relevant to this analysis; therefore, use of these programs is reported collectively.

3-Year Impacts of Housing and Services Interventions for Homeless Families

32

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

SUB and UC families. The ninth and final row shows the

Exhibit 3-3. Number of Months of Subsidy Receipt During

percentages of families in the SUB and UC groups who used

Followup Period by SUB Families Who Ever Used Offered

none of the six nonshelter types of programs shown in the

Permanent Subsidy

exhibit during the 37-month followup period and who did not

a

80

use emergency shelter from the 7th month after random as-

signment onward. These families made no use of the homeless

70

assistance system after their initial shelter stay. About 5 percent

60

of SUB families and one-fourth of UC families fall into this group.

50

The mean and median numbers of months of use for each

40

program type are also shown in the exhibit (third and fourth

30

columns for SUB families, fifth and sixth columns for UC

families) for only those families who ever used a given program

20

type.57 As one might expect, given that housing subsidies were

10

readily available to SUB families, the number of months of SUB

intervention use is higher for participating SUB families (medi-

Percent of SUB families who ever used SUB

0

an of 33 months) than for the 12.7 percent of UC families who

3 to 5.75 6 to 8.75

used the permanent housing subsidies offered to the SUB group

9 to 11.75

0.25 to 2.75

12 to 14.7515 to 17.7518 to 20.7521 to 23.7524 to 26.7527 to 29.7530 or more

by coming off waiting lists for assisted housing (median of 22

Number of months of SUB receipt

months). Almost none of the SUB families used the permanent

SUB = priority access to permanent housing subsidy.

subsidy for less than 12 months, and the vast majority used it

a Percentages are weighted for survey nonresponse to represent all families in compar-

ison sample.

for 31 months or more (shown in Exhibit 3-3). For most other

Note: N = 419.

program types, median durations of use are longer for the UC

Source: Family Options Study Program Usage Data

group than for the SUB group.

Whereas the previous columns consider all experience from

during each calendar month for the first 32 months after ran-

between randomization and the survey, the last two columns

dom assignment.59 It is not surprising that program use for SUB

consider the program use as of the month of the 37-month

families is dominated by permanent housing subsidies. More

survey. Although most outcomes in the report are expected to

notable is that the use of permanent housing subsidies contin-

be influenced by assistance received during the entire followup

ued at a high level throughout the followup period, falling off

period, some outcomes will be particularly strongly influenced

only slightly over time, suggesting that most families were able

by assistance received at the time of the followup survey

to comply with rules imposed by landlords and by the subsidy

response. The first row of the seventh column shows that 68

program. SUB families also made some use of other programs.

percent of SUB families received permanent housing subsidies

The bottom panel shows the proportions of program use over

offered to the SUB group in the month they responded to the

time for the UC families in the SUB-versus-UC comparison.

37-month followup survey. Whereas the use of temporary

Compared with the SUB group, the UC group made more

assistance programs had fallen off by the time of the followup

extensive use of programs other than permanent housing

survey, 31 percent of UC families compared with 73 percent of

subsidies following their initial stay in emergency shelter. The

SUB families were receiving some form of permanent subsidy.

UC families’ use of rapid re-housing had largely ended by

Most UC families (59 percent) were not participating in a

month 20, but their use of transitional housing continued past

homeless or housing program at the time they responded to the

this point, only beginning to decrease in the third year (month

followup survey compared with 24 percent of SUB families.

25 and following). Use of permanent housing subsidies steadily

Exhibit 3-4 provides a more detailed picture of the timing of

increased throughout the followup period for the UC group,

program use by the families in the SUB-versus-UC compari-

because families used regular waiting list processes rather than

son.58 The top panel shows the proportions of families within

the study’s priority access to gain admittance to those pro-

the SUB group who are receiving different types of assistance

grams; however, it remained well under one-half of the level of

57 Hence, 0 values are not used in computing the means, nor do they pull downward the medians of the various distributions.

58 Exhibit 3-4 is closely related to the analysis of the costs of this pairwise comparison presented in Chapter 9. The reader should note that the exhibit does not indicate the two-way flows of families moving into and out of these program types from month to month. Instead, it reflects only the overall usage level in a given month.

59 Month 32 is the latest month for which the study team has data for all the families who responded to the 37-month survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

33

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

Exhibit 3-4. SUB Versus UC: Program Use in Each Month of Followup Period

Panel A: Program Use of SUB Families for 32 Months After RA

100

Unknown/no program use

90

80

70

60

50

Any permanent housing subsidy

40

30

Rapid re-housing

20

Percent of families using program type in month

10

Transitional housing

Emergency shelter

0

0

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

Panel B: Program Use of UC Families for 32 Months After RA

100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

Rapid re-housing

20

Percent of families using program type in month

Transitional housing

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

Notes: Families who have more than one type of program use in a calendar month are counted fractionally in each type.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

34

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

use of permanent housing subsidies by the SUB families, who

3.3. Impacts on Housing Stability

got priority access to such programs. Appendix Exhibit H-1

in the Permanent Housing Subsidy

shows additional information about program use in the second

(SUB) Versus Usual Care (UC)

half of the followup period, from month 19 after random

assignment until the month of the 37-month survey response.

Comparison

Exhibits 3-2 and 3-4 show that most families assigned to the

Proponents of permanent housing subsidies view the lack of

SUB group used permanent housing subsidies after departing

housing affordability as the root cause of homelessness among

from emergency shelter. Nevertheless, a small proportion of

families and believe that, because families who experience

SUB families used other programs in addition to the program

homelessness are very poor, they are likely to require long-term

to which they were offered priority access, which is consistent

rental subsidies to become stably housed.60 The SUB-versus-

with the design of the study: families were not required to use

UC comparison in the current study provides a direct test of

the programs offered to the group to which they were assigned

this hypothesis by measuring the effects of making the SUB

and were also not forbidden from using other programs that

programs easily available to families compared with a situation

were available to them in their community. The intent of the

in which permanent housing subsidies are relatively difficult to

study was to maximize use of the programs offered in the as-

access in the near term.

signed group (in this case, maximize use of the SUB programs

At 20 months after random assignment, the priority access to a

by the SUB families) and thereby to create as wide a contrast as

permanent housing subsidy provided to the SUB group had a

possible between program mixes for the different assignment

strong positive effect on housing stability. What do the housing

groups (in this case, SUB versus UC). As shown in the exhibits,

stability effects of assignment the SUB intervention at 3 years

the mix of programs used was very different for the SUB group

after random assignment tell?

than for the UC group. The contrast in usage is large, with

83.3 percent of SUB families using the programs that were

Exhibit 3-5 shows the experimentally based evidence of effects

part of the SUB intervention compared with 12.9 percent of

on homelessness, housing independence, residential moves,

UC families. Considering all forms of permanent subsidy,

and housing quality at 37 months. All the rows of the exhibit

(permanent housing subsidies offered to the SUB group, per-

(and other impact exhibits in the balance of this report) have

manent supportive housing, public housing, and project-based

the same format. The first three columns of the exhibit provide

assistance), the contrast between the two groups is 88 percent

information about the SUB families—the number of families

compared with 38 percent, a somewhat narrower gap but still

with data on a particular outcome and the mean value and

large. This difference in the use of permanent housing subsidies

standard deviation of the outcome. The next three columns

by the SUB and UC groups is large enough that concerns

provide the corresponding information for the UC families

about the study’s ability to detect an impact in the presence of

included in this particular pairwise comparison.61 The seventh

nonparticipation and crossover are minimal.

column is the difference between the mean value (or propor-

tion) of the SUB families and the mean value (or proportion) of

As is conventional in random assignment analyses, the goal

the UC families, referred to as the impact of assignment to SUB

here is to estimate the intention-to-treat (ITT) impact—that

relative to usual care.62 Asterisks to the right of this column

is, the impact of offering a program to families, regardless

denote the statistical significance of the impact estimate, with

of whether they actually use that program (or some other

more asterisks indicating higher levels of statistical significance.

program). The remainder of this chapter reports estimated

The eighth column of the exhibit contains the standard error

impacts in the various outcome domains that—if statistically

of the impact estimates, which is used to test for statistical

significant—can be causally attributed to the offer of a perma-

significance and can be used to construct a confidence interval

nent housing subsidy to the families randomly assigned to the

around the impact estimate. The results are weighted to adjust

SUB group at the start of the followup period in contrast with

for survey nonresponse.

no such privileged access being available to UC families.

The last column shows the standardized effect size of the

impact, calculated by dividing the impact estimate by the stan-

dard deviation of the outcome for all study families assigned to

60 For further discussion of hypotheses about the SUB intervention, see Chapter 3 of the Short-Term Impacts report (Gubits et al., 2015).

61 The UC families in this comparison are those who could have been randomized to the SUB group. The mean values of outcomes for all UC families are shown in Chapter 2.

62 As explained in Appendix C, the mean values and the impact estimate are regression adjusted for baseline covariates.

3-Year Impacts of Housing and Services Interventions for Homeless Families

35

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

Exhibit 3-5. SUB Versus UC: Impacts on Housing Stability at 37 Months

SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the followup period

At least 1 night homelessb or doubled up (past 6 mo.) or in shelter in past

500

17.0

(36.7)

395

38.1

(48.5)

– 21.1***

(3.0)

– 0.37

12 months (%) [confirmatory]c

At least 1 night homelessb or doubled up in past 6 months (%)

500

15.8

(35.3)

395

34.0

(47.3)

– 18.2***

(2.9)

– 0.33

At least 1 night homelessb in past 6 months (%)

500

8.9

(27.8)

395

17.6

(37.8)

– 8.6***

(2.3)

– 0.20

At least 1 night doubled up in past 6 months (%)

501

11.4

(30.8)

395

28.0

(44.8)

– 16.6***

(2.7)

– 0.32

Any stay in emergency shelter in past 6 months (%) [Program Usage Data]

501

3.3

(17.1)

395

8.1

(26.9)

– 4.9***

(1.7)

– 0.15

Any stay in emergency shelter in months 21 to 32 after RA (%) [Program

501

4.6

(20.9)

395

18.8

(38.4)

– 14.3***

(2.3)

– 0.33

Usage Data]

Number of days homelessb or doubled up in past 6 months

499

18.6

(48.1)

393

47.5

(74.1)

– 28.9***

(4.5)

– 0.33

Number of days homelessb in past 6 months

500

8.7

(33.3)

395

18.9

(48.1)

– 10.2***

(2.9)

– 0.18

Number of days doubled up in past 6 months

500

11.7

(38.4)

393

33.3

(63.6)

– 21.5***

(3.7)

– 0.29

Housing independence

Living in own house or apartment at followup (%)

501

84.7

(35.9)

395

69.1

(46.3)

15.5***

(2.9)

0.29

Living in own house or apartment with no housing assistance (%)

500

14.2

(34.9)

392

40.0

(48.7)

– 25.8***

(3.1)

– 0.45

Living in own house or apartment with housing assistance (%)

500

70.4

(45.6)

392

29.2

(46.1)

41.2***

(3.2)

0.80

Number of places lived

Number of places lived in past 6 monthsd

501

1.3

(0.8)

395

1.6

(1.1)

– 0.2***

(0.1)

– 0.20

Housing quality

Persons per room

479

1.2

(0.6)

375

1.6

(1.2)

– 0.4***

(0.1)

– 0.31

Housing quality is poor or fair (%)

478

28.8

(44.8)

374

32.5

(46.8)

– 3.7

(3.2)

– 0.07

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-UC comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

the UC group.63 The standardized effect size is thus a measure

of the exhibit shows evidence for the confirmatory outcome of

of impact relative to natural variability in the outcome. Such

the study. The outcome is constructed from a combination of

standardized effect sizes are a conventional way to compare

survey and Program Usage Data: (1) at least 1 night in shelter

impact magnitudes across outcomes and domains with different

or a place not meant for human habitation or doubled up in

scales. For example, one may compare the standardized effect

the past 6 months (from the followup survey), or (2) at least

sizes for housing stability outcomes in Exhibit 3-4 with those

1 night in emergency shelter in the past 12 months (from the

for other outcomes in other domains shown in this chapter.

study’s Program Usage Data). Of the families assigned to the

Standardized effect sizes may also allow for the size of effects

UC group, 38 percent experienced one of these two situations.

found in this study to be compared with the size of effects on

For the SUB group, that proportion was 17 percent, represent-

similar outcomes that are defined differently in other studies.

ing a reduction in homelessness of 21 percentage points, more

than one-half of the homelessness measured by this outcome

Exhibit 3-5 shows that the SUB intervention reduced stays in

for the UC group. This impact is highly statistically significant

shelter or places not meant for human habitation in the period

(even after the adjustment for multiple comparisons applied to

before the 37-month survey by a large amount. The first row

this confirmatory outcome).64

63 The standard deviations for the entire UC group are shown in Chapter 2. The entire UC group is used in computing effect sizes so that the effect sizes across impact comparisons will have common metrics.

64 The study estimates impacts on this confirmatory outcome for each of the six paired comparisons and four pooled comparisons. Seven of these estimates (impacts in the six paired comparisons and the pooled SUB+CBRR+PBTH-versus-UC comparison) have been prespecified as “confirmatory tests.” A multiple comparison procedure is performed to compute adjusted p-values for these tests to reduce the possibility of chance findings of statistical significance. See Appendix C for a discussion of the study’s approach to the multiple comparisons problem, the role of the confirmatory outcome in this approach, and the details of the adjustment procedure.

3-Year Impacts of Housing and Services Interventions for Homeless Families

36

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

The next three rows present results for three outcomes con-

be missing for the SUB group and the UC group), leaving little

structed solely from survey data: (1) at least 1 night homeless

concern about bias in the comparison of stay rates between the

or doubled up in the past 6 months, (2) at least 1 night home-

two groups. The Program Usage Data show a gap beginning to

less in the past 6 months, and (3) at least 1 night doubled up in

emerge between the shelter use of SUB and UC families in the

the past 6 months. The impact estimates in these three rows of

third month after study entry, a gap that reaches 8 percentage

the exhibit show that, compared with usual care, the SUB in-

points by the fifth month, with 15 percent of SUB families having

tervention caused substantial, statistically significant reductions

at least 1 night in shelter (black line) compared with 23 percent

in all three of these survey-based measures of homelessness.65

of UC families (dark gray line). This slight lag in the emergence

The next row uses a different data source—largely Homeless

of a gap is consistent with the need for families assigned to

Management Information System (HMIS) data—to measure

the SUB group to have their incomes verified and to find and

the proportion of families using at least 1 night of emergency

lease a unit—see the rising rate of permanent housing subsidy

shelter during the same period of the 6 months before the

use along the light gray line in the exhibit. Families may have

survey response.66 Access to the SUB intervention reduced the

remained in shelter during that process. A gap of 6 to 9 per-

proportion by more than one-half, from 8 to 3 percent.67

centage points remains through the 24th month. From the 10th

month onward, the share of SUB families in shelter is much less

The sixth row of Exhibit 3-4 shows the impact on any stay

than one-half the proportion of UC families.

in emergency shelter in months 21 to 32 after random

assignment, which is the latest 12-month period for which data

The last three homelessness outcomes examined (see again

are available for every survey respondent family.68 About 19

Exhibit 3-5) measure the number of days in the past 6 months

percent of UC families spent at least 1 night in emergency shel-

that a family was homeless or doubled up. Assignment to the

ter during this period. Only 5 percent of SUB families spent at

SUB group reduced the amount of time spent homeless or dou-

least 1 night in emergency shelter during this time, a reduction

bled up by an average of 4 weeks in the past 6 months relative

of 14 percentage points. Thus, with the availability of a housing

to assignment to the UC group.69

subsidy, the shelter usage rate in this period was reduced by

The housing independence outcomes in the next panel of

three-fourths.

Exhibit 3-5 measure whether a family lived in its own house

Exhibit 3-6 provides a more detailed characterization of the

or apartment at the time of the followup survey, either with or

timing of emergency shelter stays for SUB and UC families.

without housing assistance. The SUB intervention increased the

For reference, it also shows month-by-month usage of any

proportion of families living in their own dwelling place from

permanent housing subsidy in the two groups.

69 to 85 percent relative to usual care. This difference is the

net result of two opposing effects. As would be expected, the

The Program Usage Data are missing the initial stay in emergency

proportion of SUB families living in their own places without

shelter for about 20 percent of families. The study team has no

housing assistance (14 percent) was much lower than the

reason to believe, however, that missing data rates are associated

corresponding proportion for UC families (40 percent). By

with randomly assigned group (that is, data are equally likely to

contrast, and more than offsetting that effect, the proportion of

65 All impacts in this table, with the exception of the first row, are considered exploratory and are not adjusted for the presence of multiple comparisons. Likewise, all impacts in other study domains are also considered exploratory.

66 Outcomes regarding shelter stays are based on study Program Usage Data; these outcomes are described in Appendix A.

67 The proportions of families using emergency shelter during the past 6 months are less than one-half of those reporting being homeless in the past 6 months. This discrepancy is likely due to two factors: (1) some of the homelessness captured in the survey measure is for stays in places not meant for human habitation rather than in emergency shelter, and (2) measurement error in the survey (recall bias) and Program Usage Data (lack of coverage of all local shelters in HMIS records and lack of coverage for families who moved away from the community where they enrolled in the study). The relative importance of these factors is unknown.

68 Although most families have longer followup than 32 months, the families with the shortest survey followup periods responded to the survey in their 32nd month after random assignment. Therefore, Program Usage Data for survey respondents are available for the full respondent sample only through month 32 after random assignment.

69 Dividing the average number of days spent homeless or doubled up in the past 6 months for SUB families by the percentage who experienced either state (18.6

days/0.158 = 117.7 days) reveals that those who did experience either state spent 118 days, on average, either homeless or doubled up in the past 6 months. Performing the same calculation for UC families (47.5 days/0.34 = 139.7 days) reveals that UC families who experienced either state spent more time (139 days), on average, either homeless or doubled up in the past 6 months than did families assigned to the SUB group. These calculations show that about 1 week of the overall 4-week difference is explained by the difference in proportions experiencing homelessness or doubled up, and the other 3 weeks are explained by a difference in the average length of time spent homeless or doubled up conditional on having experienced either state.

3-Year Impacts of Housing and Services Interventions for Homeless Families

37

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

Exhibit 3-6. SUB Versus UC: Percent of Families With Any Stay in Emergency Shelter and Any Permanent Housing Subsidy During Month, by Month After RA

100

90

80

SUB: Used any permanent housing subsidy

70

60

50

Percent of families

40

UC: Used any permanent housing subsidy

30

20

UC: Used emergency shelter

SUB: Used emergency shelter

10

0

0

5

10

15

20

25

30

35

Month after RA

SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

Notes: Percentages are weighted for survey nonresponse to represent all families in the SUB-versus-UC comparison sample. Missing data on emergency shelter stays bias the percentages somewhat downward. The baseline stay in emergency shelter does not appear in the data for 16.7 percent of UC respondent families. The missing data rate for subsequent stays in emergency shelter is unknown. Any permanent housing subsidy includes the permanent subsidy programs offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

Source: Family Options Study Program Usage Data

SUB families living in their own places with housing assistance

of moves during the final 6 months of the followup period

(70 percent) was much higher than the corresponding propor-

by almost one-half (42 percent, when using more digits than

tion for UC families (29 percent).70

shown in Exhibit 3-5).71

The stability offered by the SUB intervention also reduced the

The last two rows in Exhibit 3-5 show how the SUB inter-

average number of places where families lived during the past

vention affected the nature of the housing occupied by study

6 months from 1.6 to 1.3. Because this outcome has a lower

families as of the 37-month followup survey by considering

bound of 1 (the family had to have lived in at least one place),

the number of persons per room (a measure of crowding) and

the UC group mean of 1.6 compared with 1.3 for the SUB

residence in poor-quality housing. The number of persons

group means that the SUB intervention reduced the number

per room is a standard proxy for overcrowding. The SUB

70 Although the survey response indicates that 70 percent of the SUB families were living in their own house or apartment with housing assistance at the time of the survey, the Program Usage Data show that the proportion of families using permanent housing subsidies offered to the SUB group, public housing, or project-based vouchers in the survey month is 73 percent. This discrepancy between response to the survey item and the Program Usage Data (largely based on HUD administrative records for these program types) suggests some measurement error in one or both of these data sources.

71 This outcome counts each place in which the family lived only once. Thus, it is not technically the same as the number of moves plus one additional place lived. It is possible for a family to move out of a place (for example, a housing unit shared with friends or relatives) and then move back into the same unit during the 6-month period. Its interpretation as a measure of housing instability, however, is essentially the same.

3-Year Impacts of Housing and Services Interventions for Homeless Families

38

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

intervention reduced the number of persons per room from 1.6

At 20 months after random assignment, relative to usual

to 1.2. Nearly one-third (31 percent) of UC families reported

care, assignment to the SUB intervention reduced recent and

poor or fair housing at the 37-month survey, as did 28 percent

ongoing child separations and foster care placements among

of families assigned to the SUB group; the difference is not

children who were with the family in shelter at baseline.

statistically significant.

Exhibit 3-7 shows that no such effects were evident 37 months

after random assignment.

In sum, the SUB intervention had a strong, positive effect on

housing stability compared with usual care for every measure

At 37 months, assignment to the SUB intervention increased

considered, except for housing quality.

separations of spouses and partners who had been with the

family in shelter at baseline (a result that was not found at the

3.4. Impacts on Family Preservation

earlier followup point). Of SUB families, who had a spouse or

partner present at baseline, 48 percent experienced a separation

in the Permanent Housing Subsidy

of that spouse in the past 6 months at the latter followup point,

(SUB) Versus Usual Care (UC)

compared with 34 percent of UC families. It is not clear wheth-

Comparison

er this increase in separations is beneficial or detrimental to

SUB families. No impacts were detected at either time point on

Any effects of assignment to the SUB intervention on family

family reunifications with children or partners among the much

preservation would be expected to be indirect, via the effects

smaller group of families separated in these ways at baseline.

on housing stability. To test for effects of priority access to a

permanent housing subsidy on family preservation, Exhibit 3-7

Exhibit 3-8 reports impacts on outcomes from the child welfare

reports estimated impacts on family preservation from the SUB-

agency administrative data collected in 5 of the 12 study sites.

versus-UC comparison.

No effect is detected on either the proportion of family heads

Exhibit 3-7. SUB Versus UC: Impacts on Family Preservation at 37 Months

SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

486

13.5

(34.1)

389

16.9

(35.9)

– 3.3

(2.6)

– 0.08

Family has at least one foster care placement in past 6 monthsb (%)

492

3.0

(17.2)

392

4.2

(18.6)

– 1.1

(1.4)

– 0.05

Spouse/partner separated in past 6 months, of those with spouse/partner

125

47.8

(49.9)

116

34.3

(48.7)

13.4**

(6.5)

0.23

present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at least one

95

44.1

(49.9)

78

37.3

(48.6)

6.8

(8.0)

0.12

child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent at RA (%)

51

29.3

(45.1)

34

17.2

(41.0)

12.1

(10.9)

0.26

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 3-8. SUB Versus UC: Impacts on Child Welfare Outcomes

SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Had a formal child separation that began after RA (%)

262

8.8

(27.8)

250

10.7

(31.6)

– 1.9

(2.6)

– 5.80

Total days during followup separated from at least one childb

262

52.9 (182.9)

250

71.2 (228.7)

– 18.3

(15.8)

– 0.07

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days.

Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions. Sample limited to five sites where child welfare records were collected (Alameda County, Baltimore, Kansas City, Minneapolis, and Phoenix).

Source: State child welfare agency records

3-Year Impacts of Housing and Services Interventions for Homeless Families

39

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

with a formal child separation that began after random assign-

Reductions in psychological distress and intimate partner

ment or on the total days during the followup period of being

violence for SUB families were also evident at 20 months after

separated from at least one child.

random assignment (the Short-Term Impacts report [Gubits et

al., 2015]; Exhibit 3-9). At that earlier point, relative to usual

3.5. Impacts on Adult Well-Being

care, improvements in three other outcomes—goal-oriented

thinking, alcohol dependence, and the combined measure

in the Permanent Housing Subsidy

of alcohol dependence and drug abuse—were detected. No

(SUB) Versus Usual Care (UC)

effects on these outcomes were found 37 months after random

Comparison

assignment.

The SUB intervention did not include services related to adult

3.6. Impacts on Child Well-Being

well-being.72 Even so, Exhibit 3-9 provides evidence of two

longer-run impacts in this domain. At the 37-month point,

in the Permanent Housing Subsidy

having priority access to a permanent housing subsidy reduced

(SUB) Versus Usual Care (UC)

psychological distress, relative to usual care, by about one-

Comparison

tenth of a standard deviation and intimate partner violence by

one-third. In combination with the effect of assignment to the

Any effects of assignment to the SUB intervention on child

SUB intervention on increased separation from partners (shown

well-being would be expected to be indirect and to occur

in the previous section), it is plausible that access to permanent

through the substantial effect of priority access to a permanent

housing subsidies enabled family heads to leave abusive

housing subsidy on housing stability. At 20 months, having

relationships. This interpretation is consistent with qualitative

priority access to a permanent housing subsidy led to positive

evidence from the evaluation presented in the Effects of Housing

effects relative to usual care on two outcomes—reducing chil-

Vouchers on Welfare Families.73

dren’s recent absences from school and reducing movements

Exhibit 3-9. SUB Versus UC: Impacts on Adult Well-Being at 37 Months

SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

500

32.2

(46.5)

394

29.1

(45.5)

3.1

(2.9)

0.06

Adult mental health

Goal-oriented thinkingb

495

4.45

(1.04)

394

4.51

(0.95)

– 0.06

(0.07)

– 0.05

Psychological distressc

499

6.69

(5.56)

393

7.42

(5.80)

– 0.73*

(0.38)

– 0.11

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

496

22.7

(41.6)

391

23.6

(42.0)

– 0.8

(2.9)

– 0.02

Adult substance use

Alcohol dependence or drug abused (%)

500

12.3

(32.3)

393

14.4

(35.3)

– 2.1

(2.4)

– 0.06

Alcohol dependenced (%)

501

8.8

(28.0)

394

10.7

(30.9)

– 1.9

(2.1)

– 0.06

Drug abused (%)

500

4.3

(19.6)

394

6.3

(24.4)

– 2.0

(1.6)

– 0.08

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

498

7.8

(26.3)

392

11.8

(32.5)

– 4.0*

(2.2)

– 0.12

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

72 Services related directly to housing, such as assistance with searching for housing and negotiating with landlords, were permitted but not required.

73 Mills et al. (2006).

3-Year Impacts of Housing and Services Interventions for Homeless Families

40

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

among schools—but, overall, it appeared to have little effect

3.7. Impacts on Self-Sufficiency in

on child well-being. At 37 months, the study team found a

the Permanent Housing Subsidy

handful of favorable effects and one single adverse effect.

(SUB) Versus Usual Care (UC)

Considering first the outcomes measured across age groups

Comparison

(Exhibit 3-10), for children in the SUB families, the analysis

continues to find a reduction in the number of schools attended

By increasing housing stability, assignment to the SUB interven-

since random assignment relative to children in UC families.

tion may plausibly have indirect effects on family self-sufficiency

The reduction—one fewer school move for every six children—

relative to usual care. In particular, the opportunity to obtain

is not independent of the effect observed at 20 months,

stable housing with a sharply and permanently lower burden

because this outcome measures school mobility since random

for housing costs may enable adult family members to transfer

assignment. In addition, children in the SUB group had fewer

attention from staying housed to issues of employment and

sleep problems, fewer behavior problems, and more prosocial

earnings and even enhancing their skills through education

behavior at 37 months, all based on the parental report. The

and training participation. On the other hand, the ability to

size of these effects, scored on multi-item scales, ranges from

obtain housing with limited out-of-pocket costs (30 percent

0.11 to 0.13 standard deviations.

of income) makes available household financial resources go

further—lessening the pressure to work, earn, and acquire

Exhibit 3-11 shows estimated impacts on outcomes measured

new skills and education. Unlike the other active interventions

by age group. Young children in families assigned to the SUB

studied in this report, the programs offered to families assigned

group have more positive childcare or preschool attitudes and

to the SUB intervention do not include case management

older children have more positive school attitudes than their

guidance or referrals to services intended to increase skills or

counterparts in the UC families. Children ages 3.5 to 7 years

encourage work.

in SUB families, however, have lower executive functioning as

measured by the Head Toes Knees Shoulders assessment, by

The Short-Term Impacts report (Gubits et al., 2015) found that,

0.12 standard deviations.

in the first 20 months after random assignment, assignment

to the SUB intervention reduced some important aspects of

Overall, across 29 outcomes in the child well-being domain,

family self-sufficiency relative to usual care and increased

7 effects reach statistical significance, 6 of them favoring the

others. Assignment to the SUB intervention caused reductions

SUB group, indicating a positive effect of the SUB intervention

in the proportion of family heads working at the time of the

compared with the usual care on child well-being.

Exhibit 3-10. SUB Versus UC: Impacts on Child Well-Being Across Age Groups at 37 Months SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

588

1.9

(0.9)

460

2.0

(0.9)

– 0.2**

(0.1)

– 0.13

Grade completion (not held back) (%)

485

92.53 (27.54)

377

89.54 (30.15)

2.99

(2.20)

0.08

School gradesc

443

3.1

(0.9)

332

3.1

(0.8)

0.0

(0.1)

0.04

Child physical health

Poor or fair health in past 30 days (%)

678

7.9

(25.2)

532

5.9

(23.8)

2.0

(1.8)

0.07

Well-child checkup in past year (%)

677

88.8

(31.2)

529

91.5

(29.0)

– 2.7

(2.0)

– 0.07

Child has regular source of health care (%)

677

91.6

(26.2)

530

90.9

(28.5)

0.6

(2.4)

0.02

Sleep problemsd

678

1.99

(1.04)

533

2.19

(1.09)

– 0.19** (0.08)

– 0.13

Child behavioral strengths and challenges

Behavior problemse

638

0.44

(1.26)

506

0.67

(1.26)

– 0.23**

(2.4)

– 0.06

Prosocial behaviorf

640 – 0.15

(1.11)

509 – 0.30

(1.09)

0.16**

(2.1)

– 0.06

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

3-Year Impacts of Housing and Services Interventions for Homeless Families

41

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

Exhibit 3-11. SUB Versus UC: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

211

35.6

(48.9)

170

42.3

(49.6)

– 6.7

(5.5)

– 0.11

Child care or preschool absences in past monthd

86

0.86

(0.96)

71

0.75

(0.84)

0.11

(0.16)

0.09

Positive child care or preschool experiencese

89

0.89

(0.37)

73

0.81

(0.40)

0.08

(0.06)

0.16

Positive child care or preschool attitudesf

90

4.74

(0.70)

72

4.53

(0.73)

0.21** (0.10)

0.25

Child care or preschool conduct problemsg (%)

93

7.3

(29.7)

74

4.5

(19.9)

2.9

(4.3)

0.10

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

183

72.0

(44.1)

154

73.5

(45.9)

– 1.6

(5.9)

– 0.03

Ages 3 years, 6 months to 7 years

Verbal abilityi

254 – 0.18

(1.01)

184 – 0.34

(1.03)

0.16

(0.14)

0.12

Math abilityj

253 – 0.21

(0.99)

186 – 0.25

(0.93)

0.04

(0.11)

0.04

Executive functioningk (self-regulation)

248

16.08 (16.19)

179

18.45 (16.63)

– 2.36*

(1.27)

– 0.12

Ages 5 to 17 yearsl

School enrollmentc (%)

470

98.6

(11.2)

365

97.3

(14.7)

1.3

(1.2)

0.07

School absences in past monthd,m

195

0.93

(0.91)

139

1.01

(1.00)

– 0.08

(0.13)

– 0.06

Positive school experiencese,m

196

0.56

(0.58)

140

0.51

(0.62)

0.05

(0.08)

0.05

Positive school attitudesf,m

195

4.21

(1.09)

140

3.98

(1.14)

0.22*

(0.13)

0.15

School conduct problemsg,m (%)

197

22.9

(41.1)

142

30.8

(46.1)

– 7.9

(5.6)

– 0.13

Ages 8 to 17 years

Anxietyn

285

35.27

(7.51)

221

35.07

(7.85)

0.20

(0.75)

0.02

Fearso

287

63.31 (14.34)

222

62.02 (14.84)

1.29

(1.27)

0.07

Substance usep (%)

283

4.20 (20.19)

219

6.91 (26.82)

– 2.71

(2.24)

– 0.07

Goal-oriented thinkingq

277

21.96

(5.10)

214

22.10

(4.86)

– 0.13

(0.54)

– 0.02

School effort in past monthr

283

2.82

(0.77)

218

2.86

(0.78)

– 0.04

(0.08)

– 0.04

Arrests or police involvement in past 6 monthss (%)

152

9.24 (27.05)

123

10.57 (28.65)

– 1.32

(3.99)

– 0.03

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Notes: Impact estimates and outcome means are regression-adjusted for baseline characteristics and are weighted to adjust for survey non-response. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

3-Year Impacts of Housing and Services Interventions for Homeless Families

42

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

20-month survey compared with usual care, proportion with

Exhibit 3-12 shows effects on self-sufficiency outcomes for the

any employment since random assignment, and average num-

SUB-versus-UC comparison during the longer, 37-month fol-

ber of months worked since random assignment.

lowup period. Of the 22 outcomes examined, 8 had statistically

significant effects. No effects are evident on either working

Even with the reduction in work effort, however, SUB families

for pay at the time of the survey (first row) or the proportion

appeared to be in a better financial position than UC families

of families who had any earnings in the past month (ninth

20 months after random assignment: the additional resources

row); however, during the period between the 20-month and

represented by the housing subsidy improved the food security

37-month surveys, the study team finds a reduction in work

of SUB families and decreased economic stress.

effort in the proportion who performed any work for pay (a

Exhibit 3-12. SUB Versus UC: Impacts on Self-Sufficiency at 37 Months

SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

500

35.7

(48.2) 395

36.6

(48.3)

– 0.9

(3.2)

– 0.02

Any work for pay since 20-month surveyb (%)

466

58.2

(49.3) 346

64.1

(48.3)

– 5.9*

(3.3)

– 0.12

Months worked for pay since 20-month surveyb,c

464

6.6

(7.8) 343

7.5

(8.0)

– 1.0*

(0.6)

– 0.12

Any work for pay since RA (%)

500

67.9

(46.2) 394

73.7

(44.8)

– 5.7**

(2.8)

– 0.11

Months worked for pay since RAc

494

11.4

(13.3) 386

13.1

(13.3)

– 1.7**

(0.8)

– 0.11

Hours of work per week at current main jobd

498

10.6

(16.2) 395

11.7

(16.5)

– 1.0

(1.1)

– 0.05

Income sources and amounts

Annualized current earnings ($)

491

5,817 (10,173) 387

6,066 (9,720) – 249

(676)

– 0.02

Total family income ($)

470 10,933 (9,420) 383 11,816 (10,387) – 883

(689)

– 0.07

Anyone in family had earnings in past month (%)

501

45.8

(49.9) 395

49.3

(50.1)

– 3.6

(3.3)

– 0.06

Anyone in family received TANF in past month (%)

501

29.6

(45.4) 395

25.2

(44.2)

4.4

(2.9)

0.09

Anyone in family received SSDI in past month (%)

498

7.8

(26.6) 395

9.0

(30.2)

– 1.2

(1.8)

– 0.04

Anyone in family received SSI in past month (%)

501

14.0

(34.7) 395

13.6

(34.4)

0.4

(1.8)

0.01

Anyone in family received SNAP/Food Stamps in past month (%)

501

82.2

(38.6) 395

81.4

(38.9)

0.8

(2.7)

0.02

Anyone in family received WIC in past month (%)

501

25.4

(43.4) 395

24.4

(43.8)

1.0

(2.9)

0.02

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

499

38.2

(48.7) 394

40.6

(49.0)

– 2.3

(3.3)

– 0.04

Number of weeks in school/training programs since RA

496

7.3

(14.7) 389

8.9

(17.5)

– 1.6

(1.1)

– 0.08

Participated in 2 weeks or more of school since RA (%)

499

13.0

(33.2) 394

12.1

(32.2)

0.8

(2.3)

0.02

Participated in 2 weeks or more of basic education since RA (%)

499

2.5

(15.3) 394

2.7

(17.2)

– 0.2

(1.0)

– 0.01

Participated in 2 weeks or more of vocational education since RA (%)

499

10.4

(31.6) 394

16.5

(36.5)

– 6.1***

(2.3)

– 0.15

Food security

Household is food insecure (%)

501

38.9

(48.9) 395

48.5

(50.0)

– 9.6***

(3.5)

– 0.17

Food insecurity scalee

499

1.58

(2.00) 394

2.04

(2.11) – 0.46*** (0.14)

– 0.19

Economic stressors

Economic stress scalef

497

– 0.22

(0.44) 394

– 0.10

(0.49) – 0.12*** (0.03)

– 0.21

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

3-Year Impacts of Housing and Services Interventions for Homeless Families

43

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

reduction of 6 percentage points) and in the average number

not include the value of the housing subsidy provided by the

of months worked in this period (a reduction of 1 month).74

voucher or Supplemental Nutrition Assistance Program (SNAP)

During the entire followup period since random assignment,

benefits. The analysis also finds no effect on receipt of public

having priority access to a permanent housing subsidy reduced

assistance through Temporary Assistance for Needy Families;

the proportion with any work for pay by 6 percentage points

SNAP; Special Supplemental Nutrition Program for Women,

and reduced the average number of months worked by about

Infants, and Children; Social Security Disability Insurance; or

2 months compared with usual care.

Supplemental Security Income benefits.

The Short-Term Impacts report (Gubits et al., 2015) found a

The third section of Exhibit 3-12 shows impacts on education

reduction in working for pay and in the proportion of families

and training since random assignment. Having priority access

who had earnings at 20 months after random assignment. The

to a permanent housing subsidy did not cause a detectable

37-month analysis does not find statistically significant impacts

impact on the proportion of family heads who participated

on these two outcomes. Given our relatively small sample sizes,

in any type of school or training during the 37-month period

however, the study team is unable to confirm that the effects

but did appear to cause a reduction in the share that engaged

of assignment to the SUB intervention in these areas at the

in vocational education (10 percent of SUB families compared

37-month point are smaller than they were at the 20-month point.

with 16 percent of UC families).

Administrative data provide another opportunity to examine

Although no effect on total income was evident, the financial

intervention effects on earnings and employment of family heads.

position of SUB families appears to have improved with access

Exhibit 3-13 shows impact estimates on outcomes for the 11th

to a permanent housing subsidy compared with UC families at

through 14th calendar quarters following the quarter of random

37 months after random assignment. Assignment to the SUB

assignment. Assignment to the SUB group relative to assignment

intervention lowered the share of households classified as food

to the UC group reduced the proportion of family heads with

insecure from 49 to 39 percent and reduced the average score

any employment during this year from 57.5 percent to 52 percent.

on a food insecurity scale by a 0.19 standardized effect size.

The magnitude of this effect on employment is consistent with

These results are not surprising: total household resources—

the effects found in survey data on employment over the 37-month

including both cash income (which did not differ between

followup period and in the period since the 20-month survey.

groups) and the value of the housing subsidies—were much

The analysis of administrative data finds no effect on either

higher for the SUB families. Some of these additional resources

total earnings during the year or the number of quarters em-

appear to have been spent on food, thus decreasing food

ployed (that is, with positive earnings) during the year.

insecurity. Assignment to the SUB intervention also reduced

economic stress (measured through survey items that asked

The second section of Exhibit 3-12 shows no longer-run effect

about the frequency of not being able to afford rent, medical

of assignment to the SUB group on total family income in

care, clothing, and so on) by 0.12 points on a scale from -1 to 1.

the previous calendar year. The family income measure does

Exhibit 3-13. SUB Versus UC: Earnings and Employment

SUB

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Earnings in quarters 11 to 14 after RA (2015Q3$)

579

5,658 (9,005) 517

5,555 (9,366)

103

(559)

0.01

Any employment in quarters 11 to 14 after RA (%)

579

52.0

(50.0) 517

57.5

(49.5)

– 5.5*

(3.0)

– 0.11

Number of quarters employed in quarters 11 to 14 after RA

579

1.6

(1.7) 517

1.7

(1.7)

– 0.1

(0.1)

– 0.05

SUB = priority access to permanent housing subsidy. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Note: See Chapter 2 and Appendix B for outcome definitions.

Source: Quarterly wage records from the National Directory of New Hires

74 Outcomes for the period between the surveys are available for analysis for only those family heads who responded to both the 20-month and 37-month waves of the surveys. Measures specific to the period between the 20-month and 37-month surveys were collected for only those who had been 20-month survey respondents. Family heads who did not respond to the 20-month survey were asked questions about the entire followup period since random assignment, reporting information that could not be broken out by a narrower time window.

3-Year Impacts of Housing and Services Interventions for Homeless Families

44

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

3.8. Summary of the Permanent

The Short-Term Impacts report (Gubits et al., 2015) found no-

Housing Subsidy (SUB) Versus Usual

table indirect benefits of having priority access to a permanent

Care (UC) Comparison Across

housing subsidy for several family preservation indicators

and adult well-being measures 20 months after random

Domains

assignment. A few of these indirect effects are evident 3 years

after random assignment. At that point, having priority access

Having priority access to a permanent housing subsidy pro-

to a permanent housing subsidy reduced psychological distress

duced a notable difference in program use during the 3 years

and intimate partner violence relative to usual care, the latter a

after random assignment between SUB and UC families. Of all

one-third reduction. In the family preservation domain, short-

families assigned to the SUB group, 83 percent received the

term reductions in child separations and foster care placements

housing subsidies offered to families assigned to the SUB group

are not detected at 37 months; however, assignment to the SUB

at some point in the followup period compared with 13 percent

group at that point caused more separations between spouses

of families assigned to the UC group. On a broader measure of

and partners relative to usual care, an effect not detected in the

receipt of any type of permanent housing subsidy, the contrast

20-month analysis.

was also large: 88 percent for the SUB group compared with 38

percent for the UC group.

Some positive effects of priority access to a permanent housing

subsidy on child development are evident at 37 months. The

This contrast in program use led to notable impacts on the

analysis continues to find a reduction in the number of schools

housing experiences of SUB families compared with UC fami-

attended since random assignment and also newly evident

lies at 37 months after random assignment. The most striking

effects of more positive school attitudes, fewer sleep problems,

effect of assignment to the SUB intervention relative to usual

fewer behavior problems, and more prosocial behavior. No

care was its prevention of a substantial share of subsequent

effects were found on direct measures of verbal and math

stays in shelter or places not meant for human habitation. Of

ability for younger children or on self-reported outcomes of

families assigned to the SUB group, 17 percent spent at least

mental health, school effort, and delinquent behavior for older

1 night homeless or doubled up in the 6 months before the

children. The study team finds one adverse effect of priority

37-month followup survey or at least 1 night in emergency

access to a permanent housing subsidy on the executive

shelter in the 12 months before the survey compared with

functioning of children ages 3.5 to 7 years. This single adverse

38 percent of families assigned to the UC group. That is,

effect is difficult to interpret in light of the other detected

assignment to the SUB group after 7 days in emergency shelter

effects of assignment to the SUB intervention, all of which are

reduced subsequent homelessness by more than one-half. As-

improvements in well-being.

signment to the SUB group also caused substantial, statistically

significant reductions in all other measures of homelessness

In the self-sufficiency domain, the study team finds evidence

at the 3-year followup point: the proportion of families who

that having priority access to a permanent housing subsidy

experienced (1) at least 1 night in a shelter or a place not

reduced work effort during the full 3-year followup period and

meant for human habitation in the past 6 months, (2) at least

during the second half of the followup period. The team did

1 night doubled up in the past 6 months, (3) at least 1 night in

not find evidence of a reduction in the proportion of family

a shelter or places not meant for human habitation or doubled

heads working for pay at the time of the 37-month survey or

up in the past 6 months, and (4) at least one stay in emergency

of a reduction in the proportion of families who had earnings

shelter in months 21 to 32 after random assignment.

income at that point—both of which were observed 20 months

after random assignment. Given the relatively small sample siz-

The effects of priority access to a permanent housing subsidy

es, the team was unable to confirm that the effect of assignment

were also evident at the 3-year followup point for measures of

to the SUB intervention on work effort at the 37-month point

housing independence. Altogether, 85 percent of SUB families

was smaller than it was at the 20-month point. The 37-month

were living in their own house or apartment compared with 69

analysis finds no effect of priority access to a permanent

percent of UC families. As expected, however, the proportion

housing subsidy on receipt of any type of public assistance or

of SUB families living in their own dwelling place without

disability benefits.

housing assistance (14 percent) is much lower than the corre-

sponding proportion of UC families (40 percent). Assignment

Families assigned to the SUB group appear be in a better finan-

to the SUB intervention also reduced the average number of

cial position than UC families 3 years after random assignment.

places where families lived in the past 6 months and housing

Without counting housing assistance as income, no difference

crowding relative to usual care. In contrast with the short-term

in income is evident however, the housing assistance is valu-

findings, no improvement in housing quality was evident.

able. The freed-up resources appear to lead to an improvement

3-Year Impacts of Housing and Services Interventions for Homeless Families

45

FAMILY OPTIONS STUDY

Chapter 3. Impacts of Permanent Housing Subsidy (SUB) Compared With Usual Care (UC)

in the food security of SUB families, lowering the percentage of

random assignment in multiple domains. Families assigned to

households classified as food insecure from 36 to 28 percent

the SUB group at that point had greater housing stability, less

relative to UC families, and to a decrease in economic stress.

psychological distress, less intimate partner violence, better

child outcomes, less food insecurity, and less economic stress

Overall, evidence suggests that families assigned to the SUB

than their counterparts assigned to the UC group. Chapter 6

group continue to benefit substantially from having priority

addresses how the SUB intervention compares with the two

access to permanent housing subsidy assistance 3 years after

other active interventions: CBRR and PBTH.

3-Year Impacts of Housing and Services Interventions for Homeless Families

46

FAMILY OPTIONS STUDY

CHAPTER 4.

IMPACTS OF COMMUNITY-BASED

RAPID RE-HOUSING (CBRR) COMPARED

WITH USUAL CARE (UC)

This chapter presents estimates of the impact of the chapter present the 3-year effects of being assigned to the assignment to the community-based rapid re-housing

CBRR group (compared with being assigned to the UC group)

(CBRR) intervention relative to usual care in the study

on outcomes within each of the study domains: housing sta-

communities. The goal of the analyses presented in this chapter

bility, family preservation, adult well-being, child well-being,

is to determine the extent to which an offer of temporary rental

and self-sufficiency. The final section summarizes the 3-year

assistance to help families exit shelter rapidly increases families’

impacts of the CBRR intervention relative to usual care.

housing stability and improves other family outcomes 3 years

after receiving the offer.

4.1. Community-Based Rapid

At 20 months after random assignment, assignment to the

Re-Housing (CBRR) Intervention

CBRR group did not affect housing stability, family preserva-

tion, or adult well-being compared with assignment to the UC

The CBRR intervention provides program participants with

group. Assignment to the CBRR group may have had some

priority access to temporary rental assistance (usually for 7 to

consequences for children compared with assignment to the

8 months) and limited services focused on housing search

UC group, but the indications were limited and mixed in

assistance, self-sufficiency, and basic services coordination.76

direction. Assignment to the CBRR group did lead to improved

The programs studied largely conformed to standards that

family income in the year before the 20-month survey (annual

HUD and the U.S. Interagency Council on Homelessness later

family income was about $1,100 higher for families assigned to

established for rapid re-housing programs in guidance issued

the CBRR group), greater food security, and increased receipt

in 2012.77 The CBRR intervention was available in all 12 study

of Supplemental Nutrition Assistance Program benefits.75

sites and was provided by 27 CBRR programs across the sites.

This chapter begins with a brief description of the assistance

Nearly all the CBRR providers in the Family Options Study

offered to the families assigned to the CBRR intervention. It

were community-based nonprofit organizations. The only

then reviews the extent to which families in both the CBRR and

exceptions were in Louisville, Kentucky, and Phoenix, Arizona,

UC groups used temporary subsidies and other housing and

where city government agencies provided CBRR programs.

services programs available to them in the study sites during

CBRR programs were funded by the rapid re-housing compo-

the course of the followup period. Then, the core sections of

nent of the Homelessness Prevention and Rapid Re-Housing

75 Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015) also presented results showing that assignment to the CBRR group led to more rapid departures from emergency shelter than assignment to the UC group by about 2 weeks, but not more rapid departures than for families assigned to the SUB, group or to the PBTH group. Revised analysis using updated Program Usage Data on length of emergency shelter stays, however, has resulted in changes to the findings about length of initial shelter stay. For the full study sample (not limited to 20-month survey respondents), families assigned to the CBRR group left shelter, on average, 1 week faster than families assigned to the UC group. This difference is not statistically significant. The results of the revised analysis are shown in Appendix I.

76 Chapter 7 of Gubits et al., 2015 provides additional details about the CBRR intervention. The adjective “community-based” in the name of the intervention is intended to describe the usual providers of the assistance. The study does not distinguish different types of rapid re-housing programs and so the terms community-based rapid re-housing assistance and rapid re-housing assistance are used interchangeably in the text.

77 http://usich.gov/population/families/core-components-rrh/.

3-Year Impacts of Housing and Services Interventions for Homeless Families

47

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Program (HPRP) in all the sites except one.78, 79 The HPRP rapid

used rapid re-housing and other types of homeless and housing

re-housing funding could be used to provide rental assistance

assistance during the followup period. Parallel information is

(up to 18 months), security deposits, utility deposits and pay-

also presented for the 434 included UC families.

ments, help with moving costs, and hotel and motel vouchers.

Exhibit 4-1 shows the use of eight types of homeless and hous-

HPRP also could fund case management for participating

ing assistance programs. The first column shows the percentage

families. As defined by the study, families are considered to

of families assigned to the CBRR group who ever used each

have used rapid re-housing assistance only when they have

program type during the followup period. The second row

received rental assistance. (This rental assistance was usually

(shaded in the exhibit) shows the takeup of rapid re-housing

accompanied by other HPRP-funded assistance, including some

by the families assigned to that intervention; 58.5 percent of

case management.) Any rental assistance paid for with HPRP

families referred to a CBRR program received rapid re-housing

funds had to meet rent reasonableness standards, and units had

assistance at some point during the followup period—usually

to pass a habitability inspection. The inspection requirements

meaning they followed up on the referral, were deemed eligible

were slightly less stringent than the Housing Quality Standards

by the program, found a housing unit, and received one of the

required for the Housing Choice Voucher program form of

types of temporary rental assistance offered to families assigned

permanent housing subsidy (SUB).

to the CBRR intervention.81, 82

4.2. Program Use by Families

The second column shows the percentage of families assigned

in the Community-Based Rapid

to the UC group who ever used each program type during

the followup period.83 The shaded row of the second column

Re-Housing (CBRR) Versus Usual

shows that 22.5 percent of the UC families received rapid

Care (UC) Comparison

re-housing assistance during the followup period, despite not

being given priority access to rapid re-housing. These families

Each impact comparison in the study may be thought of as a

may have sought rapid re-housing rental assistance after learn-

distinct experiment or test. This chapter addresses only the

ing of its availability (perhaps from friends or family members)

comparison between the CBRR intervention and usual care,

or they may have already been clients of the community-based

without reference to the families who were randomized to the

nonprofit organizations that administered the local CBRR

SUB or PBTH interventions. In total, 1,144 families took part

programs.84

in the test of the CBRR intervention versus usual care. These

families all had both the CBRR intervention and usual care

The first row of the exhibit and rows three through six show

available to them at the time of random assignment and were

participation in other nonshelter types of homeless and housing

assigned to one of these two interventions; 569 families were

assistance programs. The seventh row shows participation

assigned to the CBRR group, and 575 families were assigned

in any form of permanent subsidy. Any form of permanent

to the UC group.80 Of these 1,144 families, 868 (434 CBRR

housing subsidy includes the permanent subsidy programs of-

families and 434 UC families), or 76 percent, responded to

fered to the SUB group, permanent supportive housing, public

the 37-month followup survey and are included in the CBRR-

housing, and project-based vouchers/Section 8 projects. Of the

versus-UC comparison reported in this chapter. The current

UC families in the CBRR-versus-UC comparison, 38 percent

section describes the extent to which these 434 CBRR families

found their way to permanent housing subsidy programs and

78 HPRP was authorized through the American Recovery and Reinvestment Act of 2009. Across the nation, communities received $1.5 billion in HPRP funding, a one-time funding stream available for 3 years from program inception, to provide homelessness prevention and rapid re-housing assistance to individuals and families facing homelessness.

79 In Boston, the CBRR intervention was funded by the State of Massachusetts. The Boston programs offered assistance very similar to HPRP, although rental assistance could be provided for longer periods. The Minneapolis and Salt Lake City CBRR programs supplemented HPRP funds with state funds and other ARRA funds, respectively.

80 In the entire study, 746 families were randomly assigned to the UC group. Only 575 of these families had the CBRR intervention available to them when they were randomized, however. Therefore, only those 575 UC families are part of the CBRR-versus-UC comparison sample. All 569 families randomly assigned to the CBRR intervention during the course of the study had usual care available to them, so all are part of the CBRR-versus-UC comparison sample.

81 All percentages, means, and medians in the exhibit are weighted to adjust for survey nonresponse and, hence, as best as possible, represent the full sample of 1,144 families assigned to this experimental contrast. The findings on program use are thus in line with similarly weighted impact estimates provided subsequently in the chapter.

82 The rapid re-housing use in the table for the CBRR group also includes a small number of CBRR families who did not use the rapid re-housing assistance when initially offered, but who did use rapid re-housing rental assistance at a later time during the followup period.

83 The percentages in the first seven rows of these columns are not mutually exclusive because some families used more than one program type during the followup period.

84 Emergency shelter staff committed to not referring UC families to active interventions to which they did not have priority access. This commitment may not have been upheld in all cases.

3-Year Impacts of Housing and Services Interventions for Homeless Families

48

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Exhibit 4-1. CBRR Versus UC: Program Use Since RA

Percent Ever Used

Number of Months Used From RA to

Percent Used in

From RA to 37-Month

37-Month Followup Survey,

Month of Followup

Type of Housing Assistance

Followup Surveya

if Ever Used Type of Assistance

Survey Response

CBRR

UC

CBRR

UC

CBRR

UC

Mean

Median

Mean

Median

Permanent housing subsidies offered to the SUB groupb

9.8

12.2

20.0

21.5

19.2

19.5

8.5

10.2

Rapid re-housingc

58.5

22.5

8.0

7.5

7.8

6.5

2.3

1.1

Transitional housingd

23.2

27.5

9.9

7.5

11.6

9.0

4.2

3.3

Permanent supportive housing

9.8

11.7

15.9

12.0

17.3

17.0

6.7

8.0

Public housing

10.7

9.8

18.1

18.5

18.7

16.5

9.4

8.2

Project-based vouchers/Section 8 projects

5.6

6.3

18.4

17.5

17.3

15.5

5.3

4.8

Any permanent housing subsidye

35.4

37.9

18.3

18.5

19.2

19.5

29.7

31.1

Emergency shelterf

90.7

90.0

4.3

2.5

4.6

2.8

2.6

5.2

No use of homeless or housing programsg

9.2

24.4

—

—

—

—

61.3

59.9

N

434

434

—

—

—

—

434

434

CBRR = priority access to community-based rapid re-housing. UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/ Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

27 percent found their way to transitional housing—despite the

columns for CBRR families, fifth and sixth columns for UC

lack of preferential access to those programs through the study.

families) for only those families who ever used a given program

The proportions of the CBRR group that used programs other

type.86 The number of months of rapid re-housing assistance

than rapid re-housing programs are roughly similar to those of

use (median of 8 months) for the families assigned to the CBRR

the UC group; thus, assignment to the CBRR group did not act

group is similar to that of the 23 percent of UC families who

as a conduit to permanent subsidies to a greater extent than did

received rapid re-housing assistance (median of 7 months).87

being in the UC group.85 The ninth row shows the percentages

Additional detail about the use of the rapid re-housing by

of families in the CBRR and UC groups who used none of the

CBRR families is shown in Exhibit 4-2. This exhibit shows that

six types of programs during the 37-month followup period

nearly one-half (49 percent) of CBRR families who used rapid

and did not use emergency shelter from the 7th month after

re-housing did so for less than 7 months, and 86 percent did

random assignment onward. About 9 percent of CBRR families

so for less than 12 months.88 These relatively short periods of

and 24 percent of UC families fall into this group.

use may be surprising, given that the program rules permit use

of rapid re-housing for up to 18 months. Instead, these short

The mean and median number of months of use for each

periods of use reflect the reality of how the program was being

program type are also shown in the exhibit (third and fourth

administered in the study sites and how families were using it.

85 Although proportions that used any permanent housing subsidy are similar in the CBRR and UC groups, Exhibit 4-5 shows that assignment to the CBRR group caused some delay in the use of permanent housing subsidies.

86 Hence, 0 values are not factored into the means, nor do they pull downward the medians of the various distributions.

87 Because the distribution of receipt durations is nonnormal (shown in Exhibit 4-2), the study team used the median rather than the mean to describe the typical length of rapid re-housing rental assistance receipt.

88 By policy, HPRP-funded rapid re-housing assistance was limited to 18 months. Exhibit 4-2 shows that very few families received community-based rapid re-housing for longer periods.

3-Year Impacts of Housing and Services Interventions for Homeless Families

49

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Exhibit 4-2. Number of Months of Rapid Re-Housing

random assignment.89, 90 It is not surprising that, in the first

Receipt During Followup Period by CBRR Families Who

10 months after random assignment, program use for CBRR

Ever Used Rapid Re-housing

families is dominated by rapid re-housing. After this period,

however, use of rapid re-housing declines steadily. After month

14

16, all program use levels off, with only about 40 percent of

families using any type of program in any month. The use of

a

12

permanent housing subsidies steadily increases throughout the

10

followup period and accounts for more than one-half of the

8

remaining program use after month 18.

The bottom panel shows the proportions of program use over

6

time for the UC families. Compared with the CBRR group,

4

the UC group used rapid re-housing much less (as engineered

Percent of CBRR families

by the study) and used transitional housing and permanent

who ever used rapid re-housing 2

housing subsidies somewhat more extensively following their

0

initial stay in emergency shelter. After about month 18, the

total and by-type program use of the UC group is similar to

2 to 2.75

3 to 3.75

4 to 4.75

5 to 5.75

6 to 6.75

7 to 7.75

8 to 8.75

9 to 9.75

0.25 to 1.75

10 to 10.75

11 to 11.75

12 to 12.75

13 to 13.75

14 to 14.75

15 to 15.75

16 to 16.75

17 to 17.75

18 to 18.75

19 or more

that of the CBRR group. Appendix Exhibit H-1 shows addi-

Number of months of rapid re-housing receipt

tional information about program use in the second half of the

CBRR = priority access to community-based rapid re-housing.

followup period, from month 19 after random assignment until

a Percentages are weighted for survey nonresponse to represent all families in compar-

the month of the 37-month survey response.

ison sample.

Note: N = 252.

As Exhibits 4-1 and 4-3 make clear, the CBRR families used a

Source: Family Options Study Program Usage Data

range of programs in addition to the program to which they

were referred by the study, a pattern that is consistent with

Whereas the previous columns consider all experience between

the design of the study. Families were not required to use the

randomization and the survey, the last two columns consider

program to which they were given priority access and were also

program use as of the month of the 37-month survey. Although

not forbidden from using other programs that were available

most outcomes in the report are expected to be influenced by

to them in their community. The intent of the study was to

assistance received during the entire followup period, some

maximize use of the programs to which families assigned to the

outcomes will be particularly strongly influenced by assistance

intervention were given priority access (in this case, maximize

received at the time of the followup survey response. The

use of rapid re-housing by the CBRR families) and to create

second row of the seventh and eighth columns shows that, in

the largest possible contrast between the program mixes of

both the CBRR and UC groups, rapid re-housing assistance had

different assignment groups (in this case, CBRR versus UC). As

ended by the followup survey for most of the families who ever

shown in the exhibit, the use of rapid re-housing was different

received it. Most CBRR families (61 percent) and UC families

for the CBRR and UC groups. The contrast in usage of rapid

(60 percent) were not participating in a homeless or housing

re-housing—58.5 percent for CBRR families and 22.5 percent

program at the time they responded to the followup survey.

for UC families—is sizable, although smaller than the analo-

Thus, differences are expected in the outcomes of CBRR and

gous contrast between the SUB and UC groups (where use of

UC families only to the extent that these outcomes reflect a

some form of permanent subsidy was higher for both groups).

lasting influence of families having been offered temporary

rental assistance to help them leave homelessness.

As is conventional in random assignment analyses, our goal

is to estimate the intention-to-treat (ITT) impact—that is, the

Exhibit 4-3 provides a more detailed picture of the timing of

impact of offering a program to families, regardless of whether

program use by the families in the CBRR-versus-UC compar-

they actually used that program (or some other program). This

ison. The top panel shows the proportions of families within

goal is consistent with the policy option of making a treatment

the CBRR group who are receiving different types of assistance

available to a family but not requiring the family to use that

during each calendar month for the first 32 months after

treatment.

89 Month 32 is the latest month for which the study team has data for all the families who responded to the 37-month survey.

90 Exhibit 3-4 is closely related to the analysis of the costs of this pairwise comparison presented in Chapter 9. The reader should note that the exhibit does not indicate the two-way flows of families moving into and out of these program types from month to month. Instead, it reflects only the overall usage level in a given month.

3-Year Impacts of Housing and Services Interventions for Homeless Families

50

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Exhibit 4-3. CBRR Versus UC: Program Use in Each Month of Followup Period

Panel A: Program Use of CBRR Families for 32 Months After RA

100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

20

Rapid re-housing

Percent of families using program type in month

Transitional housing

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

Panel B: Program Use of UC Families for 32 Months After RA

100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

Rapid re-housing

20

Percent of families using program type in month

Transitional housing

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

CBRR = priority access to community-based rapid re-housing. UC = usual care.

RA = random assignment.

Notes: Families who have more than one type of program use in a calendar month are counted fractionally in each type.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

51

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Because not all families randomly assigned to the CBRR group

4.3. Impacts on Housing Stability

used rapid re-housing and because some families assigned to

in the Community-Based Rapid

the UC group did use rapid re-housing, the true ITT impact

Re-Housing (CBRR) Versus Usual

is likely smaller than it would have been had the gap in rapid

re-housing usage been wider (assuming that rapid re-housing

Care (UC) Comparison

truly has a nonzero impact on families who use it). In particu-

Proponents of community-based rapid re-housing share a

lar, the difference in the use of rapid re-housing by the CBRR

view with the proponents of permanent housing subsidies: the

and UC groups is narrow enough, given the relatively small

lack of housing affordability is the root cause of homelessness

sample size available for analysis, that the study may have failed

among families. Because permanent housing subsidies are

to detect as statistically significant one or more ITT impacts

constrained by limited appropriations, proponents of rapid

large enough to be of policy importance.

re-housing argue that limited resources dedicated to home-

The remainder of this chapter reports estimated impacts in the

lessness could be stretched to create the best outcomes for the

various outcome domains that—if statistically significant—can

most people by making subsidies temporary. The CBRR-versus-

be causally attributed to the offer of a temporary housing

UC comparison offers evidence on whether priority access

subsidy with light case management to the families randomly

to the temporary subsidies is an effective tool for improving

assigned to the CBRR group at the start of the followup period

housing stability relative to usual care.

in contrast with no such directed referral or privileged access

Exhibit 4-4 shows the impacts of assignment to the CBRR

being available to UC families.

group on homelessness, housing independence, residential

moves, and housing quality. Relative to assignment to the UC

Exhibit 4-4. CBRR Versus UC: Impacts on Housing Stability at 37 Months

CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the followup period

At least 1 night homelessb or doubled up in past 6 months or in shelter in

434

39.7

(48.8) 434

37.8

(48.5)

1.9

(3.6)

0.03

past 12 months (%) [confirmatory]c

At least 1 night homelessb or doubled up in past 6 months (%)

434

35.5

(47.7) 434

33.7

(47.2)

1.8

(3.5)

0.03

At least 1 night homelessb in past 6 months (%)

434

16.8

(36.6) 434

17.0

(37.4)

– 0.1

(2.8)

0.00

At least 1 night doubled up in past 6 months (%)

434

30.5

(45.8) 434

27.6

(44.8)

2.8

(3.3)

0.05

Any stay in emergency shelter in past 6 months (%) [Program Usage Data]

434

6.8

(24.2) 434

8.8

(28.0)

– 2.0

(2.2)

– 0.06

Any stay in emergency shelter in months 21 to 32 after RA (%) [Program

434

16.3

(36.2) 434

18.8

(38.8)

– 2.5

(2.9)

– 0.06

Usage Data]

Number of days homelessb or doubled up in past 6 months

433

50.2

(74.1) 431

46.2

(73.6)

4.0

(5.6)

0.05

Number of days homelessb in past 6 months

434

15.7

(41.6) 434

18.5

(48.4)

– 2.8

(3.7)

– 0.05

Number of days doubled up in past 6 months

433

38.4

(66.7) 431

31.7

(61.4)

6.7

(4.7)

0.09

Housing independence

Living in own house or apartment at followup (%)

434

68.8

(46.7) 434

69.3

(46.2)

– 0.6

(3.4)

– 0.01

Living in own house or apartment with no housing assistance (%)

432

39.4

(48.6) 431

41.3

(49.1)

– 1.9

(3.5)

– 0.03

Living in own house or apartment with housing assistance (%)

432

29.9

(45.9) 431

27.8

(45.4)

2.0

(3.3)

0.04

Number of places lived

Number of places lived in past 6 monthsd

434

1.6

(1.0) 434

1.6

(1.1)

0.0

(0.1)

0.02

Housing quality

Persons per room

412

1.6

(1.3) 411

1.6

(1.2)

0.0

(0.1)

0.03

Housing quality is poor or fair (%)

415

30.4

(45.9) 410

34.5

(47.2)

– 4.1

(3.6)

– 0.08

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is not statistically significant at the .10 level for the CBRR-versus-UC comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

52

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) group, at the 3-year followup survey, assignment to the CBRR

survey. The last six rows of Exhibit 4-4 show that assignment

group does not appear to reduce homelessness in this sample

to the CBRR group has no effect on three measures: housing

of families.

independence, the number of places families lived, and housing

quality relative to assignment to the UC group.

None of the eight impact estimates for homelessness and

doubled-up outcomes is statistically different from 0. Assign-

One main goal of the CBRR intervention is to reduce the use

ment to the CBRR group has no effect on the proportion of

of emergency shelter. Exhibit 4-5 shows the month-by-month

families experiencing homelessness 3 years after random

impacts of random assignment to the CBRR group versus the

assign ment in measures based on the survey, based on Program

UC group on the proportions of families who had at least 1 night

Usage Data, or based on a combination of the two. The estimates

in emergency shelter and with use of any permanent housing

also provide no evidence of effects on the number of days spent

subsidy. Compared with proportions of the UC group, similar

homeless or doubled up in the 6 months before the 37-month

proportions of the CBRR group used emergency shelter in most

followup survey. Nearly two-fifths of both the CBRR group

months. Only during some months within the 1st year after

(39.7 percent) and the UC group (37.8 percent) reported being

random assignment does it appear that assignment to the CBRR

homeless or doubled up in the past 6 months or appeared in

group diverts families from using emergency shelter relative to

shelter records for the past year at the time of the followup

assignment to the UC group.91

Exhibit 4-5. CBRR Versus UC: Percent of Families With Any Stay in Emergency Shelter and Any Permanent Housing Subsidy During Month, by Number of Months After RA

100

90

80

70

60

50

UC: Used emergency shelter

Percent of families

40

30

UC: Used any permanent housing subsidy

20

CBRR: Used

CBRR: Used any permanent housing subsidy

emergency shelter

10

0 0

5

10

15

20

25

30

35

Month after RA

CBRR = priority access to community-based rapid re-housing. UC = usual care.

RA = random assignment.

Notes: Percentages are weighted for survey nonresponse to represent all families in the SUB-versus-UC comparison sample. Missing data on emergency shelter stays bias the percentages somewhat downward. The baseline stay in emergency shelter does not appear in the data for 16.7 percent of UC respondent families. The missing data rate for subsequent stays in emergency shelter is unknown. Any permanent housing subsidy includes the permanent subsidy programs offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

Source: Family Options Study Program Usage Data

91 CBRR proportions in emergency shelter are lower than those of the UC group by statistically significant amounts (each at the .10 level) in months 3, 5, and 11.

3-Year Impacts of Housing and Services Interventions for Homeless Families

53

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Although having priority access to rapid re-housing appears to

4.4. Impacts on Family Preservation

have little effect on the use of emergency shelter, it does appear

in the Community-Based Rapid

to reduce the use of permanent housing subsidies (including

Re-Housing (CBRR) Versus Usual

permanent supportive housing programs and mainstream

housing programs) during the first half of the followup period.

Care (UC) Comparison

The proportions of the CBRR families using any permanent

Any effects of assignment to the CBRR group on family preserva-

housing subsidy are about 6 percentage points lower than the

tion would be expected to be indirect, via increases in housing

corresponding proportions for the UC families in months 6 to 14

stability, but no such effects were detected. Exhibit 4-6 shows

(and are statistically significantly different in months 2 to 14).

that no evidence from survey data is found of longer-term

These differences disappear in the second half of the followup

effects on family separations from children or from spouses or

period, when use of permanent housing subsidies by the CBRR

partners who were with the family at baseline. Nor is evidence

group catches up to that of the UC group.

found of effects on reunifications of the much smaller number

Overall, it appears that the offer of priority access to rapid

of family members who were separated from the family at

re-housing for the CBRR families has no effect on housing stability

baseline. No effect was detected on foster care placements.

during the 37-month followup period relative to usual care.

Exhibit 4-7 shows impacts on outcomes from the child welfare

agency administrative data collected in 5 of the 12 study sites.

No effect is detected on the proportion of family heads with a

formal child separation that began after random assignment.

Exhibit 4-6. CBRR Versus UC: Impacts on Family Preservation at 37 Months

CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

430

15.3

(35.9) 424

15.8

(35.6)

– 0.5

(2.6)

– 0.01

Family has at least one foster care placement in past 6 monthsb (%)

430

3.8

(18.4) 428

3.2

(17.2)

0.6

(1.6)

0.03

Spouse/partner separated in past 6 months, of those with spouse/partner

110

44.8

(49.0) 112

36.2

(48.9)

8.6

(7.2)

0.15

present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at least one

80

35.0

(49.3)

82

31.6

(45.8)

3.4

(7.4)

0.06

child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent at RA (%)

41

31.3

(44.9)

46

21.1

(40.1)

10.2

(10.8)

0.22

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey.

Exhibit 4-7. CBRR Versus UC: Impacts on Child Welfare Outcomes

CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Had a formal child separation that began after RA (%)

220

10.5

(30.7) 194

11.3

(31.8)

– 0.7

(2.9)

– 2.24

Total days during followup separated from at least one childb

220

47.4 (176.3) 194

76.7 (247.9) – 29.3*

(15.5)

– 0.12

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days. Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions. Sample limited to five sites where child welfare records were collected (Alameda County, Baltimore, Kansas City, Minneapolis, and Phoenix).

Source: State child welfare agency records

3-Year Impacts of Housing and Services Interventions for Homeless Families

54

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) The study team did find an effect on total time with any separa-4.5. Impacts on Adult Well-Being

tion. Assignment to the CBRR group reduced the total number

in the Community-Based Rapid

of days separated from at least one child during the 3-year

Re-Housing (CBRR) Versus Usual

followup period by 29 days. This difference represents a reduc-

tion of about 5 months for those family heads who experienced

Care (UC) Comparison

a separation.92 The study team investigated whether this effect

The theory and goals of the CBRR intervention compared with

was associated with the particular 5 sites where administrative

those of usual care do not hypothesize important effects on

data were collected. In the survey data, the team finds a reduction

adult well-being. Consistent with this expectation, 20 months

of 10 percentage points for CBRR families versus UC families

after random assignment, no differences were detected between

for the subsample of these 5 sites on the proportion of family

the CBRR and UC groups on any measure of adult well-being.

heads with any foster care or adoption. The magnitude of this

Because having priority access to the temporary subsidies of the

effect contrasts with the estimate for the full sample of less than

CBRR intervention did not lead to improvements in well-being

1 percentage point (shown in the first row of Exhibit 4-6). This

in the short term, one would not expect longer-term benefits

evidence shows that the effect detected in the administrative

to emerge well after the subsidies ended, and, indeed, they

data may arise because the analysis sample is limited to these

did not. As shown in Exhibit 4-8, no evidence indicates 3-year

particular 5 sites. The study team therefore suspects that the

impacts on any measure of adult physical or mental health,

effect in 5-site administrative data does not generalize to all

trauma or intimate partner violence, or substance dependence

12 sites.

or abuse.

Exhibit 4-8. CBRR Versus UC: Impacts on Adult Well-Being at 37 Months

CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

433

31.5

(46.2) 434

30.2

(45.8)

1.3

(3.1)

0.02

Adult mental health

Goal-oriented thinkingb

429

4.46

(0.98) 431

4.57

(0.98) – 0.10

(0.07)

– 0.09

Psychological distressc

432

6.74

(5.63) 433

6.90

(5.72) – 0.16

(0.39)

– 0.02

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

430

20.8

(39.7) 433

21.3

(41.7)

– 0.5

(3.0)

– 0.01

Adult substance use

Alcohol dependence or drug abused (%)

433

9.3

(29.3) 433

11.1

(31.7)

– 1.7

(2.2)

– 0.05

Alcohol dependenced (%)

433

7.5

(26.9) 434

7.9

(26.9)

– 0.4

(2.0)

– 0.01

Drug abused (%)

433

3.0

(15.8) 433

4.7

(22.0)

– 1.8

(1.4)

– 0.07

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

434

7.6

(26.5) 433

9.2

(29.9)

– 1.6

(2.0)

– 0.05

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey.

92 Dividing mean number of days separated by the proportion of family heads with at least one separation (not shown) results in an average of 443 days for CBRR family heads with at least one separation compared with an average of 585 days for UC family heads.

3-Year Impacts of Housing and Services Interventions for Homeless Families

55

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) 4.6. Impacts on Child Well-Being

expected to be indirect. In particular, spending 4 to 6 months

in the Community-Based Rapid

in stable housing within families’ own communities with a

Re-Housing (CBRR) Versus Usual

sharply lower burden of housing costs could enable adult fam-

ily members to concentrate more on employment and earnings

Care (UC) Comparison

and even enhance their skills through education and training

participation. In some CBRR programs, case management

Any effects of assignment to the CBRR group on child well-

guidance and referrals may further enhance efforts at work and

being would be expected to be indirect, via increases in housing

access to resources that make families more self-sufficient.

stability. Given the lack of effect of assignment to the CBRR

group on housing stability compared with the UC group at

For the 37-month period following families’ assignment to the

37 months, the study team would not expect strong effects on

CBRR intervention, the analysis shows significant effects of as-

child well-being, and, indeed the team finds none. The team

signment to the CBRR intervention on 2 of 20 outcomes examined,

finds evidence of one effect on outcomes measured across age

or about the number that would be expected by chance alone

groups—children in the CBRR group have lower behavior

(see Exhibit 4-11). Assignment to the CBRR intervention led

problems with an effect size of 0.12 standard deviations

to increased participation in Temporary Assistance for Needy

(Exhibit 4-9). The study team finds no evidence of age-specific

Families (TANF) in the month before the 37-month survey (28

effects (Exhibit 4-10). With 1 significant effect out of 29—fewer

percent of families assigned to the CBRR group reported receiv-

than would be expected by chance—the team concludes that

ing TANF compared with 21 percent of families assigned to the

assignment to the CBRR group had little effect on children’s

UC group). Assignment to the CBRR intervention also reduced

well-being relative to assignment to the UC group.

family heads’ participation in 2 or more weeks of vocational

education in the 3-year period since random assignment com-

4.7. Impacts on Self-Sufficiency in

pared with usual care. In other self-sufficiency areas, most notably

the Community-Based Rapid Re-

in the subdomains of employment, income, and food security,

the study team finds no evidence of impacts of assignment

Housing (CBRR) Versus Usual Care

to the CBRR group relative to assignment to the UC group.

(UC) Comparison

Exhibit 4-12 shows impact estimates from administrative data

Any effects of assignment to the CBRR group on family self-

on outcomes for quarters 11 to 14 after the quarter of random

sufficiency relative to assignment to the UC group would be

assignment. The analysis of administrative data finds no effect

Exhibit 4-9. CBRR Versus UC: Impacts on Child Well-Being Across Age Groups at 37 Months CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

514

2.1

(1.0) 521

2.1

(0.9)

0.0

(0.1)

0.01

Grade completion (not held back) (%)

427

91.52 (28.51) 428

92.48 (26.33) – 0.96

(2.01)

– 0.03

School gradesc

399

3.1

(0.9) 387

3.1

(0.9)

0.0

(0.1)

0.00

Child physical health

Poor or fair health in past 30 days (%)

597

5.5

(22.5) 581

6.1

(24.4)

– 0.6

(1.6)

– 0.02

Well-child checkup in past year (%)

597

90.8

(28.9) 580

90.0

(28.6)

0.8

(2.3)

0.02

Child has regular source of health care (%)

598

93.7

(23.8) 580

92.6

(25.7)

1.0

(1.9)

0.03

Sleep problemsd

597

2.08

(1.03) 583

2.19

(1.10) – 0.12

(0.08)

– 0.08

Child behavioral strengths and challenges

Behavior problemse

576

0.38

(1.22) 560

0.58

(1.28) – 0.20** (0.10)

– 0.12

Prosocial behaviorf

577

– 0.18

(1.13) 561

– 0.23

(1.10)

0.06

(0.09)

0.04

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

3-Year Impacts of Housing and Services Interventions for Homeless Families

56

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Exhibit 4-10. CBRR Versus UC: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

187

36.8

(49.0) 169

38.0

(49.3)

– 1.2

(6.0)

– 0.02

Child care or preschool absences in past monthd

63

0.55

(0.89)

73

0.90

(0.93) – 0.34

(0.22)

– 0.31

Positive child care or preschool experiencese

65

0.86

(0.38)

74

0.80

(0.37)

0.06

(0.09)

0.13

Positive child care or preschool attitudesf

65

4.42

(0.92)

74

4.62

(0.66) – 0.19

(0.17)

– 0.23

Child care or preschool conduct problemsg (%)

69

5.9

(28.4)

75

5.6

(22.6)

0.3

(5.3)

0.01

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

169

73.3

(44.6) 155

67.6

(46.9)

5.7

(6.6)

0.10

Ages 3 years, 6 months to 7 years

Verbal abilityi

195

– 0.20

(1.01) 191

– 0.33

(1.02)

0.14

(0.12)

0.11

Math abilityj

194

– 0.27

(0.81) 193

– 0.31

(0.90)

0.04

(0.10)

0.03

Executive functioningk (self-regulation)

185

16.05 (16.35) 189

17.40 (16.35) – 1.35

(1.48)

– 0.07

Ages 5 to 17 yearsl

School enrollmentc (%)

413

97.3

(15.4) 417

97.4

(14.5)

– 0.2

(1.5)

– 0.01

School absences in past monthd,m

141

0.82

(0.95) 143

1.01

(0.93) – 0.19

(0.14)

– 0.15

Positive school experiencese,m

140

0.52

(0.58) 145

0.45

(0.60)

0.06

(0.08)

0.07

Positive school attitudesf,m

141

4.16

(1.13) 145

4.16

(1.07)

0.00

(0.12)

0.00

School conduct problemsg,m (%)

140

24.7

(44.6) 147

32.9

(46.8)

– 8.2

(5.7)

– 0.13

Ages 8 to 17 years

Anxietyn

263

35.61

(7.21) 260

35.27

(7.48)

0.34

(0.69)

0.03

Fearso

266

64.27 (14.39) 260

63.47 (14.72)

0.80

(1.31)

0.04

Substance usep (%)

260

5.97 (23.36) 253

8.63 (29.36) – 2.67

(2.72)

– 0.07

Goal-oriented thinkingq

255

22.53

(4.68) 251

22.52

(4.27)

0.01

(0.47)

0.00

School effort in past monthr

262

2.70

(0.82) 255

2.81

(0.78) – 0.10

(0.08)

– 0.10

Arrests or police involvement in past 6 monthss (%)

151

9.31 (28.14) 148

8.27 (26.32)

1.04

(3.41)

0.03

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Notes: Impact estimates and outcome means are regression-adjusted for baseline characteristics and are weighted to adjust for survey non-response. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS.

3-Year Impacts of Housing and Services Interventions for Homeless Families

57

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) Exhibit 4-11. CBRR Versus UC: Impacts on Self-Sufficiency at 37 Months

CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

434

39.6

(49.2) 434

39.1

(48.8)

0.5

(3.4)

0.01

Any work for pay since 20-month surveyb (%)

406

62.2

(48.0) 387

66.5

(47.9)

– 4.2

(3.4)

– 0.09

Months worked for pay since 20-month surveyb,c

406

7.3

(7.8) 383

7.6

(7.9)

– 0.3

(0.5)

– 0.04

Any work for pay since RA (%)

434

75.1

(41.6) 433

75.2

(44.0)

– 0.1

(3.0)

0.00

Months worked for pay since RAc

430

14.4

(13.4) 420

13.9

(13.6)

0.5

(0.8)

0.03

Hours of work per week at current main jobd

433

12.8

(17.2) 433

12.7

(16.9)

0.1

(1.2)

0.01

Income sources and amounts

Annualized current earnings ($)

425

7,127 (11,047) 422

7,154 (11,183)

– 27

(748)

0.00

Total family income ($)

415 11,837 (9,299) 423 12,343 (11,782) – 505

(725)

– 0.04

Anyone in family had earnings in past month (%)

434

50.4

(50.0) 434

51.4

(50.1)

– 1.1

(3.5)

– 0.02

Anyone in family received TANF in past month (%)

434

28.0

(44.4) 434

21.4

(41.6)

6.6**

(2.9)

0.13

Anyone in family received SSDI in past month (%)

434

10.7

(29.0) 434

9.1

(31.1)

1.6

(2.0)

0.05

Anyone in family received SSI in past month (%)

434

13.8

(34.6) 434

14.1

(36.6)

– 0.3

(2.2)

– 0.01

Anyone in family received SNAP/Food Stamps in past month (%)

434

82.9

(36.6) 434

82.3

(38.8)

0.6

(2.7)

0.01

Anyone in family received WIC in past month (%)

434

25.9

(44.7) 434

23.9

(42.5)

2.0

(3.0)

0.04

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

434

39.5

(49.3) 433

39.7

(48.7)

– 0.2

(3.4)

0.00

Number of weeks in school/training programs since RA

425

7.4

(16.5) 425

8.9

(16.8)

– 1.5

(1.2)

– 0.07

Participated in 2 weeks or more of school since RA (%)

433

7.9

(27.6) 433

10.9

(30.6)

– 2.9

(2.0)

– 0.08

Participated in 2 weeks or more of basic education since RA (%)

433

3.7

(19.4) 433

2.4

(15.8)

1.3

(1.2)

0.07

Participated in 2 weeks or more of vocational education since RA (%)

433

10.5

(32.0) 433

14.6

(35.1)

– 4.1*

(2.4)

– 0.10

Food security and

Household is food insecure (%)

434

40.2

(48.8) 434

44.4

(49.8)

– 4.2

(3.6)

– 0.07

Food insecurity scalee

434

1.70

(2.05) 432

1.87

(2.07) – 0.18

(0.15)

– 0.07

Economic stressors

Economic stress scalef

431

– 0.15

(0.50) 433

– 0.13

(0.49) – 0.02

(0.03)

– 0.03

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 4-12. CBRR Versus UC: Earnings and Employment

CBRR

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Earnings in quarters 11 to 14 after RA (2015Q3$)

557

6,591 (10,587) 553

6,167 (9,966)

424

(629)

0.04

Any employment in quarters 11 to 14 after RA (%)

557

57.7

(49.5) 553

58.7

(49.2)

– 0.9

(3.0)

– 0.02

Number of quarters employed in quarters 11 to 14 after RA

557

1.7

(1.7) 553

1.7

(1.7)

0.0

(0.1)

0.01

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Note: See Chapter 2 and Appendix B for outcome definitions.

Source: Quarterly wage records from the National Directory of New Hires

3-Year Impacts of Housing and Services Interventions for Homeless Families

58

FAMILY OPTIONS STUDY Chapter 4. Impacts of Community-Based Rapid Re-Housing (CBRR) Compared With Usual Care (UC) of assignment to the CBRR group relative to assignment to the

in experiences between the CBRR and UC families during the

UC group on total earnings, the proportion of family heads em-

longer term. The vast majority of the evidence—involving doz-

ployed, or the number of quarters employed during the year.

ens of outcomes in five domains—suggests equivalent results

for housing stability, family preservation, adult well-being,

4.8. Summary of the Community-

child well-being, and self-sufficiency, with or without privi-

leged access to rapid re-housing through the CBRR intervention

Based Rapid Re-Housing (CBRR)

after 7 days in shelter.

Versus Usual Care (UC) Comparison

Across Domains

It is most striking that, relative to usual care, the study team

did not find evidence that assignment to the CBRR intervention

In the CBRR-versus-UC comparison, 59 percent of families

affected housing stability at the 37-month followup point.

assigned to the CBRR group and 23 percent of families assigned

Chapter 6 reports about how the CBRR intervention compares

to the UC group received rapid re-housing rental assistance.

with the other two active interventions: SUB and PBTH.

This contrast in program use did not lead to notable differences

3-Year Impacts of Housing and Services Interventions for Homeless Families

59

FAMILY OPTIONS STUDY

CHAPTER 5.

IMPACTS OF PROJECT-BASED

TRANSITIONAL HOUSING (PBTH)

COMPARED WITH USUAL CARE (UC)

This chapter presents estimates of the impact of assign- 5.1. Project-Based Transitional ment to the PBTH intervention relative to usual care

Housing (PBTH) Intervention

in the study communities. The goal of the analyses

presented in this chapter is to determine the extent to which

The PBTH intervention provides families with priority access

being offered priority access to a unit in a PBTH program

to supervised programs in which they stay for a finite period of

increases families’ housing stability and improves other family

time (usually limited to no more than 24 months) and during

outcomes 3 years after receiving the offer.

which they are provided case management and a wide array

of services identified through an assessment of family needs.

The analysis reported in the Family Options Study: Short-Term

Services may be offered directly by case managers or other

Impacts of Housing and Services Interventions for Homeless Families

staff or via referral to other providers. The PBTH intervention

(hereafter, the Short-Term Impacts report; Gubits et al., 2015)

was offered to study families in all but one study site and was

found that, at 20 months after random assignment, assignment

provided by 46 PBTH programs across the sites.93

to the PBTH group reduced the proportion of families who

reported stays in shelters or places not meant for human

Transitional housing is intended to prepare families for perma-

habitation in the 6 months before the survey. Assignment to

nent housing by providing services that help overcome barriers

the PBTH group also reduced the proportion of families who

to housing stability and address other psychosocial needs

had a stay in emergency shelter during months 7 to 18 of the

that the family may have. For this study, the team selected

followup period. These findings may have reflected the fact that

transi tional housing programs that provide housing primarily

about 22 percent of families assigned to the PBTH group were

in “project-based” facilities or housing units. The study’s defi-

using transitional housing at the time of the 20-month survey.

nition excluded programs that allow for families to “transition

In the four other outcome domains, the vast majority of indica-

in place” in private-market apartments or single-family homes,

tors examined at that time revealed little difference between the

taking over responsibility for the housing unit’s lease toward or

families in the PBTH group and families in the UC group.

at the end of the program of transitional assistance. The study

excluded transition-in-place programs to generate a strong

This chapter begins with a brief description of the assistance

contrast between project-based transitional housing and com-

offered to the families assigned to the PBTH intervention. It

munity-based rapid re-housing (CBRR) programs, which also

then reviews the extent to which families in both the PBTH and

are time limited and use scattered-site housing units in which

UC groups used transitional housing and other housing and

the family can stay and pay the rent at the end of the CBRR

services programs available to them in the study sites during

program. The study’s PBTH intervention referred a few families

the course of the followup period. Then, the core sections of

to transitional housing programs with scattered-site locations,

the chapter present the 3-year effects of being assigned to the

but, at program completion (or when the family reached the time

PBTH group (compared with being assigned to the UC group)

limit), families were required to relocate to other housing units.

on outcomes within each of the study domains: housing sta-

bility, family preservation, adult well-being, child well-being,

The PBTH programs studied often received funding from fed-

and self-sufficiency. The final section summarizes the 3-year

eral Supportive Housing Program (SHP) grants, which results

impacts of the PBTH intervention relative to usual care.

in some consistency across PBTH programs. For instance, the

93 The PBTH intervention was not offered in Boston. Also, the PBTH intervention was very limited in Minneapolis, with only four families randomly assigned to the PBTH

group. Further details about the PBTH intervention are provided in Chapter 8 of the Short-Term Impacts report (Gubits et al., 2015).

3-Year Impacts of Housing and Services Interventions for Homeless Families

60

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) SHP grant limits transitional housing assistance to 24 months,

This section also presents parallel information for the 259

funds a broad range of supportive services, and sets parameters

included UC families. The data on program use do not distin-

for the way in which programs must calculate participant rent

guish between subtypes of transitional housing and include

contributions when they choose to require them; however, not

transition-in-place assistance, so this section uses “transitional

all the PBTH programs in the study receive funding from SHP

housing” rather than “ project-based transitional housing” to

grants. Most PBTH programs have a wide range of funding

describe the broader category of assistance.

sources, including private foundation grants and local fund-

Exhibit 5-1 shows the use of eight types of homeless and hous-

raising proceeds. Some programs are faith based, and many of

ing assistance programs. The first column shows the percentage

those programs are completely privately funded.

of families assigned to the PBTH group who ever used each

All PBTH programs in the study provide only temporary hous-

program type during the followup period. The third row (shad-

ing assistance. The study team allowed any time limit on tenure

ed in the exhibit) shows the use of some type of transitional

but specifically sought programs that offered at least 6 months

housing by the families assigned to the PBTH group; 53.2

of assistance. Nearly all programs provided a maximum of 24

percent of families in the PBTH group received transitional

months of assistance. Programs offering referrals to permanent

housing assistance at some point during the followup period—

housing assistance at the end of the transitional housing period

meaning they either followed up on the referral and moved

were included in the PBTH intervention for the study, but pro-

into the PBTH facility or entered another transitional housing

grams that guaranteed permanent assistance were not included.

program.95, 96

The second column shows the percentage of families assigned

5.2. Program Use by Families in the

to the UC group who ever used each program type during

Project-Based Transitional Housing

the followup period.97 The shaded row of the second column

(PBTH) Versus Usual Care (UC)

shows that 34.6 percent of the UC families received transitional

Comparison

housing assistance during the followup period, despite not

being given priority access to transitional housing programs.

Each impact comparison in the study may be thought of as a

The study team requested that emergency shelter staff not refer

distinct experiment or test, and this chapter addresses only

UC families to one of the active interventions. Nevertheless, as

the comparison between the PBTH group and the UC group,

shown in the exhibit, some families did learn about transitional

without reference to the families who were randomly assigned

housing programs in their communities, and these programs

to the SUB or CBRR intervention. In total, 707 families took

had program slots available at some point during the followup

part in the test of the PBTH intervention versus usual care. All

period.

these families had both project-based transitional housing and

Rows one, two, and four through six of the exhibit show

usual care available to them at the point of random assignment

participation in other nonshelter types of homeless and housing

and were assigned to one of these two interventions; 368

assistance programs. The seventh row shows participation

families were assigned to the PBTH group and 339 families

in any form of permanent subsidy. Any form of permanent

were assigned to the UC group.94 Of those 707 families, 552

subsidy includes the permanent subsidy programs offered to

(293 PBTH families and 259 UC families), or 78 percent,

the SUB group, permanent supportive housing, public housing,

responded to the 37-month followup survey and are included

and project-based vouchers/Section 8 projects. Of the UC

in the PBTH-versus-UC impact comparison reported in this

families in the PBTH-versus-UC comparison, 35 percent found

chapter. The current section describes the extent to which the

their way to permanent housing subsidy programs and 18

293 PBTH families used transitional housing and other types of

percent found their way to rapid re-housing, despite the lack

homeless and housing assistance during the followup period.

of preferential access to those programs through the study.

94 In the entire study, 746 families were randomly assigned to the UC group, but only 339 of those families had project-based transitional housing available to them when they were randomized. Therefore, only those 339 UC families are part of the PBTH-versus-UC comparison sample. All 368 families randomly assigned to the PBTH group during the course of the study had usual care available to them, so all are part of the PBTH-versus-UC comparison sample.

95 All percentages, means, and medians in the exhibit are weighted to adjust for survey nonresponse and, hence, as best as possible, represent the full experimental sample of 707 families. The findings here on program use are thus in line with similarly weighted impact estimates provided later in the chapter.

96 The unweighted number of 37-month respondent PBTH families who used transitional housing during the followup period is 160 families. Of those, 88 families were confirmed by enrollment verification to have used the program to which they were referred by the study. It is not known how many of the other 72 families received project-based transitional housing or some other form of transitional housing.

97 The percentages in the first six rows of these columns are not mutually exclusive because some families used more than one program type during the followup period.

3-Year Impacts of Housing and Services Interventions for Homeless Families

61

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-1. PBTH Versus UC: Program Use Since RA

Percent Ever Used

Number of Months Used From RA to

Percent Used in

From RA to 37-Month

37-Month Followup Survey,

Month of Followup

Type of Homeless or Housing Assistance

Followup Surveya

if Ever Used Type of Assistance

Survey Response

PBTH

UC

PBTH

UC

PBTH

UC

Mean

Median

Mean

Median

Permanent housing subsidies offered to the SUB groupb

9.6

10.6

14.0

10.5

17.1

17.5

7.4

7.9

Rapid re-housingc

13.7

17.8

7.4

6.5

6.9

6.0

0.7

1.8

Transitional housingd

53.2

34.6

14.5

13.0

10.9

7.5

9.3

6.2

Permanent supportive housing

11.0

11.6

16.3

14.0

17.8

15.5

7.6

8.3

Public housing

8.3

8.5

17.4

17.5

17.4

17.5

7.1

5.8

Project-based vouchers/Section 8 projects

6.2

7.2

17.5

17.5

18.3

13.5

3.9

5.4

Any permanent housing subsidye

33.0

34.7

17.2

16.5

19.3

19.5

26.0

27.1

Emergency shelterf

83.6

89.7

3.3

2.2

4.4

2.7

3.5

4.3

No use of homeless or housing programsg

18.1

26.9

—

—

—

—

61.5

61.5

N

293

259

—

—

—

—

293

259

PBTH = priority access to project-based transitional housing. UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

The proportions of the PBTH group that used programs other

followup period. Additional detail about the use of transitional

than transitional housing programs are roughly similar to those

housing assistance by PBTH families is shown in Exhibit 5-2.

of the UC group.98 The ninth row shows the percentages of

The exhibit shows that, although the median family who used

families in the PBTH and UC groups who used none of the

transitional housing did so for 13 months, about 31 percent

seven types of programs during the followup period and did

used transitional housing for 20 or more months and 10

not use emergency shelter from the 7th month after random

percent used transitional housing for 30 or more months. So

assignment onward. About 18 percent of PBTH families and 27

even though nearly all PBTH providers in the study generally

percent of UC families fall into this group.

limited assistance to 24 months, some families participated for

a longer period.

The mean and median numbers of months of use for each

program type are also shown in the exhibit (third and fourth

Whereas the previous columns consider all experience between

columns for PBTH families, fifth and sixth columns for UC

randomization and the survey, the last two columns of Exhibit 5-1

families) for only those families who ever used a given program

consider the program use as of the month of the 37-month

type.99 As one might expect, given that transitional housing

survey. Although most outcomes in the report are expected to

was readily available to PBTH families, the number of months

be influenced by assistance received during the entire followup

of transitional housing use is higher for the families assigned

period, some outcomes are expected to be particularly strongly

to the PBTH group (median of 13 months) than for the 35

influenced by assistance received at the time of the followup

percent of UC families who found their way to transitional

survey response. The shaded row of the seventh column shows

housing (median of 8 months) at some point during the

that transitional housing assistance had ended by the followup

98 Although proportions that used any permanent housing subsidy are similar in the PBTH and UC groups, Exhibit 5-5 shows that assignment to the PBTH group caused some delay in the use of permanent housing subsidies.

99 Hence, 0 values are not factored into the means, nor do they pull downward the medians of the various distributions.

3-Year Impacts of Housing and Services Interventions for Homeless Families

62

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-2. Number of Months of Transitional Housing

followup period, program use for PBTH families is dominated

Receipt During Followup Period by PBTH Families Who

by transitional housing. In the second half, the use of transi-

Ever Used TH

tional housing declines steadily. After month 24, all program

use levels off, with only about 36 to 38 percent of families

a

14

using any type of program in any month. The use of permanent

12

housing subsidies steadily increases throughout the followup

period and accounts for more than one-half of monthly pro-

10

gram use after month 26 and two-thirds of monthly program

8

use in month 32.

The bottom panel shows the proportions of program use over

6

time for the UC families. Compared with the PBTH group, the

4

UC group used transitional housing much less (the result the

study design attempted to achieve) and used rapid re-housing

2

and permanent housing subsidies somewhat more extensively

Percent of PBTH families who ever used TH

0

following their initial stay in emergency shelter. Total program

use levels off after the first year, with only a little less than

2 to 3.754 to 5.75

6 to 7.758 to 9.75

40 percent of UC families using any type of program in any

0.25 to 1.75

10 to 11.75

12 to 13.75

14 to 15.75

16 to 17.75

18 to 19.75

29 to 21.75

22 to 23.75

24 to 25.75

26 to 27.75

28 to 29.75

30 or more

Number of months of TH receipt

month. Appendix Exhibit H-3 shows additional information

PBTH = priority access to project-based transitional housing. TH = transitional housing.

about program use in the second half of the followup period,

a Percentages are weighted for survey nonresponse to represent all families in compar-

from month 19 after random assignment until the month of the

ison sample.

37-month survey response.

Note: N = 160.

Source: Family Options Study Program Usage Data

As Exhibits 5-1 and 5-3 make clear, the PBTH families used a

range of programs in addition to the program to which they

survey for most PBTH families (83 percent) who ever received

were referred by the study, a pattern that is consistent with

it and for most UC families (82 percent) who ever received

the design of the study. Families were not required to use the

it.100 In the month of the followup survey response, 9 percent

program to which they were given priority access and were also

of PBTH families (and 6 percent of UC families) were still

not forbidden from using other programs that were available

in transitional housing. About one-fourth of the families in

to them in their community. The intent of the study was to

both groups were receiving some form of permanent housing

maximize use of the programs offered to families assigned to the

subsidy in the month of the followup survey. Most PBTH and

PBTH intervention (in this case, maximize use of the PBTH pro-

UC families (62 percent for both groups) were not participating

grams by the PBTH families) and to create the largest possible con-

in a homeless or housing program at the time they responded

trast between the program mixes of different assignment groups

to the followup survey. Thus, differences are expected in the

(in this case, PBTH versus UC). As shown in the exhibits, the

outcomes of PBTH and UC families only to the extent that

use of transitional housing was different for the PBTH and

these outcomes reflect a lasting influence of families having

UC groups. The contrast in use of transitional housing—53.2

been offered temporary rental assistance to help them leave

percent for PBTH families and 34.6 percent for UC families—is

homelessness.

sizable, although smaller than the analogous contrast in either

Exhibit 5-3 provides a more detailed picture of the timing of

the SUB-versus-UC or CBRR-versus-UC comparisons.

program use by the families in the PBTH-versus-UC compari-

As is conventional in random assignment analyses, the goal in

son. The top panel shows the proportions of families within the

this study is to estimate the intention-to-treat (ITT) impact—

PBTH group who received different types of assistance during

that is, the impact of offering a program to families, regardless

each calendar month for the first 32 months after random

of whether they actually used that program (or some other

assignment.101, 102 It is not surprising that, in the first half of the

100 Because 9.3 percent of PBTH families were still using transitional housing in the followup survey month, and because 53.2 percent had used transitional housing at some point during the followup period, it can be calculated that 1 – (9.3/53.2) = 82.5 percent of PBTH families who used transitional housing at some point had stopped using it by the survey month. A similar calculation, 1 – (6.2/34.6), yields 82.1 percent for UC families.

101 Month 32 is the latest month for which the study team has data for all the families who responded to the 37-month survey.

102 Exhibit 5-3 is closely related to the analysis of the costs of this pairwise comparison presented in Chapter 9. The reader should note that the exhibit does not indicate the two-way flows of families moving into and out of these program types from month to month. Instead, it reflects only the overall usage level in a given month.

3-Year Impacts of Housing and Services Interventions for Homeless Families

63

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-3. PBTH Versus UC: Program Use in Each Month of Followup Period

Panel A: Program Use of PBTH Families for 32 Months After RA

100

90

80

Unknown/no program use

70

60

50

40

Any permanent housing subsidy

30

Rapid re-housing

20

Percent of families using program type in month

Transitional housing

10

Emergency shelter

0

0

1

2

3

4

5

6

7

8

9

10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

Panel B: Program Use of UC Families for 32 Months After RA

100

90

80

70

Unknown/no program use

60

50

40

Any permanent housing subsidy

30

Percent of families using program type in month

Rapid re-housing

20

Transitional housing

10

Emergency shelter

0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

Month after RA

PBTH = priority access to project-based transitional housing. UC = usual care.

RA = random assignment.

Note: Families who have more than one type of program use in a calendar month are counted fractionally in each type.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

64

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) program). This goal is consistent with the policy option of

The Short-Term Impacts report (Gubits et al., 2015) found

making a treatment available to a family but without the ability

that assignment to the PBTH group reduced the proportion of

to require a family to use that treatment.

families who reported stays in shelters or places not meant for

human habitation in the 6 months before the 20-month survey.

Because not all families randomly assigned to the PBTH group

Assignment to the PBTH intervention also reduced the pro-

used transitional housing and because some families assigned to

portion of families who had a stay in emergency shelter during

the UC group did use transitional housing, the true ITT impact

months 7 to 18 of the followup period. These findings may

is likely smaller than it would have been had the gap in tran-

reflect the fact that about 22 percent of families assigned to the

sitional housing usage been wider (assuming that transitional

PBTH group were using transitional housing at the time of the

housing truly has a nonzero impact on families who use it). In

20-month survey compared with only 10 percent of families

particular, given the relatively small sample size available for

assigned to the UC group.

analysis, the difference in the use of transitional housing by

the PBTH and UC groups is narrow enough that the study may

Exhibit 5-4 shows the impacts of assignment to the PBTH in-

have failed to detect as statistically significant one or more ITT

tervention on homelessness, housing independence, residential

impacts large enough to be of policy importance.

moves, and housing quality 3 years after random assignment,

compared with usual care.103 The 37-month analysis finds a

The remainder of the chapter reports estimated impacts in the

difference between the PBTH and UC groups on only a single

various outcome domains that—if statistically significant—can

homelessness measure—the proportion of families who had a

be causally attributed to the offer of project-based transitional

stay in emergency shelter during months 21 to 32 after random

housing to the families randomly assigned to the PBTH group

assignment. Assignment to the PBTH group reduced this pro-

in contrast with no such directed referral or privileged access

portion relative to assignment to the UC group by 6 percentage

being available to families randomly assigned to the UC group.

points, from 15 to 9 percent. The analysis did not find evidence

of impact on any other indicators of homelessness, housing

5.3. Impacts on Housing Stability

independence, number of moves, or housing quality.

in the Project-Based Transitional

Exhibit 5-5 shows the month-by-month impacts on the propor-

Housing (PBTH) Versus Usual Care

tion of families who had at least 1 night in emergency shelter

(UC) Comparison

during the month. This exhibit illustrates that a somewhat lower

proportion of the PBTH group was in emergency shelter during

Proponents of project-based transitional housing emphasize

most of the followup period compared with the proportion of

that most families who become homeless have barriers that

the UC group.104 This reduction in shelter use, however, mostly

make it difficult for them to secure and maintain housing.

disappears by the end of the followup period. Therefore, the

Thus, housing subsidies alone may be insufficient to ensure

reduction in shelter use observed during months 21 to 32 after

housing stability and other desirable outcomes. Family needs

random assignment can be attributed largely to a difference

may arise from poverty, health, disability, or other problems

during months 21 to 26. The lack of a statistically significant

that led to homelessness to begin with or from the disruptive

difference in shelter use during the 6 months before the follow-

effects of homelessness on parents and children. Proponents of

up survey (shown in the fifth row of Exhibit 5-4) is further

project-based transitional housing believe that, by addressing

evidence that the reduction in shelter use observed in earlier

these barriers and needs in a supervised residential setting,

months had faded by the end of the third year. These findings

PBTH programs lay the best foundation for ongoing stability.

support the proposition that reductions in study-defined home-

The services offered by the PBTH programs would be expected

lessness and shelter use observed in the short-term survey were

to affect stability, adult well-being, and self-sufficiency directly

a direct consequence of the differential in transitional housing

and affect family preservation and child well-being more indi-

use deliberately induced by the study, rather than a difference

rectly. The PBTH-versus-UC comparison offers evidence about

in post-program stability.

whether this approach improves stability and other family

outcomes 3 years after random assignment.

103 The homeless outcomes in this study diverge from the homeless definition final rule in that they do not include stays in transitional housing in their definitions of being homeless. Additional impacts on the use of transitional housing during the followup period are provided in Appendix E.

104 Statistically significant differences appear in 16 of the 32 calendar months (months 2 to 8, 13, 17 to 23, and 25). Because of missing data on baseline stays, less than 100 percent of all PBTH and UC families are observed in shelter during month 0 (the calendar month of random assignment). The difference in the month 0 (significant at the .05 level) should be considered a chance difference. The difference in unobserved initial shelter stays should be increasingly unrelated to differences observed later in the followup period (because the initial shelter stays that are unobserved in the data become increasingly likely to have ended as time elapsed in the followup period).

3-Year Impacts of Housing and Services Interventions for Homeless Families

65

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-4. PBTH Versus UC: Impacts on Housing Stability at 37 Months

PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the followup period

At least 1 night homelessb or doubled up in past 6 months or in shelter in

292

40.5

(49.4)

259

40.2

(49.2)

0.3

(4.7)

0.01

past 12 months (%) [confirmatory]c

At least 1 night homelessb or doubled up in past 6 months (%)

292

37.7

(48.9)

259

37.5

(48.5)

0.2

(4.7)

0.00

At least 1 night homelessb in past 6 months (%)

293

18.6

(39.9)

259

18.3

(38.3)

0.3

(3.6)

0.01

At least 1 night doubled up in past 6 months (%)

292

29.2

(45.8)

259

31.7

(46.8)

– 2.5

(4.4)

– 0.05

Any stay in emergency shelter in past 6 months (%) [Program Usage Data]

293

7.3

(26.4)

259

6.3

(25.5)

1.1

(2.4)

0.03

Any stay in emergency shelter in months 21 to 32 after RA (%)

293

9.1

(29.4)

259

15.1

(35.8)

– 6.0**

(2.9)

– 0.14

[Program Usage Data]

Number of days homelessb or doubled up in past 6 months

292

46.8

(72.6)

258

49.3

(74.4)

– 2.5

(6.9)

– 0.03

Number of days homelessb in past 6 months

293

14.5

(44.1)

259

16.1

(44.2)

– 1.6

(4.0)

– 0.03

Number of days doubled up in past 6 months

292

34.6

(64.2)

258

35.9

(65.4)

– 1.3

(6.1)

– 0.02

Housing independence

Living in own house or apartment at followup (%)

293

65.0

(47.5)

259

71.6

(46.0)

– 6.6

(4.2)

– 0.12

Living in own house or apartment with no housing assistance (%)

292

40.0

(49.3)

257

47.1

(49.7)

– 7.2

(4.4)

– 0.13

Living in own house or apartment with housing assistance (%)

292

25.0

(43.2)

257

24.1

(44.0)

0.9

(3.7)

0.02

Number of places lived

Number of places lived in past 6 monthsd

290

1.6

(1.0)

259

1.6

(1.0)

0.0

(0.1)

– 0.03

Housing quality

Persons per room

275

1.8

(1.3)

243

1.7

(1.3)

0.0

(0.1)

0.01

Housing quality is poor or fair (%)

273

34.3

(47.2)

243

30.2

(45.4)

4.1

(4.5)

0.08

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is not statistically significant at the .10 level for the PBTH-versus-UC comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

Exhibit 5-5 also shows month-by-month impacts on the use

5.4. Impacts on Family Preservation

of any form of permanent housing subsidy for the PBTH and

in the Project-Based Transitional

UC families. The exhibit shows that, throughout most of the

Housing (PBTH) Versus Usual Care

followup period, smaller proportions of PBTH families received

assistance from permanent housing subsidies (including per-

(UC) Comparison

manent supportive housing programs and mainstream housing

Exhibit 5-6 shows the impacts of assignment to the PBTH

programs) compared with proportions of UC families. Families

group relative to impacts of assignment to the UC group

assigned to the PBTH group presumably were pursuing PBTH

regarding family preservation outcomes. Any effects of

referrals they received at the time of random assignment and

assignment to the PBTH group on family preservation would be

were thus less inclined to seek permanent subsidy assistance

expected to be indirect, arising via effects on housing stability,

than were their counterparts in the UC group. The proportions

self-sufficiency, and adult well-being. At 3 years after random

of the PBTH families using any permanent housing subsidy

assignment, no evidence of effects on family separations from

are about 4 percentage points lower than the corresponding

children or from spouses or partners who were with the family

proportions for the UC group in months 6 to 16 (and are statis-

at baseline was found. Nor was evidence of effects on reunifi-

tically significantly different in month 6 and months 8 to 10).

cations of the much smaller number of family members who

The reduction disappears by the end of the followup period,

were separated from the family at baseline found. No effect was

when use of permanent housing subsidies by the PBTH group

detected on foster care placements.

catches up to that of the UC group.

3-Year Impacts of Housing and Services Interventions for Homeless Families

66

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-5. PBTH Versus UC: Percent of Families With Any Stay in Emergency Shelter and Any Permanent Housing Subsidy During Month, by Number of Months after RA

100

90

80

70

60

50

Percent of families

40

UC: Used emergency shelter

30

UC: Used any permanent housing subsidy

20

PBTH: Used

emergency

PBTH: Used any permanent housing subsidy

shelter

10

0

0

5

10

15

20

25

30

35

Month after RA

PBTH = priority access to project-based transitional housing. UC = usual care.

RA = random assignment.

Notes: Percentages are weighted for survey nonresponse to represent all UC families in the study. Missing data on emergency shelter stays biases the percentages somewhat downward. The baseline stay in emergency shelter does not appear in the data for 18.7 percent of UC respondent families. The missing data rate for subsequent stays in emergency shelter is unknown.

Source: Family Options Study Program Usage Data

Exhibit 5-6. PBTH Versus UC: Impacts on Family Preservation at 37 Months

PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

283

20.9

(40.9)

252

18.9

(38.7)

2.0

(3.5)

0.05

Family has at least one foster care placement in past 6 monthsb (%)

286

3.2

(19.3)

255

4.6

(19.4)

– 1.4

(1.8)

– 0.07

Spouse/partner separated in past 6 months, of those with spouse/partner

82

38.9

(47.3)

80

27.1

(47.1)

11.7

(7.8)

0.20

present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families at least one child

56

40.6

(49.9)

54

44.7

(48.2)

– 4.1

(11.3)

– 0.07

absent at RA (%)

Spouse/partner reunified, of those with spouse/partner at RA (%)

28

24.9

(47.6)

28

23.6

(35.6)

1.3

(14.0)

0.03

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

67

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-7 reports impacts on outcomes from the child welfare

suggests that focusing on psychosocial challenges in supervised

agency administrative data collected in 5 of the 12 study sites.

settings lays the groundwork for later success in independent

No effect was detected on either the proportion of family

housing. At 20 months, little evidence indicated that focusing

heads with a formal child separation that began after random

on psychosocial challenges could have a direct effect on

assignment or on the total days during the followup period of

psychosocial outcomes (that is, the first step in the theory); the

being separated from at least one child.

PBTH intervention affected only 1 outcome of 8 in the adult

well-being domain—reducing drug abuse. Nearly one-fourth

5.5. Impacts on Adult Well-Being

(22 percent) of families assigned to the PBTH group were in

transitional housing programs at the 20-month followup sur-

in the Project-Based Transitional

vey, and, given median stays of 12 months, others had left only

Housing (PBTH) Versus Usual Care

recently. It is thus plausible that additional benefits of intensive

(UC) Comparison

services provided in PBTH programs may emerge during the

longer term. As shown in Exhibit 5-8, at 37 months, evidence

Adult well-being is a central focus of PBTH programs, and

again indicates impact on only one outcome; assignment to

the theory underlying project-based transitional housing

the PBTH intervention reduced the proportion of family heads

Exhibit 5-7. PBTH Versus UC: Impacts on Child Welfare Outcomes

PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Had a formal child separation that began after RA (%)

177

14.9

(35.5)

157

16.9

(37.9)

– 2.0

(4.0)

– 6.03

Total days during followup separated from at least one childb

177

97.5 (258.2)

157

116.4 (282.8) – 18.8

(25.7)

– 0.08

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days.

Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions. Sample limited to five sites where child welfare records were collected (Alameda County, Baltimore, Kansas City, Minneapolis, and Phoenix).

Source: State child welfare agency records

Exhibit 5-8. PBTH Versus UC: Impacts on Adult Well-Being at 37 Months

PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

291

32.1

(46.0)

258

32.8 (47.8)

– 0.7

(4.1)

– 0.01

Adult mental health

Goal-oriented thinkingb

287

4.41

(1.04)

258

4.38 (1.02)

0.03

(0.09)

0.03

Psychological distressc

292

6.75

(5.49)

258

6.83 (5.76)

– 0.07

(0.45)

– 0.01

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

291

16.9

(39.2)

256

23.5 (43.2)

– 6.6*

(3.5)

– 0.14

Adult substance use

Alcohol dependence or drug abused (%)

293

13.4

(34.4)

258

10.6 (31.2)

2.8

(2.8)

0.07

Alcohol dependenced (%)

293

10.4

(30.8)

258

7.4 (25.5)

3.0

(2.5)

0.09

Drug abused (%)

293

5.3

(23.4)

259

6.1 (23.4)

– 0.8

(2.1)

– 0.03

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

292

7.5

(27.0)

258

8.5 (29.6)

– 1.1

(2.4)

– 0.03

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

68

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) experiencing symptoms of post-traumatic stress disorder

5.7. Impacts on Self-Sufficiency

(PTSD) by 7 percentage points relative to a proportion of 24

in the Project-Based Transitional

percent in the UC group. No evidence indicates that assign-

Housing (PBTH) Versus Usual Care

ment to the PBTH group had an effect on the other measures

of physical or mental health, intimate partner violence, or

(UC) Comparison

substance dependence or abuse relative to usual care.

Along with housing stability and adult well-being, self-sufficiency

is also a central focus of PBTH programs. The 20-month analy-

5.6. Impacts on Child Well-Being

sis found no effects of assignment to the PBTH group on any of

in the Project-Based Transitional

the self-sufficiency outcomes examined. Exhibit 5-11 shows a

Housing (PBTH) Versus Usual Care

significant impact on only 1 of the 20 self-sufficiency outcomes

(UC) Comparison

during the 3-year followup period; assignment to the PBTH

reduced the number of weeks that family heads reported hav-

The theory behind project-based transitional housing suggests

ing participated in school or training programs since random

that effects of assignment to the PBTH group on children may

assignment by 3.5 weeks. Families assigned to the UC group in

take longer to develop, particularly if the PBTH intervention

this comparison reported 8.9 weeks in school or training since

influences housing stability or adult well-being. In the absence of

random assignment and families assigned to the PBTH group

widespread effects of the PBTH intervention on these out comes,

reported 5.4 weeks.

as shown in previous sections, however, it is unsurprising that the

Exhibit 5-12 shows impact estimates from administrative data

team finds no impact on child well-being. None of the 9 cross-

on outcomes for quarters 11 to 14 after the quarter of random

age tests in Exhibit 5-9 and 2 of the 20 age-specific tests in

assignment. The analysis of administrative data finds no effect

Exhibit 5-10 reach statistical significance, 1 in each direction:

of assignment to the PBTH group relative to assignment to the

young children in families assigned to the PBTH group show

UC group on total earnings, the proportion of family heads em-

lower levels of executive functioning, but older children show

ployed, or the number of quarters employed during the year.

more goal-oriented thinking. Given the small number of effects

and their inconsistent direction, results are best interpreted as

Simply by chance, the study team would have expected effects

showing little overall impact on child well-being.

on some outcomes. Thus, an effect on only 1 of 23 outcomes is

Exhibit 5-9. PBTH Versus UC: Impacts on Child Well-Being Across Age Groups at 37 Months PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

340

2.2

(1.0)

301

2.1

(0.9)

0.1

(0.1)

0.05

Grade completion (not held back) (%)

286

86.68 (33.62)

252

91.35 (27.69) – 4.66

(3.02)

– 0.12

School gradesc

253

3.0

(0.9)

230

3.1

(0.9)

– 0.0

(0.1)

– 0.03

Child physical health

Poor or fair health in past 30 days (%)

395

5.9

(21.4)

347

5.9

(22.2)

0.0

(2.2)

0.00

Well-child checkup in past year (%)

395

88.7

(32.1)

346

91.2

(28.2)

– 2.5

(2.9)

– 0.07

Child has regular source of health care (%)

391

88.2

(29.3)

350

90.3

(26.7)

– 2.1

(3.8)

– 0.06

Sleep problemsd

394

2.17

(1.10)

349

2.28

(1.09) – 0.11

(0.10)

– 0.08

Child behavioral strengths and challenges

Behavior problemse

375

0.53

(1.19)

325

0.59

(1.29) – 0.06

(0.12)

– 0.03

Prosocial behaviorf

375 – 0.26

(1.22)

327

– 0.34

(1.13)

0.07

(0.10)

0.05

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

3-Year Impacts of Housing and Services Interventions for Homeless Families

69

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-10. PBTH Versus UC: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

118

33.3

(48.3)

104

41.7

(49.1)

– 8.4

(6.6)

– 0.14

Child care or preschool absences in past monthd

46

0.59

(0.80)

40

0.74

(1.04) – 0.15

(0.21)

– 0.14

Positive child care or preschool experiencese

51

0.77

(0.53)

43

0.83

(0.39) – 0.06

(0.10)

– 0.13

Positive child care or preschool attitudesf

50

4.34

(1.16)

42

4.57

(0.80) – 0.23

(0.23)

– 0.28

Child care or preschool conduct problemsg (%)

55

9.0

(33.6)

43

9.9

(29.4)

– 0.9

(5.8)

– 0.03

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

99

64.4

(47.7)

95

70.7

(45.3)

– 6.3

(7.3)

– 0.11

Ages 3 years, 6 months to 7 years

Verbal abilityi

133 – 0.42

(1.06)

115

– 0.28

(1.10) – 0.13

(0.17)

– 0.10

Math abilityj

133 – 0.35

(1.00)

116

– 0.28

(1.12) – 0.07

(0.17)

– 0.06

Executive functioningk (self-regulation)

121

15.51 (15.77)

108

19.76 (16.75) – 4.25** (1.86)

– 0.21

Ages 5 to 17 yearsl

School enrollmentc (%)

279

95.8

(14.5)

247

99.1

(11.0)

– 3.4

(2.7)

– 0.17

School absences in past monthd,m

102

0.95

(0.94)

97

1.14

(1.04) – 0.19

(0.15)

– 0.15

Positive school experiencese,m

105

0.54

(0.64)

98

0.38

(0.67)

0.16

(0.10)

0.19

Positive school attitudesf,m

105

4.13

(1.09)

98

4.06

(1.14)

0.07

(0.14)

0.05

School conduct problemsg,m (%)

105

27.7

(45.8)

98

27.3

(46.7)

0.3

(6.9)

0.01

Ages 8 to 17 years

Anxietyn

168

34.88

(7.72)

158

35.81

(6.98) – 0.93

(0.95)

– 0.09

Fearso

171

62.65 (14.68)

158

64.62 (14.21) – 1.97

(1.58)

– 0.10

Substance usep (%)

163

10.00 (32.91)

152

8.62 (29.01)

1.38

(3.79)

0.04

Goal-oriented thinkingq

161

22.75

(4.98)

150

21.55

(4.73)

1.21*

(0.64)

0.19

School effort in past monthr

170

2.74

(0.84)

155

2.75

(0.74) – 0.01

(0.08)

– 0.01

Arrests or police involvement in past 6 monthss (%)

83

15.65 (36.57)

91

9.31 (26.79)

6.34

(4.82)

0.17

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Notes: Impact estimates and outcome means are regression-adjusted for baseline characteristics and are weighted to adjust for survey non-response.

See Chapter 2 and Appendix B for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

3-Year Impacts of Housing and Services Interventions for Homeless Families

70

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) Exhibit 5-11. PBTH Versus UC: Impacts on Self-Sufficiency at 37 Months

PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

293

38.3

(49.0) 259

37.7

(48.3)

0.6

(4.1)

0.01

Any work for pay since 20-month surveyb (%)

259

56.5

(49.5) 221

63.6

(48.7)

– 7.1

(4.4) – 0.15

Months worked for pay since 20-month surveyb,c

257

7.1

(8.3) 219

7.8

(8.6)

– 0.7

(0.8) – 0.09

Any work for pay since RA (%)

293

70.1

(45.0) 259

71.5

(46.1)

– 1.4

(3.8) – 0.03

Months worked for pay since RAc

290

13.3

(13.7) 252

14.3

(14.6)

– 1.0

(1.2) – 0.07

Hours of work per week at current main jobd

292

12.6

(18.0) 259

12.2

(16.5)

0.5

(1.5)

0.02

Income sources and amounts

Annualized current earnings ($)

288

7,429 (12,113) 252

6,367 (9,966)

1062

(962)

0.09

Total family income ($)

290 12,987 (11,654) 247 13,178 (11,338) – 191

(1,038) – 0.01

Anyone in family had earnings in past month (%)

293

54.6

(49.9) 259

55.0

(50.0)

– 0.4

(4.3) – 0.01

Anyone in family received TANF in past month (%)

292

24.9

(43.0) 259

21.1

(40.4)

3.8

(3.8)

0.08

Anyone in family received SSDI in past month (%)

293

8.0

(26.4) 259

9.0

(29.1)

– 1.0

(2.2) – 0.03

Anyone in family received SSI in past month (%)

292

13.4

(33.7) 259

15.2

(38.6)

– 1.8

(2.5) – 0.04

Anyone in family received SNAP/Food Stamps in past month (%)

292

82.1

(38.9) 259

86.1

(35.5)

– 4.0

(3.0) – 0.09

Anyone in family received WIC in past month (%)

292

28.1

(46.0) 259

24.7

(42.8)

3.5

(3.8)

0.07

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

290

35.8

(48.4) 259

37.7

(48.4)

– 1.8

(4.5) – 0.03

Number of weeks in school/training programs since RA

289

5.4

(12.3) 254

8.9

(17.3)

– 3.5**

(1.6) – 0.17

Participated in 2 weeks or more of school since RA (%)

290

9.8

(30.5) 259

12.1

(32.1)

– 2.3

(3.1) – 0.06

Participated in 2 weeks or more of basic education since RA (%)

290

2.3

(13.0) 259

1.0

(12.4)

1.3

(1.2)

0.07

Participated in 2 weeks or more of vocational education since RA (%)

290

9.5

(30.5) 259

11.6

(31.1)

– 2.1

(2.9) – 0.05

Food security and

Household is food insecure (%)

293

46.0

(49.8) 259

47.9

(50.1)

– 1.9

(4.6) – 0.03

Food insecurity scalee

289

1.87

(2.02) 257

1.90

(2.04) – 0.02

(0.19) – 0.01

Economic stressors

Economic stress scalef

290

– 0.13

(0.51) 257

– 0.11

(0.47) – 0.02

(0.04) – 0.04

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 5-12. PBTH Versus UC: Earnings and Employment

PBTH

UC

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Earnings in quarters 11 to 14 after RA (2015Q3$)

349

6,496 (10,179) 314

5,690 (9,365)

806

(759)

0.08

Any employment in quarters 11 to 14 after RA (%)

349

56.1

(49.7) 314

57.1

(49.6)

– 1.0

(3.9)

– 0.02

Number of quarters employed in quarters 11 to 14 after RA

349

1.7

(1.7) 314

1.7

(1.7)

0.0

(0.1)

– 0.01

PBTH = priority access to project-based transitional housing. UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Note: See Chapter 2 and Appendix B for outcome definitions.

Source: Quarterly wage records from the National Directory of New Hires

3-Year Impacts of Housing and Services Interventions for Homeless Families

71

FAMILY OPTIONS STUDY Chapter 5. Impacts of Project-Based Transitional Housing (PBTH) Compared With Usual Care (UC) probably best interpreted as resulting from chance and not as

reduced the proportion of family heads experiencing PTSD

evidence of an effect of assignment to the PBTH intervention.

symptoms relative to usual care. Apart from this beneficial

The team therefore concluded that no evidence indicates

effect, the vast majority of indicators in the adult well-being,

self-sufficiency effects for assignment to the PBTH group relative

family preservation, child well-being, and self-sufficiency

to assignment to the UC group at the 3-year followup survey.

domains revealed equivalent results among families who were

assigned to the PBTH group and families who were assigned to

5.8. Summary of Project-Based

the UC group.

Transitional Housing (PBTH) Versus

The general lack of impacts on adult well-being and family

Usual Care (UC) Comparison Across self-sufficiency is noteworthy, given the emphasis placed by Domains

PBTH programs on delivering help and improvement in these

domains. Only 2 of the 28 indicators examined for results

For the PBTH-versus-UC comparison, 53 percent of families

in this respect showed any impact from assignment to the

assigned to the PBTH group and 35 percent of families assigned

PBTH intervention. Overall, evidence did not emerge 3 years

to the UC group accessed transitional housing during the

after random assignment to the PBTH group that the goals of

followup period. This contrast in program use is smaller than

project-based transitional housing as a distinctive approach

for other comparisons of active interventions with usual care.105

to assisting families facing unstable housing situations had

been achieved. One potential reason for the lack of statistically

By 3 years after random assignment, having priority access to

significant effects in the PBTH-versus-UC comparison is that

project-based transitional housing led to generally equivalent

services similar to those that the PBTH intervention provided

housing stability outcomes as did usual care. By this point,

were, in many cases, available to families in emergency shelter.

the reduction in emergency shelter use that was observed

Chapter 6 reports about how the PBTH intervention compares

through most of the followup period had disappeared. In the

with the other two active interventions: SUB and CBRR.

adult well-being domain, assignment to the PBTH intervention

105 Although the takeup rate for PBTH programs provides a weaker test of the intervention than might be hoped for, the low takeup of transitional housing on the part of many families assigned to the PBTH group is of policy interest. It is not clear to what extent this low takeup represents families who decline programs or programs that decline families. Qualitative data from a small number of families in this study (80 in all, 19 assigned to the PBTH group) suggest that both processes were important. When families declined offers, the location of programs was often an issue. Families offered permanent housing subsidies and community-based rapid re-housing had more opportunity to live in neighborhoods of their choice that were near jobs, children’s schools, and support networks (Fisher et al., 2014).

3-Year Impacts of Housing and Services Interventions for Homeless Families

72

FAMILY OPTIONS STUDY

CHAPTER 6.

IMPACTS OF PERMANENT HOUSING

SUBSIDY (SUB) COMPARED WITH

COMMUNITY-BASED RAPID RE-

HOUSING (CBRR), SUB COMPARED

WITH PROJECT-BASED TRANSITIONAL

HOUSING (PBTH), AND CBRR

COMPARED WITH PBTH

This chapter presents 37-month impact estimates were assigned to one of the interventions. This approach enfor the three pairwise comparisons of interventions

sures that the groups of families being compared are as similar

that do not involve usual care. These pairwise

as possible. It also means, however, that the group of families

comparisons contrast assignment to the SUB, CBRR, and PBTH

representing each intervention differs, depending on the group

interventions with each other. In each of these comparisons,

with which it is compared. For example, the group of SUB

the goal is to determine the extent to which, 3 years after study

families in the SUB-versus-CBRR comparison overlaps but is

entry, priority access to a particular type of program leads to

not identical to the group of SUB families in the SUB- versus-

better, worse, or no different family outcomes than those expe-

PBTH comparison.106 If the group of families representing an

rienced by families who were eligible for that type of program

intervention were always the same, then the results for any

but were offered priority access to another type of program.

comparison of interventions could be logically deduced from

The chapter begins with a brief description of the analysis

the comparison of each intervention with usual care. Since the

samples for these comparisons. Next, it addresses the SUB-

groups of families representing an intervention are not always

versus-CBRR comparison, first showing how much families in

the same, it is not possible to logically deduce the results of the

the SUB and CBRR groups used housing and services programs

comparisons from the comparisons of each intervention with

available to them, and then presenting effects on outcomes in

usual care. Some signs and magnitudes of estimates differ from

the five study domains. The same set of information is then

those suggested by the comparisons with usual care. Thus,

presented in turn for the SUB-versus-PBTH and CBRR-versus-

this chapter describes the pairwise comparisons of the active

PBTH comparisons.

interventions with each other.

Exhibit 6-1 shows the number of families who are included

6.1. Analysis Samples for Pairwise

in the various pairwise comparisons that comprise the entire

Comparisons

study. Each column of the exhibit shows the number of

families on both sides of a particular comparison. Each row

As addressed previously in this report, families were included

shows how the number of families representing a particular

in comparisons of two interventions only if they passed the

intervention varies by pairwise comparison.

initial screening for an available slot for each intervention and

106 By definition, usual care was available to all study families. Therefore, the samples of SUB, CBRR, and PBTH families in the comparisons with UC families contain all the families assigned to those interventions. The samples in the three pairwise comparisons of the active interventions with each other are subsets of these larger samples.

3-Year Impacts of Housing and Services Interventions for Homeless Families

73

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-1. Sample Sizes in the Six Pairwise Comparisons

Sample Size in Pairwise Comparisona

Assigned

SUB

CBRR

PBTH

SUB

SUB

CBRR

Intervention

Versus

Versus

Versus

Versus

Versus

Versus

UC

UC

UC

CBRR

PBTH

PBTH

SUB

501

—

—

362

215

—

CBRR

—

434

—

290

—

180

PBTH

—

—

293

—

201

184

UC

395

434

259

—

—

—

Total

896

868

552

652

416

364

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

a Sample sizes are numbers of families who responded to the 37-month followup survey.

Source: Family Options Study 37-month followup survey

6.2. The Permanent Housing

6.2.1. Program Use by Families in the SUB-

Subsidy (SUB) Versus Community-

Versus-CBRR Comparison

Based Rapid Re-Housing (CBRR)

Exhibit 6-2 shows the use of eight types of homeless and

Comparison

housing programs by the 362 SUB families and 290 CBRR

families analyzed in the SUB-versus-CBRR comparison.108

The SUB-versus-CBRR comparison contrasts the permanent

The first column shows the same general pattern of usage for

housing subsidy of the SUB intervention with the temporary

these SUB families as for all SUB families (see Exhibit 3-2 in

rental assistance (usually lasting 7 to 8 months, but potentially

Chapter 3). Likewise, the proportions of these CBRR families

renewable for up to 18 months) of the CBRR intervention. Both

shown in the second column are similar to the proportions of

these interventions provided priority access to the rental subsi-

all CBRR families (see Exhibit 4-1 in Chapter 4). The first two

dy, and both required families to engage in a housing search to

columns show that 82 percent of families assigned to the SUB

locate a suitable private-market rental unit.107 The supportive

group used the permanent housing subsidies they were offered

services provided in the SUB intervention were limited to

and 61 percent of CBRR families used rapid re-housing rental

assistance with finding housing. The services offered by CBRR

assistance at some point during the period of observation. The

providers were also focused on the housing search. In addition

number of months of program use during the whole followup

to case management and assistance with housing search, CBRR

period (in columns three through six) and the proportions

programs provided services such as assistance obtaining public

using a particular program in the month of the followup survey

benefits and financial literacy/money management information

response (in columns seven and eight) are also similar to those

to most of the families assigned who used the rapid re-housing

in the previous exhibits. A large difference between the groups

program to which they received priority access.

exists in the proportion of families participating in any program

during the month of the followup survey, with 75 percent of

The subsidy in both interventions represented a substantial

SUB families participating in some program (most using perma-

fraction of monthly rent. Beyond the length of the subsidy, a few

nent housing subsidies they were offered) and only 42 percent

differences in the administration of the programs are noteworthy.

of CBRR families participating in any type of program.

A standard formula set the subsidy amount in SUB programs,

but subsidy determination in CBRR programs varied among

As expected, because of priority access to permanent housing

providers, typically allowing for at least some case manager

subsidies, the proportion that ever used any form of permanent

discretion in setting the subsidy amount. To continue to receive

subsidy was higher for families assigned to the SUB group (87

rapid re-housing assistance, families had to have incomes below

percent) than for families assigned to the CBRR group (38 per-

certain thresholds. Most CBRR programs asked questions about

cent). At the 37-month survey, 72 percent of SUB families were

income every 3 months as part of the recertification process to

using some form of permanent housing subsidy compared with

assess continued need for assistance. This frequency was much

33 percent of CBRR families. This differential is 15 percentage

greater than the annual income recertification that providers of

points narrower than it was at the time of the 20-month survey,

a permanent housing subsidy required.

107 The minimal share (about 8 percent) of SUB families provided with public housing in Honolulu or with project-based vouchers in Bridgeport did not need to engage in housing searches.

108 The SUB-versus-CBRR comparison sample consists of 435 families assigned to the SUB group and 382 families assigned to the CBRR group. Of those 817 families, 362

SUB families and 290 CBRR families (80 percent) responded to the 37-month followup survey.

3-Year Impacts of Housing and Services Interventions for Homeless Families

74

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-2. SUB Versus CBRR: Program Use Since RA

Percent Ever Used

Number of Months Used

Percent Used in

From RA to 37-Month

From RA to 37-Month Followup Survey,

Month of Followup

Type of Homeless or

Followup Surveya

if Ever Used Type of Assistance

Survey Response

Housing Assistance

SUB

CBRR

SUB

CBRR

SUB

CBRR

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

82.3

9.8

31.2

32.5

20.7

21.5

67.4

8.4

SUB groupb

Rapid re-housingc

14.7

60.5

6.1

4.5

8.8

7.5

0.3

2.6

Transitional housingd

7.4

19.8

7.3

5.3

9.1

7.5

0.9

4.1

Permanent supportive housing

3.9

11.6

11.4

9.5

16.1

12.0

3.1

8.0

Public housing

1.1

10.9

23.3

24.5

19.4

18.5

1.2

10.0

Project-based vouchers/Section 8 projects

1.1

7.0

11.1

14.5

16.9

17.5

0.6

6.4

Any permanent housing subsidye

87.4

38.4

30.3

32.5

18.7

18.5

72.3

32.6

Emergency shelterf

84.9

90.4

3.0

2.0

4.6

2.3

1.8

2.5

No use of homeless or housing programsg

4.5

9.5

—

—

—

—

24.7

58.3

N

362

290

—

—

—

—

362

290

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

when use of permanent housing subsidies had decreased for

that, relative to assignment to the SUB intervention, assignment

the SUB group (by 5 percentage points) and had increased for

to the CBRR intervention will reduce the length of the shelter

the CBRR group (by 10 percentage points).

stay at the time of study entry.

6.2.2. Impacts of the SUB Intervention

The Family Options Study: Short-Term Impacts of Housing and

Services Interventions for Homeless Families (hereafter, the Short-Term

Compared With the CBRR Intervention

Impacts report; Gubits et al., 2015) found that, 20 months after

The study team hypothesizes that assignment to the SUB

random assignment, assignment to the SUB intervention reduced

intervention relative to assignment to the CBRR intervention

homelessness and doubling up compared with assign ment to the

will reduce homelessness and improve housing stability. The

CBRR intervention. It also led to having more families living in

hypothesis is based on the premise that many of the very poor

their own place and experiencing fewer residential moves. In

families who experience homelessness will need long-term

domains other than housing stability, the SUB intervention had

rental subsidies to remain stably housed. The magnitude of

a few beneficial impacts compared with the CBRR intervention,

this expected difference has been unknown before this study,

but it reduced family income and work for pay. Both inter-

however. Differential effects on more distal outcomes are

ventions reduced the length of the initial shelter stay by the

theorized to depend on the magnitude of the housing stability

same amount—about one-half month. The remainder of this

effect. To the extent that permanent housing subsidies provide

section presents the impact evidence of how these interventions

greater residential stability or reduce parental stress (stemming

compare at 3 years after random assignment.

from moves or from fear of homelessness) more than temporary

subsidies do, the benefits of assignment to the SUB intervention

Impacts on Housing Stability in the SUB-Versus-

in other areas such as child well-being and family preservation

CBRR Comparison

may be larger than the effects of the CBRR intervention.

Exhibit 6-3 shows the effect on housing stability of being

Although assistance from CBRR programs is temporary rather

assigned to the SUB group relative to being assigned to the

than permanent, its emphasis on restoring families to conven-

CBRR group. The first panel of Exhibit 6-3 shows that, relative

tional housing as swiftly as possible leads the team to expect

to assignment to the CBRR intervention, assignment to the SUB

3-Year Impacts of Housing and Services Interventions for Homeless Families

75

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-3. SUB Versus CBRR: Impacts on Housing Stability at 37 Months

SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the followup period

At least 1 night homelessb or doubled up in past 6 months or in shelter in

362

17.2

(37.2)

290

37.6

(48.2)

– 20.4***

(3.6) – 0.36

past 12 months (%) [confirmatory]c

At least 1 night homelessb or doubled up in past 6 months (%)

362

16.1

(35.9)

290

31.7

(46.8)

– 15.5***

(3.6) – 0.28

At least 1 night homelessb in past 6 months (%)

362

10.0

(29.2)

290

13.8

(33.8)

– 3.8

(2.8) – 0.09

At least 1 night doubled up in past 6 months (%)

362

11.2

(31.0)

290

26.5

(44.4)

– 15.3***

(3.3) – 0.29

Any stay in emergency shelter in past 6 months (%) [Program Usage Data]

362

3.8

(18.6)

290

4.6

(20.0)

– 0.8

(1.9) – 0.02

Any stay in emergency shelter in months 21 to 32 after RA (%) [Program

362

5.2

(22.3)

290

18.0

(37.2)

– 12.8***

(2.9) – 0.30

Usage Data]

Number of days homelessb or doubled up in past 6 months

361

19.9

(50.5)

289

44.3

(71.7)

– 24.4***

(5.4) – 0.28

Number of days homelessb in past 6 months

362

10.0

(35.8)

290

13.6

(40.3)

– 3.6

(3.5) – 0.06

Number of days doubled up in past 6 months

361

12.8

(41.0)

289

32.9

(64.6)

– 20.1***

(4.5) – 0.27

Housing independence

Living in own house or apartment at followup (%)

362

82.6

(37.2)

290

71.2

(46.5)

11.4***

(962)

0.09

Living in own house or apartment with no housing assistance (%)

361

15.4

(36.3)

288

38.2

(47.9)

– 22.7*** (1,038) – 0.01

Living in own house or apartment with housing assistance (%)

361

67.0

(46.8)

288

33.6

(47.3)

33.3***

(4.3) – 0.01

Number of places lived

Number of places lived in past 6 monthsd

362

1.3

(0.8)

290

1.5

(1.0)

– 0.1**

(0.1) – 0.12

Housing quality

Persons per room

346

1.2

(0.5)

277

1.5

(1.2)

– 0.3***

(0.1) – 0.25

Housing quality is poor or fair (%)

345

26.9

(44.0)

278

28.2

(45.2)

– 1.3

(3.8) – 0.02

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-CBRR comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

intervention caused reductions in six of the nine homelessness

data available for all study families), assignment to the SUB

and doubled-up measures examined. Relative to assignment

group reduced the proportion of families who had a stay in

to the CBRR intervention, assignment to the SUB intervention

emergency shelter from 18 to 5 percent relative to assignment

reduced the confirmatory outcome of being homeless or

to the CBRR group. No difference is detected in the proportions

doubled up in the past 6 months or in emergency shelter in the

of SUB and CBRR groups using emergency shelter during the

past 12 months from 38 to 17 percent. This effect represents a

6 months before the followup survey (different months relative

20-percentage-point reduction, more than one-half of the prev-

to random assignment for each family but centered around

alence in the CBRR group. This reduction appears to be largely

months 32 to 37). This latter finding in combination with the

driven by a decrease in doubling up—caused by assignment

survey estimates on homelessness suggests that the effect of

to the SUB group—rather than by a decrease in homelessness.

assignment to the SUB intervention (relative to assignment to

Of families assigned to the SUB group in this comparison, 11

the CBRR intervention) on emergency shelter use faded near

percent spent at least 1 night doubled up in the 6 months be-

the end of the 3-year followup period because of a reduction in

fore the survey compared with 27 percent of families assigned

shelter use over time by CBRR families.

to the CBRR group, a reduction by more than one-half. The

The second panel of Exhibit 6-3 shows that, relative to assign-

impact estimates for survey measures of homelessness (stays in

ment to the CBRR intervention, assignment to the SUB interven tion

emergency shelter and places not meant for human habitation)

increases the proportion of families living in their own house

are smaller than those for the doubling-up outcomes and are

or apartment (with or without assistance) at the 37-month

not statistically significant.

followup point from 71 to 83 percent. At 3 years after random

The impact estimates for emergency shelter outcomes based

assignment, SUB families are much less likely (23 percentage

on Program Usage Data tell a mixed story. During months

points) than CBRR families to be living in their own place with

21 to 32 after random assignment (the last 12 months with

no housing assistance and much more likely (33 percentage

points) to be living in their own place with housing assistance.

3-Year Impacts of Housing and Services Interventions for Homeless Families

76

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

The third panel of the exhibit shows that SUB families had

The study team finds an effect on the proportion of family

greater residential stability than did CBRR families in the

heads with a formal child separation that began after random

months before the 37-month survey, reducing the number of

assignment. Assignment to the SUB group reduced the proportion

places families lived in the past 6 months by 0.1 places. The

by 6 percentage points from a mean of 11 percent for those

bottom panel of the exhibit shows that SUB families were living

assigned to the CBRR group. This effect contrasts with the effect

in less crowded conditions than were CBRR families, with an

of close to zero found on the survey measure of the proportion

average of 1.2 persons per room compared with 1.5 persons

of family heads with a formal child separation that began after ran-

per room for CBRR families. As at 20 months after random

dom assignment in both the full sample (Exhibit 6-4) and the

assignment, no difference in housing quality was observed.

subsample of the five sites where child welfare administrative

data were collected (not shown). The reasons for the discrep-

Impacts on Family Preservation in the SUB-Versus-

ancy between the survey and administrative impact estimates

CBRR Comparison

are not clear. No effect was detected on the total days separated

Any differential effects of the SUB intervention compared with

from at least one child.

the CBRR intervention of family preservation would be expect-

ed to be indirect, via the substantial differential effects on hous-

Impacts on Adult Well-Being in the SUB-Versus-

ing stability. Exhibit 6-4 shows that, compared with assignment

CBRR Comparison

to the CBRR group, assignment to the SUB group had no effect

As with family preservation, any differential effects of the SUB

in the survey data on family separations or reunifications in the

intervention compared with the CBRR intervention on adult

6 months before the 37-month followup survey.

well-being would be expected to be indirect, via the substantial

differential effects on housing stability. The Short-Term Impacts

Exhibit 6-5 shows impacts on outcomes from the child welfare

report (Gubits et al., 2015) observed that, compared with

agency administrative data collected in 5 of the 12 study sites.

assignment to the CBRR group, assignment to the SUB group

Exhibit 6-4. SUB Versus CBRR: Impacts on Family Preservation at 37 Months

SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

353

14.2

(34.9)

288

16.4

(35.7)

– 2.2

(3.1) – 0.05

Family has at least one foster care placement in past 6 monthsb (%)

356

3.6

(18.8)

288

4.2

(16.5)

– 0.6

(1.7) – 0.03

Spouse/partner separated in past 6 months, of those with spouse/partner

89

43.3

(50.1)

77

49.9

(50.1)

– 6.6

(8.3) – 0.11

present at RA (%)

Reunification of family members reported as separated at baseline

Spouse/partner reunified, of those with spouse/partner absent at RA (%)

76

39.4

(49.7)

47

29.6

(47.1)

9.8

(11.0)

0.17

Living in own house or apartment with no housing assistance (%)

40

30.4

(46.4)

22

26.6

(45.6)

3.8

(19.6)

0.08

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 6-5. SUB Versus CBRR: Impacts on Child Welfare Outcomes

SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Had a formal child separation that began after RA (%)

171

5.6

(23.5)

175

11.1

(31.2)

– 5.6*

(3.0) – 16.93

Total days during followup separated from at least one childb

171

42.4 (188.1)

175

47.6 (177.7)

– 5.2

(13.9)

– 0.02

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days. Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions. Sample limited to five sites where child welfare records were collected (Alameda County, Baltimore, Kansas City, Minneapolis, and Phoenix).

Source: State child welfare agency records

3-Year Impacts of Housing and Services Interventions for Homeless Families

77

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

reduced the proportion of families who reported post-traumatic

families assigned to the SUB group). Only one other effect (for

stress disorder symptoms in the month before the 20-month

two out of nine) appears in the child outcomes assessed across

survey and also reduced the incidence of intimate partner

age groups, also in the school domain (first panel of Exhibit 6-7)—

violence by more than one-half.

children in the SUB group had slightly better grades. Because

this impact did not appear in the SUB-versus-UC comparison,

Exhibit 6-6 shows the adult well-being results for the SUB-

it is perhaps best thought of as a random variation. No effects

versus-CBRR comparison for the longer followup period. The

were found on child physical health or on behavioral strengths

37-month analysis shows no evidence that assignment to the

and challenges.

SUB group had differential effects on any measure of physical

or mental health, intimate partner violence, or substance depen-

Only 3 of 20 age-specific outcomes shown in Exhibit 6-8 show

dence or abuse compared with assignment to the CBRR group.

effects, one for younger children ages 2 to 5 and two for the

older 8- to 17-year-old age group. Younger children in the SUB

Impacts on Child Well-Being in the SUB-Versus-

group had more absences from childcare or preschool than did

CBRR Comparison

young children in the CBRR group. Older children in the SUB

Exhibit 6-7 shows the effects of assignment to the SUB group

group were less than one-half as likely to use substances (3 ver-

relative to assignment to the CBRR group on child well-being

sus 8 percent) but reported less goal-oriented thinking (effect

outcomes measured across all ages. Given the impact of

size of 0.26) than did children in the CBRR group. Again, these

assignment to the SUB intervention relative to assignment to

impacts were not evident in the SUB-versus-UC comparison.

the CBRR intervention on residential stability at both 20 and

Of the 5 statistically significant effects, 3 favored the SUB group

37 months, it is not surprising that the SUB intervention has

and 2 favored the CBRR group. Only the impact on school mobility

an effect on school mobility (approximately one fewer school

was apparent in the SUB-versus-UC comparison.109 The others

moves since random assignment for every four children in

are, therefore, probably best interpreted as random variations.

Exhibit 6-6. SUB Versus CBRR: Impacts on Adult Well-Being at 37 Months

SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

362

30.9

(46.3)

289

30.8

(45.5)

0.1

(3.5)

0.00

Adult mental health

Goal-oriented thinkingb

357

4.47

(1.06)

287

4.49

(0.99)

– 0.02

(0.08) – 0.02

Psychological distressc

360

6.72

(5.68)

289

6.64

(5.70)

0.08

(0.45)

0.01

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

357

22.4

(41.6)

287

19.1

(38.6)

3.3

(0.1) – 0.12

Adult substance use

Alcohol dependence or drug abused (%)

361

11.4

(32.1)

289

9.6

(30.1)

1.8

(2.5)

0.05

Alcohol dependenced (%)

362

8.7

(28.4)

289

7.6

(27.6)

1.2

(2.2)

0.04

Drug abused (%)

361

3.6

(18.7)

289

3.1

(15.4)

0.5

(1.6)

0.02

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

361

7.1

(25.9)

290

8.9

(26.5)

– 1.8

(2.3) – 0.05

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

109 This reduction in school mobility is also corroborated by reductions in the average number of places lived by SUB families relative to CBRR families in the 6-month periods before the followup surveys.

3-Year Impacts of Housing and Services Interventions for Homeless Families

78

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-7. SUB Versus CBRR: Impacts on Child Well-Being Across Age Groups at 37 Months SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

417

1.9

(0.9)

325

2.1

(0.9)

– 0.3***

(3.5)

0.00

Grade completion (not held back) (%)

339

91.17 (28.44)

269

92.63 (28.01)

– 1.46

(0.08) – 0.02

School gradesc

311

3.1

(0.9)

252

3.0

(0.9)

0.1*

(0.45)

0.01

Child physical health

Poor or fair health in past 30 days (%)

485

7.4

(24.9)

396

6.4

(23.9)

1.0

(2.3)

0.03

Well-child checkup in past year (%)

482

89.1

(30.3)

395

91.8

(27.7)

– 2.8

(2.4) – 0.07

Child has regular source of health care (%)

484

90.9

(25.9)

396

94.3

(23.9)

– 3.4

(2.5) – 0.09

Sleep problemsd

483

1.98

(1.03)

396

2.05

(1.01)

– 0.07

(0.08) – 0.05

Child behavioral strengths and challenges

Behavior problemse

458

0.45

(1.28)

378

0.39

(1.20)

0.06

(0.11)

0.03

Prosocial behaviorf

459 – 0.21

(1.16)

379 – 0.19

(1.16)

– 0.02

(0.10) – 0.02

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

Impacts on Self-Sufficiency in the SUB-Versus-

CBRR group on total earnings, the proportion of family heads

CBRR Comparison

employed, or the number of quarters employed during the year.

The Short-Term Impacts report (Gubits et al., 2015) found that,

Turning back to Exhibit 6-9, assignment to the SUB group

20 months after random assignment, assignment to the SUB

increased the proportion of families who said they participated

group reduced work for pay, earnings, and family income rel-

in 2 or more weeks of school since random assignment (14 per-

ative to assignment to the CBRR group. Assignment to the SUB

cent of SUB families, 8 percent of CBRR families). Because this

group also decreased economic stress. Exhibit 6-9 shows the

magnitude is larger than would be expected, given the point

effects on self-sufficiency outcomes in the SUB-versus-CBRR

estimates in the SUB-versus-UC and CBRR-versus-UC comparisons,

comparison at the 3-year followup point. The study team finds

it is perhaps best thought of as a random variation. Overall,

that assignment to the SUB intervention caused differential

the study team finds some evidence that assignment to the SUB

effects on 2 of the 20 outcomes examined. Families assigned to

group decreased work effort relative to assignment to the CBRR

the SUB group worked, on average, nearly 2 months less than

group during the entire 3-year followup period, but the team

families assigned to the CBRR group during the 3-year followup

did not find that this effect was still present at the 3-year fol-

period. Given that this differential is of comparable magnitude

lowup point. The team also did not find evidence of effects on

to the effect detected at 20 months and that the estimates for

earnings, family income, nonhousing public assistance receipt,

impacts on work since the 20-month followup point and at 37

food security, or economic stress at the 3-year followup point.

months are small and statistically insignificant, it appears that

the reduction in work effort apparent at 20 months faded after

Summary of SUB-Versus-CBRR Comparison Across

that point. At the 37-month followup point, the study team

Domains

finds no evidence that assignment to the SUB group relative

For the SUB-versus-CBRR comparison, the study engineered a

to assignment to the CBRR group affected annualized current

notable contrast in the mix of program use during the 37-month

earnings or total family cash income or sources of income.

followup period. At 3 years after random assignment, 82 percent

Exhibit 6-10 shows impact estimates from administrative data

of SUB families had ever used the permanent housing subsidies

on outcomes for quarters 11 to 14 after the quarter of random

they were offered compared with only 10 percent of CBRR fam-

assignment. The analysis of administrative data finds no effect

ilies in this comparison who used these subsidies. Altogether,

of assignment to the SUB group relative to assignment to the

61 percent of CBRR families had ever used rapid re-housing

3-Year Impacts of Housing and Services Interventions for Homeless Families

79

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-8. SUB Versus CBRR: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 months SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

156

37.1

(48.7)

135

28.7

(47.0)

8.4

(5.9)

0.14

Child care or preschool absences in past monthd

64

0.85

(0.89)

38

0.43

(0.86)

0.42*

(0.24)

0.38

Positive child care or preschool experiencese

67

0.79

(0.40)

40

0.86

(0.41)

– 0.07

(0.11) – 0.14

Positive child care or preschool attitudesf

67

4.68

(0.73)

40

4.52

(0.93)

0.16

(0.23)

0.19

Child care or preschool conduct problemsg (%)

70

11.6

(32.0)

43

6.2

(32.4)

5.3

(9.2)

0.19

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

136

67.7

(45.7)

121

75.0

(43.8)

– 7.3

(6.0) – 0.13

Ages 3 years, 6 months to 7 years

Verbal abilityi

175 – 0.11

(1.02)

134 – 0.26

(1.03)

0.14

(0.11)

0.11

Math abilityj

173 – 0.22

(1.00)

134 – 0.35

(0.86)

0.13

(0.12)

0.11

Executive functioningk (self-regulation)

172

14.97 (16.00)

129

15.20 (16.11)

– 0.23

(1.43) – 0.01

Ages 5 to 17 yearsl

School enrollmentc (%)

330

98.0

(13.4)

263

96.6

(19.2)

1.4

(1.6)

0.07

School absences in past monthd,m

156

0.90

(0.90)

107

0.81

(0.99)

0.09

(0.15)

0.07

Positive school experiencese,m

157

0.58

(0.59)

107

0.50

(0.57)

0.08

(0.08)

0.09

Positive school attitudesf,m

157

4.29

(1.05)

107

4.10

(1.13)

0.19

(0.14)

0.13

School conduct problemsg,m (%)

158

22.3

(40.8)

106

26.6

(44.8)

– 4.3

(6.0) – 0.07

Ages 8 to 17 years

Anxietyn

207

35.56

(7.63)

167

34.80

(7.63)

0.76

(0.84)

0.07

Fearso

208

62.45 (14.32)

169

64.36 (14.40)

– 1.91

(1.63) – 0.10

Substance usep (%)

206

2.80 (18.16)

165

8.30 (26.05)

– 5.50**

(2.73) – 0.15

Goal-oriented thinkingq

203

21.54

(5.24)

162

23.23

(4.74)

– 1.70***

(0.61) – 0.26

School effort in past monthr

208

2.84

(0.76)

165

2.78

(0.83)

0.06

(0.10)

0.06

Arrests or police involvement in past 6 monthss (%)

117

7.93 (25.35)

94

8.68 (28.05)

– 0.75

(4.11) – 0.02

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse.

See Chapter 2 and Appendix B for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

3-Year Impacts of Housing and Services Interventions for Homeless Families

80

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-9. SUB Versus CBRR: Impacts on Self-Sufficiency at 37 Months

SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

362

36.3

(48.0) 290

36.9

(48.7)

– 0.6

(3.9) – 0.01

Any work for pay since 20-month surveyb (%)

336

59.4

(49.3) 271

61.1

(48.6)

– 1.7

(3.9) – 0.04

Months worked for pay since 20-month surveyb,c

334

7.0

(7.9) 271

7.4

(8.2)

– 0.4

(0.6) – 0.05

Any work for pay since RA (%)

362

69.3

(46.4) 290

74.1

(42.0)

– 4.9

(3.5) – 0.09

Months worked for pay since RAc

357

12.4

(13.8) 288

14.1

(13.6)

– 1.7*

(1.0) – 0.11

Hours of work per week at current main jobd

362

11.2

(16.4) 289

11.9

(17.2)

– 0.8

(1.4) – 0.04

Income sources and amounts

Annualized current earnings ($)

357

6,114 (10,296) 285

6,432 (11,288) – 318

(828) – 0.03

Total family income ($)

340 11,097 (9,734) 277 11,998 (9,448) – 901

(757) – 0.07

Anyone in family had earnings in past month (%)

362

47.5

(49.9) 290

47.1

(50.1)

0.4

(4.0)

0.01

Anyone in family received TANF in past month (%)

362

27.9

(44.8) 290

32.0

(46.2)

– 4.1

(3.7) – 0.08

Anyone in family received SSDI in past month (%)

360

7.4

(27.3) 290

10.7

(27.6)

– 3.3

(2.4) – 0.10

Anyone in family received SSI in past month (%)

362

13.3

(35.1) 290

16.1

(35.9)

– 2.9

(2.7) – 0.07

Anyone in family received SNAP/Food Stamps in past month (%)

362

79.4

(40.2) 290

82.7

(37.8)

– 3.3

(3.4) – 0.08

Anyone in family received WIC in past month (%)

362

23.9

(43.0) 290

26.9

(45.3)

– 3.0

(3.4) – 0.06

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

361

38.3

(48.7) 290

37.8

(48.6)

0.5

(4.1)

0.01

Number of weeks in school/training programs since RA

358

7.4

(14.3) 283

7.6

(16.9)

– 0.2

(1.2) – 0.01

Participated in 2 weeks or more of school since RA (%)

361

13.6

(33.4) 290

8.2

(27.1)

5.4**

(2.6)

0.14

Participated in 2 weeks or more of basic education since RA (%)

361

3.0

(17.2) 290

4.5

(21.5)

– 1.5

(1.6) – 0.09

Participated in 2 weeks or more of vocational education since RA (%)

361

10.3

(31.1) 290

12.0

(33.8)

– 1.7

(2.5) – 0.04

Food security

Household is food insecure (%)

362

38.5

(49.0) 290

41.1

(49.3)

– 2.6

(4.0) – 0.04

Food insecurity scalee

360

1.59

(2.03) 290

1.75

(2.05) – 0.16

(0.16) – 0.06

Economic stressors

Economic stress scalef

360

– 0.24

(0.45) 287

– 0.20

(0.48) – 0.04

(0.04) – 0.07

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 6-10. SUB Versus CBRR: Earnings and Employment

SUB

CBRR

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Earnings in quarters 11 to 14 after RA (2015Q3$)

426

5,578 (8,907)

373

6,322 (10,414) – 745

(711) – 0.08

Any employment in quarters 11 to 14 after RA (%)

426

51.5

(50.0)

373

56.8

(49.6)

– 5.3

(3.6) – 0.11

Number of quarters employed in quarters 11 to 14 after RA

426

1.6

(1.7)

373

1.6

(1.7)

– 0.1

(0.1) – 0.05

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Note: See Chapter 2 and Appendix B for outcome definitions.

Source: Quarterly wage records from the National Directory of New Hires

3-Year Impacts of Housing and Services Interventions for Homeless Families

81

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

compared with 15 percent of SUB families. In the month of the

In the self-sufficiency domain, the study team finds some evi-

followup survey response, 72 percent of SUB families were us-

dence that assignment to the SUB group decreased work effort

ing any form of permanent subsidy compared with 33 percent

relative to assignment to the CBRR group during the entire

of CBRR families and use of temporary subsidies had largely

3-year followup period, but the team did not find evidence that

ended (less than 3 percent in both groups).

this effect was still present at the 3-year followup point. The

team also did not find evidence of effects on earnings, family

These differences in program participation led to notable

income, nonhousing public assistance receipt, food security, or

improvements in housing stability for families assigned to the

economic stress at the 3-year followup point.

SUB group compared with assignment to the CBRR group.

Only 5 percent of SUB families compared with 18 percent of

Overall, do the families assigned to the SUB group appear to

CBRR families spent at least 1 night in an emergency shelter

be doing better at 37 months after random assignment than

during months 21 to 32 after random assignment. After month

the families assigned to the CBRR group? In some important

32, it appears that the main preventive effect of assignment

respects, the answer is “yes.” The SUB families, on average, are

to the SUB intervention relative to assignment to the CBRR

experiencing less doubling up, are more likely to live in their

intervention is in the reduction of the proportion of families

own place, and are living in less crowded conditions than are

experiencing doubling up rather than in the reduction of stays

CBRR families. Should SUB families, over time, give up the

in emergency shelter or places not meant for human habitation.

permanent housing assistance to which they were given prefer-

Assignment to the SUB group relative to assignment to the

ential access by the study—or if increasing numbers of CBRR

CBRR group, reduced the proportion of families who had at

families find their way to permanent housing subsidies—these

least 1 night doubled up in the 6 months before the 37-month

differences may shrink in the future.

survey from 27 to 11 percent. Compared with assignment to

the CBRR intervention, the SUB intervention also increased the

6.3. The Permanent Housing

proportion of families living in their own house or apartment

at the time of the followup survey from 71 to 83 percent,

Subsidy (SUB) Versus Project-

reduced crowding, and reduced the number of places where

Based Transitional Housing (PBTH)

families lived in the past 6 months. No evidence was found that

Comparison

assignment to the SUB group affected either the proportion of

families who reported homelessness (using the survey measure

The SUB-versus-PBTH comparison contrasts the permanent

of homelessness) or the proportion with an emergency shelter

housing subsidy of the SUB intervention with the temporary

stay (measured by Program Usage Data) during the 6-month

housing (up to 24 months, with a median stay of 13 months

period before the survey.

during this followup period) in agency-controlled units paired

with intensive supportive services of the PBTH intervention.

The SUB-versus-CBRR comparison shows that, at 3 years

PBTH programs offer comprehensive case management and

after random assignment, assignment to the SUB intervention

provide many supportive services directly. These services are

instead of to the CBRR intervention had no effect on any of

entirely absent from the SUB intervention.

the outcomes examined in the family preservation and adult

well-being domains. The study team finds a few effects in the

6.3.1. Program Use by Families in the SUB-

child well-being domain but finds no pattern favoring either

Versus-PBTH Comparison

the SUB or CBRR intervention. The lower number of schools

attended since random assignment due to assignment to the

Exhibit 6-11 shows the use of eight types of homeless and

SUB group is perhaps the most credible of these effects, given

housing programs by the 215 SUB families and 201 PBTH

its consistency with the reduction in the number of schools

families analyzed in the SUB-versus-PBTH comparison during

children attended found in the SUB-versus-UC comparison and

the 37-month followup period.110 The first column shows some

the reduction in the number of places families lived found in

modest differences in the general pattern of usage for these SUB

this SUB-versus-CBRR comparison.

families compared with all SUB families (shown in Exhibit 3-2 in

Chapter 3).111 The proportions of these PBTH families shown

in the second column are similar to the proportions of all

PBTH families (shown in Exhibit 5-1 in Chapter 5). The first

110 The SUB-versus-PBTH comparison sample consists of 256 families assigned to the SUB group and 240 families assigned to the PBTH group. Of those 496 families, 215

SUB families and 201 CBRR families (84 percent) responded to the 37-month followup survey.

111 When compared with all SUB families, this subset of SUB families is somewhat less likely to ever use rapid re-housing (6 percent rather than 11 percent).

3-Year Impacts of Housing and Services Interventions for Homeless Families

82

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

two columns show that 82 percent of families assigned to the

6.3.2. Impacts of the SUB Intervention

SUB group used the permanent housing subsidies they were

Compared With the PBTH Intervention

offered and 50 percent of families assigned to the PBTH group

The SUB and PBTH interventions are based on divergent views

used transitional housing. The number of months of program

about the package of housing assistance and services that home less

use (in columns three through six) and the proportions using

families need. The premise of permanent housing assistance as

the program in the month of the followup survey response (in

an intervention is that family homelessness is centrally a hous-

columns seven and eight) are largely similar to those in the

ing affordability problem. From the perspective of permanent

previous exhibits. As a result, a large difference exists in the

housing subsidy proponents, by addressing this problem with

proportion of families participating in some program during

permanent housing assistance, assignment to the SUB interven-

the survey month, with 74 percent of SUB families doing

tion should reduce homelessness and improve housing stability

so (most receiving a permanent housing subsidy offered to

relative to assignment to the PBTH intervention. By this means,

the SUB group) and less than one-half of PBTH families (39

it may improve family preservation, adult well-being, and child

percent) participating in any program.

well-being. By reducing the proportion of income that must be

As expected, because of priority access to permanent housing

devoted to housing costs, it may improve self-sufficiency.

subsidies, the proportion that ever used a permanent subsidy

By contrast, the premise for transitional housing as an

was higher for families assigned to the SUB group (87 percent)

intervention is that most families who become homeless have

than for families assigned to the PBTH group (31 percent). At

additional barriers that make it difficult for them to secure and

the 37-month survey, 70 percent of SUB families were using

maintain housing. The services offered in transitional housing

some form of permanent housing subsidy compared with 27

address psychosocial problems and barriers to housing stability

percent of PBTH families. This differential is 16 percentage

and attempt to put families on track for better employment

points narrower than it was at the time of the 20-month survey.

and earnings. By this means, proponents of project-based

Use of permanent housing subsidies had decreased for the SUB

transitional housing expect that assignment to the PBTH inter-

group (by 5 percentage points) and had increased for the PBTH

vention will improve long-term housing stability, employment,

group (by 11 percentage points).

Exhibit 6-11. SUB Versus PBTH: Program Use Since RA

Percent Ever Used

Number of Months Used

Percent Used in

From RA to 37-Month

From RA to 37-Month Followup Survey,

Month of Followup

Type of Homeless or

Followup Surveya

if Ever Used Type of Assistance

Survey Response

Housing Assistance

SUB

PBTH

SUB

PBTH

SUB

PBTH

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

82.3

7.2

31.4

33.5

13.1

11.5

65.8

6.5

SUB groupb

Rapid re-housingc

5.7

12.0

3.7

3.5

9.2

10.5

0.4

0.9

Transitional housingd

9.4

49.5

10.9

8.0

14.8

13.0

1.0

7.3

Permanent supportive housing

2.1

11.6

7.2

8.0

15.3

12.5

1.2

8.6

Public housing

1.6

8.2

22.5

20.5

17.9

15.5

1.5

6.9

Project-based vouchers/Section 8 projects

1.7

7.2

15.6

14.5

16.4

13.5

1.2

4.6

Any permanent housing subsidye

86.6

30.9

30.8

33.5

17.3

15.5

69.7

26.7

Emergency shelterf

87.9

83.0

2.6

1.9

3.5

2.3

2.5

5.2

No use of homeless or housing programsg

7.1

20.5

—

—

—

—

26.4

60.7

N

215

201

—

—

—

—

215

201

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

83

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

earnings, education, and adult well-being relative to assignment

Impacts on Housing Stability in the SUB-Versus-

to the SUB intervention and may improve family preservation

PBTH Comparison

and child well-being.

Exhibit 6-12 shows the effect on housing stability of being

assigned to the SUB group relative to being assigned to the

The Short-Term Impacts report (Gubits et al., 2015) found that,

PBTH group. The effects of the SUB intervention relative to

20 months after random assignment, assignment to the SUB

the PBTH intervention 3 years after random assignment are

group reduced homelessness and doubling up compared with

favorable, large, and statistically significant on all homelessness

assignment to the PBTH group. It also led to cases in which

outcomes.112 Assignment to the SUB group reduced the confir-

more families lived in their own place and made fewer residen-

matory outcome of being homeless or doubled up in the past

tial moves. In domains other than housing stability, assignment

6 months or in emergency shelter in the past 12 months from

to the SUB intervention had several beneficial impacts—reduced

41 to 17 percent relative to assignment to the PBTH group. This

separations from children, increased reunifications with children,

impact represents a reduction of 24 percentage points, more

lowered psychological distress, lowered number of schools

than one-half of the prevalence for the PBTH families. The next

attended, and decreased food insecurity and economic stress—

three rows show large reductions in the proportions of families

compared with assignment to the PBTH intervention but

experiencing subsequent stays in shelter or places not meant

reduced family income and work for pay. The remainder of this

for human habitation and doubling up in the past 6 months.

section presents the impact evidence of how these interventions

The preventive effect of assignment to the SUB intervention

compare at 3 years after random assignment.

on doubling up was particularly strong; 30 percent of PBTH

Exhibit 6-12. SUB Versus PBTH: Impacts on Housing Stability at 37 Months

SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the followup period

At least 1 night homelessb or doubled up in past 6 months or in shelter in

214

16.8

(38.7) 200

41.1

(49.5) – 24.4***

(4.6) – 0.43

past 12 months (%) [confirmatory]c

At least 1 night homelessb or doubled up in past 6 months (%)

214

15.7

(37.5) 200

37.8

(48.9) – 22.1***

(4.6) – 0.40

At least 1 night homelessb in past 6 months (%)

214

9.0

(29.8) 201

20.1

(40.4) – 11.1***

(3.6) – 0.25

At least 1 night doubled up in past 6 months (%)

215

11.2

(32.7) 200

29.9

(46.4) – 18.7***

(4.4) – 0.36

Any stay in emergency shelter in past 6 months (%) [Program Usage Data]

215

3.9

(20.1) 201

9.1

(28.6)

– 5.2*

(2.7) – 0.16

Any stay in emergency shelter in months 21 to 32 after RA (%) [Program

215

4.4

(21.1) 201

9.9

(30.0)

– 5.5**

(2.7) – 0.13

Usage Data]

Number of days homelessb or doubled up in past 6 months

214

16.3

(51.5) 200

48.6

(72.8) – 32.3***

(6.7) – 0.37

Number of days homelessb in past 6 months

214

7.0

(34.0) 201

16.7

(43.9)

– 9.7**

(4.0) – 0.17

Number of days doubled up in past 6 months

215

8.5

(38.9) 200

35.2

(65.0) – 26.6***

(6.1) – 0.36

Housing independence

Living in own house or apartment at followup (%)

215

84.9

(36.6) 201

66.3

(47.3)

18.5***

(4.6)

0.34

Living in own house or apartment with no housing assistance (%)

214

16.3

(36.6) 200

41.0

(49.3) – 24.7***

(4.6) – 0.43

Living in own house or apartment with housing assistance (%)

214

68.7

(46.5) 200

25.3

(43.7)

43.4***

(4.5)

0.84

Number of places lived

Number of places lived in past 6 monthsd

215

1.3

(0.9) 199

1.7

(1.1)

– 0.3***

(0.1) – 0.25

Housing quality

Persons per room

208

1.3

(0.7) 188

1.7

(1.2)

– 0.4***

(0.1) – 0.31

Housing quality is poor or fair (%)

207

28.1

(44.8) 186

32.2

(47.1)

– 4.1

(5.1) – 0.08

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-PBTH comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

112 The homeless outcomes in this study diverge from the homeless definition final rule in that they do not include stays in transitional housing in their definitions of being homeless. Additional impacts on the use of transitional housing during the followup period are provided in Appendix E.

3-Year Impacts of Housing and Services Interventions for Homeless Families

84

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

families experienced doubling up in the 6 months before the

assignment to the PBTH group, assignment to the SUB group

survey compared with only 11 percent of SUB families. As-

had no effect on the housing quality that families reported at

signment to the SUB group, relative to assignment to the PBTH

the time of the 37-month survey.

group, also reduced the proportions of families who had a stay

in emergency shelter during months 21 to 32 after random

Impacts on Family Preservation in the SUB-Versus-

assignment and during the 6 months before the survey by more

PBTH Comparison

than one-half (each by about 5 percentage points).

The Short-Term Impacts report (Gubits et al., 2015) found that,

at the 20-month followup point, families assigned to the SUB

The second panel of the exhibit shows that assignment to the

group had fewer separations from children and more reunifications

SUB group increases the proportion of families living in their

with separated children and separated spouses and partners

own house or apartment (with or without assistance) at the

than did families assigned to the PBTH group. Exhibit 6-13

time of the followup survey from 66 to 85 percent relative

shows the effects on family preservation in the SUB-versus-PBTH

to assignment to the PBTH group. SUB families are much

comparison at the 3-year followup point. As at 20 months,

less likely (25 percentage points) than are PBTH families to

assignment to the SUB intervention compared with assignment

be living in their own place with no housing assistance and

to the PBTH intervention, reduced the proportion of families

much more likely to be living in their own place with housing

who had at least one child separation at 37 months after

assistance (41 percentage points).

random assignment. No evidence of effects was found for any

The third panel of the exhibit shows that SUB families had

of the other family preservation measures from survey data at

greater residential stability than did PBTH families in the months

37 months. Neither was evidence of effects found for family

before the survey. Assignment to the SUB intervention reduced

preservation measures from the five-site administrative data on

the number of places families lived in the past 6 months by

formal child separations (Exhibit 6-14).

0.3 places. Some of this effect is likely because some PBTH

Impacts on Adult Well-Being in the SUB-Versus-

families moved out of transitional housing at the end of their

PBTH Comparison

program participation.

Exhibit 6-15 shows effects at 37 months on adult well-being

The bottom panel of the exhibit shows that SUB families were

outcomes for the SUB-versus-PBTH comparison. The beneficial

living in less crowded conditions than were PBTH families,

effects of assignment to the SUB intervention on mental

with an average of 1.3 persons per room compared with

health found at 20 months were not apparent at 37 months.

1.7 persons per room for PBTH families. Compared with

Assignment to the SUB group also led to equivalent results as

for assignment to the PBTH group on measures of physical,

intimate partner violence, and substance dependence or abuse.

Exhibit 6-13. SUB Versus PBTH: Impacts on Family Preservation at 37 Months

SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

208

15.2

(35.2)

194

23.3

(41.6)

– 8.1**

(4.0) – 0.19

Family has at least one foster care placement in past 6 monthsb (%)

210

3.8

(18.0)

195

4.6

(22.1)

– 0.8

(2.0) – 0.04

Spouse/partner separated in past 6 months, of those with spouse/partner

65

39.1

(49.7)

56

38.8

(48.9)

0.3

(9.9)

0.00

present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at least one

47

54.0

(50.5)

37

33.1

(49.2)

21.0

(13.8)

0.37

child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent at RA (%)

23

26.0

(47.0)

21

22.8

(43.6)

3.2

(15.6)

0.07

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

3-Year Impacts of Housing and Services Interventions for Homeless Families

85

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-14. SUB Versus PBTH: Impacts on Child Welfare Outcomes

SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Had a formal child separation that began after RA (%)

138

11.8

(32.1)

144

15.8

(36.8)

– 4.0

(4.2) – 12.06

Total days during followup separated from at least one childb

138

70.4 (218.9)

144

97.8 (253.5)

– 27.5

(24.0)

– 0.11

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days.

Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions. Sample limited to five sites where child welfare records were collected (Alameda County, Baltimore, Kansas City, Minneapolis, and Phoenix).

Source: State child welfare agency records

Exhibit 6-15. SUB Versus PBTH: Impacts on Adult Well-Being 37 Months

SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

214

30.4

(45.9)

199

32.6

(46.8)

– 2.2

(4.9) – 0.04

Adult mental health

Goal-oriented thinkingb

210

4.48

(0.99)

195

4.39

(1.08)

0.09

(0.11)

0.08

Psychological distressc

214

6.67

(5.78)

200

7.41

(5.50)

– 0.74

(0.57) – 0.11

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

211

22.9

(41.7)

199

23.1

(40.9)

– 0.2

(4.3)

0.00

Adult substance use

Alcohol dependence or drug abused (%)

215

16.3

(35.2)

201

16.9

(35.2)

– 0.6

(3.8) – 0.02

Alcohol dependenced (%)

215

12.6

(31.0)

201

12.7

(30.7)

– 0.2

(3.4) – 0.01

Drug abused (%)

215

5.1

(20.1)

201

6.8

(24.7)

– 1.6

(2.5) – 0.06

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

213

9.0

(29.2)

200

10.2

(29.4)

– 1.2

(3.1) – 0.03

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Impacts on Child Well-Being in the SUB-Versus-

families compared with those in PBTH families. Young children

PBTH Comparison

ages 3.5 to 7 in the SUB group show higher verbal ability (first

Of 9 impact estimates on outcomes, 4 measured across ages

panel of Exhibit 6-17) and older children ages 8 to 17 report

(Exhibit 6-16), and 2 of 20 impact estimates on age-specific

lower substance use than children in the PBTH group. Some

outcomes (Exhibit 6-17) reached statistical significance, all

of these effects (higher grades, higher verbal ability, and lower

favoring children in families assigned to the SUB group rather

substance abuse) are not observed in the SUB-versus-UC com-

than to the PBTH group. The cross-age impacts are clustered

parison and have magnitudes larger than would be expected

in the school domain: children assigned to the SUB group

from the SUB-versus-UC and PBTH-versus-UC comparisons.

had fewer school moves, are 7 percentage points more likely

These three effects are thus perhaps best thought of as random

to be on track for grade completion, and have higher grades

variations. Overall, the study team finds scattered beneficial

(an average of 0.20 higher on a 4.0 scale). In addition, parents

effects on child well-being from assignment to the SUB group

reported fewer behavioral problems for children in SUB

relative to assignment to the PBTH group.

3-Year Impacts of Housing and Services Interventions for Homeless Families

86

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-16. SUB Versus PBTH: Impacts on Child Well-Being Across Age Groups at 37 Months SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

251

1.9

(0.9)

229

2.1

(1.0)

– 0.2*

(0.1) – 0.18

Grade completion (not held back) (%)

208

93.87 (22.43)

191

86.76 (33.82)

7.11**

(3.32)

0.19

School gradesc

188

3.2

(0.8)

171

3.0

(0.9)

0.2**

(0.1)

0.18

Child physical health

Poor or fair health in past 30 days (%)

299

6.2

(26.2)

263

9.1

(24.6)

– 2.9

(2.9) – 0.09

Well-child checkup in past year (%)

299

90.3

(29.6)

263

89.3

(31.9)

1.0

(3.1)

0.03

Child has regular source of health care (%)

299

91.6

(27.7)

260

89.2

(29.0)

2.4

(3.7)

0.07

Sleep problemsd

300

2.05

(1.06)

263

2.19

(1.13)

– 0.14

(0.11) – 0.10

Child behavioral strengths and challenges

Behavior problemse

279

0.40

(1.21)

250

0.64

(1.25)

– 0.25*

(0.15) – 0.15

Prosocial behaviorf

280 – 0.10

(1.03)

250 – 0.29

(1.23)

0.19

(0.13)

0.13

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

Impacts on Self-Sufficiency in the SUB-Versus-

The 20-month analysis detected higher overall participation

PBTH Comparison

in school or training for SUB families compared with PBTH

The Short-Term Impacts report (Gubits et al., 2015) found that,

families. This difference is not found at 37 months, but two

20 months after random assignment, assignment to the SUB

other effects are detected at the later time point (Exhibit 6-18):

group reduced work for pay, earnings, and family income rela-

SUB families had a higher average number of weeks in school

tive to assignment to the PBTH group. Assignment to the SUB

or training since random assignment (9 weeks) than did PBTH

intervention also reduced food insecurity and economic stress,

families (6 weeks) and a higher proportion with at least 2 weeks

however, relative to assignment to the PBTH intervention.

in a school program (16 percent versus 10 percent). These

Exhibit 6-18 shows statistically significant effects on 5 of the 20

results are surprising, because most PBTH providers incorpo-

self-sufficiency outcomes examined in the SUB-versus-PBTH

rated some kind of employment training into their programs

comparison at the 3-year followup survey. The study team did

(although this training may have been less than 2 weeks).

not find an effect on the proportion of family heads working for

The fourth and fifth panels of Exhibit 6-18 show that, as

pay at the time of the 37-month survey or on the proportion

it was at 20 months, assignment to the SUB intervention

with any work for pay since the 20-month survey. Also no

lowered food insecurity and economic stress at 37 months

detected effect exists on the proportion with any work for pay

after random assignment. Relative to assignment to the PBTH

since random assignment (and the point estimate is minus

group, assignment to the SUB group reduced the proportion

2 percentage points), suggesting that the gap in work effort

of families who were food insecure from 47 to 34 percent and

apparent at 20 months has largely disappeared.

reduced the average level of food insecurity by a standardized

Exhibit 6-19 shows impact estimates from administrative data

effect size of 0.24. Likewise, the final row of the exhibit shows

on outcomes for quarters 11 to 14 after the quarter of random

that assignment to the SUB group reduced economic stress by a

assignment. The analysis of administrative data finds no effect

standardized effect size of 0.18.

of assignment to the SUB group relative to assignment to the

PBTH group on total earnings, the proportion of family heads

employed, or the number of quarters employed during the year.

3-Year Impacts of Housing and Services Interventions for Homeless Families

87

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-17. SUB Versus PBTH: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

103

34.8

(49.4)

76

37.5

(48.9)

– 2.7

(7.1) – 0.04

Child care or preschool absences in past monthd

40

0.95

(1.04)

33

0.55

(0.79)

0.39

(0.26)

0.35

Positive child care or preschool experiencese

41

0.79

(0.43)

36

0.84

(0.55)

– 0.05

(0.11) – 0.12

Positive child care or preschool attitudesf

42

4.62

(0.86)

35

4.76

(0.95)

– 0.14

(0.16) – 0.16

Child care or preschool conduct problemsg (%)

44

4.5

(25.5)

38

11.0

(34.3)

– 6.6

(6.4) – 0.24

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

89

68.0

(46.2)

65

69.6

(46.5)

– 1.5

(8.5) – 0.03

Ages 3 years, 6 months to 7 years

Verbal abilityi

120 – 0.16

(1.08)

89 – 0.59

(1.13)

0.43*

(0.22)

0.33

Math abilityj

120 – 0.24

(1.08)

88 – 0.48

(1.08)

0.24

(0.18)

0.20

Executive functioningk (self-regulation)

114

16.72 (16.25)

79

14.14 (15.81)

2.58

(2.21)

0.13

Ages 5 to 17 yearsl

School enrollmentc (%)

198

99.2

(12.2)

189

98.4

(14.4)

0.9

(0.9)

0.05

School absences in past monthd,m

81

0.83

(0.94)

66

0.98

(0.87)

– 0.15

(0.20) – 0.12

Positive school experiencese,m

81

0.58

(0.52)

66

0.54

(0.66)

0.03

(0.10)

0.04

Positive school attitudesf,m

80

4.23

(1.06)

66

4.01

(1.18)

0.22

(0.20)

0.15

School conduct problemsg,m (%)

81

26.5

(44.1)

66

23.6

(44.1)

2.9

(8.3)

0.05

Ages 8 to 17 years

Anxietyn

112

34.83

(6.95)

114

34.31

(7.94)

0.51

(1.32)

0.05

Fearso

113

61.89 (14.82)

115

60.86 (15.10)

1.03

(2.12)

0.05

Substance usep (%)

109

3.01 (22.91)

107

17.08 (35.83) – 14.06*** (5.26) – 0.39

Goal-oriented thinkingq

106

21.98

(4.74)

106

22.99

(5.13)

– 1.00

(0.74) – 0.16

School effort in past monthr

110

2.68

(0.82)

114

2.69

(0.86)

– 0.01

(0.13) – 0.01

Arrests or police involvement in past 6 monthss (%)

59

7.57 (28.09)

48

13.43 (33.42)

– 5.87

(8.05) – 0.15

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse.

See Chapter 2 and Appendix B for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

3-Year Impacts of Housing and Services Interventions for Homeless Families

88

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-18. SUB Versus PBTH: Impacts on Self-Sufficiency at 37 Months

SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

214

37.6

(47.8) 201

39.6

(49.3)

– 2.0

(4.9) – 0.04

Any work for pay since 20-month surveyb (%)

203

61.0

(49.1) 174

59.3

(49.1)

1.8

(5.1)

0.04

Months worked for pay since 20-month surveyb,c

202

6.9

(7.8) 173

7.2

(8.3)

– 0.3

(0.8) – 0.04

Any work for pay since RA (%)

214

72.2

(45.2) 201

74.2

(44.2)

– 2.0

(4.3) – 0.04

Months worked for pay since RAc

210

12.5

(13.4) 199

13.6

(13.7)

– 1.1

(1.2) – 0.07

Hours of work per week at current main jobd

213

11.7

(15.9) 200

13.4

(18.8)

– 1.7

(1.8) – 0.09

Income sources and amounts

Annualized current earnings ($)

209

6,539 (9,835) 198

8,292 (13,045) – 1,754

(1,217) – 0.14

Total family income ($)

201 12,107 (10,068) 199 13,337 (12,164) – 1,230

(1,043) – 0.09

Anyone in family had earnings in past month (%)

215

45.3

(49.8) 201

53.5

(49.9)

– 8.1

(5.1) – 0.14

Anyone in family received TANF in past month (%)

215

28.2

(46.0) 200

26.9

(43.7)

1.3

(4.4)

0.03

Anyone in family received SSDI in past month (%)

213

7.0

(25.6) 201

7.0

(25.5)

0.0

(2.4)

0.00

Anyone in family received SSI in past month (%)

215

14.5

(33.7) 200

12.7

(34.8)

1.8

(3.0)

0.04

Anyone in family received SNAP/Food Stamps in past month (%)

215

79.1

(39.7) 200

80.8

(40.1)

– 1.6

(4.1) – 0.04

Anyone in family received WIC in past month (%)

215

26.3

(44.5) 200

31.5

(46.2)

– 5.2

(4.6) – 0.10

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

213

39.8

(49.5) 199

38.7

(48.9)

1.1

(5.1)

0.02

Number of weeks in school/training programs since RA

210

8.5

(15.9) 198

5.6

(12.7)

2.9*

(1.5)

0.14

Participated in 2 weeks or more of school since RA (%)

213

15.9

(35.3) 199

10.2

(31.4)

5.7*

(3.3)

0.15

Participated in 2 weeks or more of basic education since RA (%)

213

1.1

(11.8) 199

1.1

(10.0)

0.0

(1.2)

0.00

Participated in 2 weeks or more of vocational education since RA (%)

213

9.2

(31.7) 199

11.7

(32.1)

– 2.4

(3.1) – 0.06

Food security

Household is food insecure (%)

215

34.4

(48.3) 201

47.2

(49.8) – 12.8***

(4.9) – 0.22

Food insecurity scalee

214

1.42

(1.92) 198

2.00

(2.03) – 0.58*** (0.20) – 0.24

Economic stressors

Economic stress scalef

212

– 0.23

(0.46) 199

– 0.12

(0.50) – 0.11*** (0.05) – 0.18

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 6-19. SUB Versus PBTH: Earnings and Employment

SUB

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Earnings in quarters 11 to 14 after RA (2015Q3$)

241

5,508 (8,486)

230

6,391

(9,810) – 883

(863) – 0.09

Any employment in quarters 11 to 14 after RA (%)

241

54.8

(49.9)

230

57.4

(49.5)

– 2.6

(4.6) – 0.05

Number of quarters employed in quarters 11 to 14 after RA

241

1.6

(1.7)

230

1.7

(1.7)

– 0.1

(0.2) – 0.04

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Note: See Chapter 2 and Appendix B for outcome definitions.

Source: Quarterly wage records from the National Directory of New Hires

3-Year Impacts of Housing and Services Interventions for Homeless Families

89

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Summary of the SUB-Versus-PBTH Comparison

The lower average work effort of SUB families relative to PBTH

Across Domains

families found at 20 months is not apparent at 37 months.

For the SUB-versus-PBTH comparison, the randomization

Overall, do the families assigned to the SUB group appear to be

resulted in a notable contrast in the mix of program use

doing better at 37 months after random assignment than the

during the 37-month followup period. At 3 years after random

families assigned to the PBTH group? In many ways, the answer

assignment, 82 percent of SUB families had ever used the

is “yes.” The SUB families, on average, are experiencing less

permanent housing subsidies they were offered (compared

homelessness and being doubled up, are more likely to live in

with only 7 percent of PBTH families who had ever used these

their own place, and are living in less crowded conditions than

subsidies) and 50 percent of PBTH families had ever used

are PBTH families. SUB families are also more food secure, have

transitional housing (compared with 9 percent of SUB families).

children who move among schools less and have better grade

In the month of the followup survey response, 70 percent

completion, and have less economic stress. Most of the ways

of SUB families were using some type of permanent housing

in which SUB families appear to be doing better than PBTH

subsidy compared with 27 percent of PBTH families. Of PBTH

families, with the exception of some of the scattered effects on

families, 7 percent were using transitional housing during

children’s well-being, may fade over time if SUB families give

this month compared with only 1 percent of SUB families.

up the permanent housing subsidies to which they had priority

These differences in program participation led to marked

access or if increasing numbers of PBTH families find their way

differences between the experiences of SUB families and PBTH

to permanent housing subsidies.

families in several areas. The most noteworthy effect of the

SUB intervention relative to the PBTH intervention was in its

6.4. The Community-Based Rapid

greater prevention of homelessness. Only 17 percent of SUB

families compared with 41 percent of PBTH families spent

Re-Housing (CBRR) Versus Project-

at least 1 night in an emergency shelter, in a place not meant

Based Transitional Housing (PBTH)

for human habitation, or doubled up in the past 6 months

Comparison

or had a stay in emergency shelter in the 12 months before

the survey. This effect was driven by a large reduction (19

The CBRR-versus-PBTH comparison contrasts having priority

per centage points) in the proportion experiencing doubling

access to temporary private-market rental assistance and

up. Also, only 4 percent of SUB families compared with 10

modest case management of the CBRR intervention with having

percent of PBTH families spent at least 1 night in emergency

priority access to temporary, agency-controlled housing paired

shelter in months 21 to 32 after random assignment.

with intensive supportive services of the PBTH intervention.

Although both interventions provide temporary assistance, the

Compared with assignment to the PBTH intervention, assign-

length of assistance differs. CBRR program assistance is usually

ment to the SUB intervention also increased the proportion of

provided for 7 to 8 months and is potentially renewable for up

families living in their own house or apartment at the time of

to 18 months. PBTH program assistance provides housing for

the followup survey from 66 to 85 percent. The greater stability

up to 24 months, with average stays of 13 months during the

afforded by the permanent housing subsidies offered to SUB

3-year followup period. PBTH programs offer comprehensive

families was evidenced in a reduction in the number of places

case management and provide many supportive services

families lived in the past 6 months and a reduction in the

directly. Some services beyond assistance with housing search,

number of schools that children attended since study entry.

largely focused on self-sufficiency, were offered to about

The SUB-versus-PBTH comparison yields a handful of statis-

three-fourths of CBRR families, in general, with lower intensity

tically significant effects in the family preservation, child well-

than services offered to PBTH families. Employment and

being, and self-sufficiency domains, almost all of which favor

training services were offered by nearly all PBTH programs

assignment to the SUB group. Among these domains, the most

(representing 92 percent of families assigned to the PBTH

notable effects of assignment to the SUB intervention relative to

group) but by only a minority of CBRR programs (representing

assignment to the PBTH intervention are an 8-percentage-point

37 percent of families assigned to the CBRR group), and most

reduction in the proportion of families who had a child sepa-

PBTH programs offered services that addressed psychosocial

ration in the past 6 months (from 23 to 15 percent), a decrease

challenges (see the Short-Term Impacts report [Gubits et al.,

in the number of schools children attended, an increase in

2015]). The CBRR case managers typically served about 36

children’s grade completion, and a decrease in the proportion

families each, roughly twice as many families as the typical

of families who were food insecure (from 47 to 34 percent).

PBTH case manager.

3-Year Impacts of Housing and Services Interventions for Homeless Families

90

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

6.4.1. Program Use by Families in the

The proportions of PBTH families using various types of pro-

CBRR-Versus-PBTH Comparison

grams shown in the second column are similar to the proportions

of all PBTH families shown in Exhibit 5-1 in Chapter 5. The

Exhibit 6-20 shows the use of eight types of homeless and

first two columns show that 55 percent of families assigned to

housing programs by the 180 CBRR families and 184 PBTH

the CBRR group used rapid re-housing (whereas only 16 per-

families analyzed in the CBRR-versus-PBTH comparison.113

cent of PBTH families did so) and 53 percent of PBTH families

The first column shows some differences in the general pattern

used transitional housing (whereas 31 percent of CBRR families

of usage for these CBRR families compared with all CBRR

did so). The number of months of program use (in columns three

families (shown in Exhibit 4-1 in Chapter 4). Compared with

through six) and the proportions using the program in the

the proportion among all families randomly assigned to the

month of the 37-month followup survey response (in columns

CBRR group, lower proportions of the CBRR families in this

seven and eight) are largely similar to those in the exhibits in

pairwise comparison used rapid re-housing (55 compared with

the chapters that address PBTH-versus-UC and CBRR-versus-

59 percent), a permanent housing subsidy of the type offered

UC comparisons. By the followup survey month, less than one-

to families assigned to the SUB group (7 compared with 10 per-

half of both groups are participating in any program. Of the

cent), and any form of permanent subsidy (32 compared with

CBRR families, 37 percent are participating in some program

35 percent). Compared with the proportion among all families

in the survey month compared with 40 percent of the PBTH

assigned to the CBRR group, a higher proportion of CBRR fam-

families.114 Only 3 percent of CBRR families were receiving

ilies in this comparison used transitional housing (31 compared

rapid re-housing in the survey month and 9 percent of PBTH

with 23 percent), and a higher proportion used no homeless

families were receiving transitional housing at this point.

or housing assistance program (including emergency shelter)

during the followup period (14 compared with 9 percent).

Exhibit 6-20. CBRR Versus PBTH: Program Use Since RA

Percent Ever Used

Number of Months Used

Percent Used in

From RA to 37-Month

From RA to 37-Month Followup Survey,

Month of Followup

Type of Homeless or

Followup Surveya

if Ever Used Type of Assistance

Survey Response

Housing Assistance

CBRR

PBTH

CBRR

PBTH

CBRR

PBTH

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

6.7

8.7

24.7

26.5

15.7

10.5

6.8

6.2

SUB groupb

Rapid re-housingc

54.6

15.8

7.1

6.5

6.8

4.5

2.7

0.4

Transitional housingd

30.7

53.4

10.0

6.5

14.0

12.0

5.1

9.1

Permanent supportive housing

9.2

10.7

17.2

15.5

15.9

14.5

7.6

6.8

Public housing

11.5

9.1

16.7

16.5

19.0

20.5

10.1

8.2

Project-based vouchers/Section 8 projects

4.8

6.1

19.2

18.5

16.5

17.5

4.9

3.8

Any permanent housing subsidye

32.3

33.3

18.9

18.5

17.4

17.5

29.3

25.1

Emergency shelterf

88.1

86.5

3.7

2.4

3.2

2.2

2.8

2.0

No use of homeless or housing programsg

14.3

16.8

—

—

—

—

60.1

63.4

N

180

184

—

—

—

—

180

184

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the month of RA to the month of 37-month followup survey response (median period duration: 38 calendar months). Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Temporary subsidies offered to the CBRR group.

d All types of transitional housing, including those offered to the PBTH group.

e Includes the types of permanent subsidy offered to the SUB group plus permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

f All families were in emergency shelter at RA. Percentages less than 100 percent for ever used emergency shelter are because of missing data on shelter use.

g Indicates no use of the first six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

113 The CBRR-versus-PBTH comparison sample consists of 232 families assigned to the CBRR group and 239 families assigned to the PBTH group. Of those 471 families, 180 CBRR families and 184 PBTH families (77 percent) responded to the 37-month followup survey.

114 These proportions of families participating in any program are calculated by subtracting from 100 percent the proportions with no use of programs in the survey month (shown in the exhibit).

3-Year Impacts of Housing and Services Interventions for Homeless Families

91

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

6.4.2. Impacts of the CBRR Intervention

lay the foundation for later housing stability. Consistent with

Compared With the PBTH Intervention

this perspective, project-based transitional housing proponents

expect that the PBTH intervention will improve long-term

Hypotheses about rapid re-housing and project-based transitional

housing stability, employment, earnings, education, and adult

housing arise from divergent views about the needs of families

well-being relative to assignment to the CBRR intervention and

who experience homelessness and the package of housing

may improve family preservation and child well-being. Because

assistance and services best suited to address these needs.

the PBTH intervention addresses barriers to housing stability

Rapid re-housing attempts to return families quickly to the

and attempts to put families on track for better employment

private rental market, with modest services directed toward

and earnings, many project-based transitional housing proponents

housing and self-sufficiency to help them stay there. From the

would expect outcomes in these domains. This section presents

perspective of community-based rapid re-housing proponents,

the 3-year experimental impact evidence on these divergent

this rapid return to the private rental market should be the

expectations.

primary objective of homeless interventions. Assignment to

the CBRR intervention is thus expected to reduce the length of

Impacts on Housing Stability in the CBRR-Versus-

the shelter stay from the time of study entry. The CBRR inter-

PBTH Comparison

vention is also expected to reduce homelessness relative to the

Exhibit 6-21 shows the effect on housing stability of being

PBTH intervention and may improve housing stability, family

assigned to the CBRR group relative to being assigned to the PBTH

preservation, adult well-being, child well-being, employment,

group. The exhibit shows significant impacts on only 1 of

and earnings. Proponents of project-based transitional housing

14 outcomes examined. The analysis shows no evidence that

emphasize that most families who become homeless have

assignment to the CBRR group relative to assignment to the

additional barriers that make it difficult for them to secure and

PBTH group led to differential experiences of homelessness,

maintain housing. The services in project-based transitional

doubling up, or use of emergency shelter at 37 months after

housing programs are designed to address these barriers and

random assignment.

Exhibit 6-21. CBRR Versus PBTH: Impacts on Housing Stability at 37 Months

CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the follow up period

At least 1 night homelessb or doubled up in past 6 months or in shelter in

180

40.1

(49.7) 183

43.2

(49.7)

– 3.1

(6.2) – 0.06

past 12 months (%) [confirmatory]c

At least 1 night homelessb or doubled up in past 6 months (%)

180

35.6

(48.8) 183

38.4

(49.0)

– 2.8

(6.0) – 0.05

At least 1 night homelessb in past 6 months (%)

180

20.2

(40.5) 184

19.8

(41.0)

0.5

(4.9)

0.01

At least 1 night doubled up in past 6 months (%)

180

30.3

(46.9) 183

28.3

(45.7)

2.0

(5.7)

0.04

Any stay in emergency shelter in past 6 months (%) [Program Usage Data]

180

7.9

(26.9) 184

4.9

(23.8)

3.0

(2.9)

0.09

Any stay in emergency shelter in months 21 to 32 after RA (%) [Program

180

12.8

(33.5) 184

11.2

(31.9)

1.6

(4.0)

0.04

Usage Data]

Number of days homelessb or doubled up in past 6 months

180

50.4

(75.7) 183

44.7

(71.4)

5.7

(9.1)

0.07

Number of days homelessb in past 6 months

180

16.2

(45.8) 184

16.5

(45.8)

– 0.4

(5.3) – 0.01

Number of days doubled up in past 6 months

180

36.4

(67.2) 183

30.7

(60.7)

5.7

(7.8)

0.08

Housing independence

Living in own house or apartment at followup (%)

180

66.4

(47.7) 184

68.1

(47.0)

– 1.7

(5.8) – 0.03

Living in own house or apartment with no housing assistance (%)

179

37.7

(48.9) 184

41.3

(48.9)

– 3.6

(5.4) – 0.06

Living in own house or apartment with housing assistance (%)

179

29.3

(44.4) 184

26.4

(45.1)

2.9

(5.2)

0.06

Number of places lived

Number of places lived in past 6 monthsd

180

1.7

(1.1) 181

1.6

(1.1)

0.1

(0.1)

0.07

Housing quality

Persons per room

168

2.0

(1.6) 175

1.7

(1.3)

0.3

(0.2)

0.25

Housing quality is poor or fair (%)

171

25.2

(44.2) 175

35.4

(48.0) – 10.1*

(5.4) – 0.19

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-PBTH comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

92

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

The bottom three panels of the exhibit show no statistically

Impacts on Adult Well-Being in the CBRR-Versus-

significant differences between CBRR and PBTH families in

PBTH Comparison

housing independence at the time of the 37-month followup

Exhibit 6-24 shows statistically significant effects on 3 of

survey or number of residential moves in the 6 months before

the 8 adult well-being outcomes in the CBRR-versus-PBTH

the survey. Families assigned to the CBRR group were less

comparison. All three significant effects show the CBRR inter-

likely to report poor or fair housing quality than were families

vention produced more favorable outcomes than did the PBTH

assigned to the PBTH group (24 compared with 34 percent).

intervention at 37 months after random assignment.

Impacts on Family Preservation in the CBRR-

Assignment to the CBRR group led to lower levels of psycho-

Versus-PBTH Comparison

logical distress than for families assigned to the PBTH group,

Exhibit 6-22 shows the effects on family preservation in the

with an effect size of 0.19. Having priority access to CBRR

CBRR-versus-PBTH comparison. The study team finds no

programs compared with PBTH programs also reduced the

evidence of differential effects of these interventions on family

number of families who reported drug abuse, from 5.6 to 2.0

separations or reunifications in the survey data at 37 months,

percent, and the number reporting either alcohol dependence

although the numbers, particularly in the case of reunifications

or drug abuse, from 14.5 to 7.4 percent. These results are

of family members separated at the time of the baseline survey,

surprising, because PBTH programs have an explicit focus

were too small to yield a strong test. The study team also finds

on adult well-being. No longer-term differences exist on the

no evidence of effects on measures of formal child separations

remaining five measures of physical and mental health, intimate

in the child welfare administrative data from five sites

partner violence, or alcohol dependence.

(Exhibit 6-23).

Exhibit 6-22. CBRR Versus PBTH: Impacts on Family Preservation at 37 Months

CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

178

16.7

(37.0)

177

20.0

(40.4)

– 3.2

(4.3) – 0.07

Family has at least one foster care placement in past 6 monthsb (%)

178

5.7

(22.0)

179

2.1

(14.8)

3.6

(2.8)

0.17

Spouse/partner separated in past 6 months, of those with spouse/partner

62

30.1

(45.0)

44

29.6

(46.2)

0.5

(9.7)

0.01

present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at least one

46

44.8

(49.8)

33

44.8

(50.6)

0.1

(13.3)

0.00

child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent at RA (%)

19

14.5

(45.2)

15

43.3

(45.8)

– 28.9

(26.8) – 0.61

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 6-23. CBRR Versus PBTH: Impacts on Child Welfare Outcomes

CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Had a formal child separation that began after RA (%)

101

14.1

(34.7)

98

14.1

(35.2)

0.0

(5.3)

0.04

Total days during followup separated from at least one childb

101

57.9 (197.6)

98

106.8 (282.4)

– 48.9

(30.2)

– 0.20

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes separations started before and after RA. Length of followup varies by site. Alameda County = 1,075 days. Baltimore = 1,071 days. Kansas City = 1,069 days.

Minneapolis = 1,046 days. Phoenix = 1,123 days.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions. Sample limited to five sites where child welfare records were collected (Alameda County, Baltimore, Kansas City, Minneapolis, and Phoenix).

Source: State child welfare agency records

3-Year Impacts of Housing and Services Interventions for Homeless Families

93

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-24. CBRR Versus PBTH: Impacts on Adult Well-Being at 37 Months

CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

179

31.8

(47.1)

184

32.8

(46.6)

– 1.0

(5.4) – 0.02

Adult mental health

Goal-oriented thinkingb

177

4.41

(0.98)

183

4.36

(1.07)

0.05

(0.13)

0.04

Psychological distressc

179

6.13

(5.62)

184

7.44

(5.82)

– 1.32**

(0.66) – 0.19

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past 30 days (%)

179

18.7

(37.5)

183

21.1

(41.8)

– 2.4

(4.7) – 0.05

Adult substance use

Alcohol dependence or drug abused (%)

179

7.4

(28.6)

184

14.5

(36.0)

– 7.1**

(3.4) – 0.19

Alcohol dependenced (%)

179

7.3

(27.8)

184

11.9

(33.2)

– 4.5

(3.1) – 0.14

Drug abused (%)

179

2.0

(14.8)

184

5.6

(23.8)

– 3.6*

(2.0) – 0.13

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

180

8.8

(28.5)

184

8.6

(29.8)

0.2

(3.5)

0.01

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Impacts on Child Well-Being in the CBRR-Versus-

Impacts on Self-Sufficiency in the CBRR-Versus-

PBTH Comparison

PBTH Comparison

Differential effects of the CBRR and PBTH interventions on

Exhibit 6-27 shows 1 statistically significant effect on 20

child well-being would be expected to be indirect, via effects

self-sufficiency outcomes examined in the CBRR-versus-PBTH

on housing stability, self-sufficiency, and adult well-being,

comparison. Compared with assignment to the PBTH group,

which were modest.

assignment to the CBRR group led to improvements in food

security at the time of the survey (46 percent of PBTH families

Exhibit 6-25 shows the cross-age impacts of assignment to the

reported being food insecure compared with 35 percent of

CBRR group rather than to the PBTH group for child well- being.

CBRR families).115 This effect is somewhat surprising, because

Of nine results examined, two reach statistical significance,

the 37-month analysis provides no evidence that assignment to

both favoring the CBRR group. Children in families assigned to

the CBRR group led to differential effects on work effort or oth-

the CBRR group were 9 percentage points more likely to have a

er measures of self-sufficiency when compared with assignment

regular source of health care than were children in PBTH fami-

to the PBTH group. Exhibit 6-20 (in a previous section) shows

lies (panel 2). They also have lower levels of behavior problems

similar levels of receipt of permanent housing subsidies, so it is

by parental report (0.13 standard deviations, panel 3). Of 20

not apparent that CBRR families have the additional financial

age-specific results in Exhibit 6-26, 1 reached significance. As

resources that would be the expected mechanism by which

shown in panel 3 of Exhibit 6-26, children of ages 8 to 17 in

food security is increased.

the CBRR group reported less school effort (one-fourth of the

distance between “could have tried a little harder” and “tried

Exhibit 6-28 shows impact estimates from administrative data

about as hard as you could,” on average) than did children in

on outcomes for quarters 11 to 14 after the quarter of random

the PBTH group. Overall, out of 29 results examined, 3 were

assignment. The analysis of administrative data finds no effect

significant and none showed a clear pattern of effects.

of assignment to the CBRR group relative to assignment to the

115 The improvement in food security is almost identical to the reduction from 45.7 to 33.8 percent (-11.9 percentage points) found in a corrected analysis of the 20-month survey data.

3-Year Impacts of Housing and Services Interventions for Homeless Families

94

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-25. CBRR Versus PBTH: Impacts on Child Well-Being Across Age Groups at 37 Months CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

206

2.2

(1.0)

212

2.2

(1.0)

0.0

(0.1)

0.01

Grade completion (not held back) (%)

176

91.40 (28.00)

176

88.64 (31.83)

2.76

(3.54)

0.07

School gradesc

163

3.0

(0.9)

155

3.0

(0.9)

0.0

(0.1)

0.00

Child physical health

Poor or fair health in past 30 days (%)

244

3.5

(18.9)

248

5.7

(21.5)

– 2.2

(2.7) – 0.07

Well-child checkup in past year (%)

245

91.7

(29.2)

248

90.3

(29.6)

1.4

(3.3)

0.04

Child has regular source of health care (%)

245

95.4

(20.8)

246

86.1

(31.3)

9.2**

(4.3)

0.25

Sleep problemsd

244

2.07

(1.02)

247

2.17

(1.08)

– 0.10

(0.11) – 0.07

Child behavioral strengths and challenges

Behavior problemse

236

0.26

(1.13)

233

0.49

(1.16)

– 0.23*

(0.12) – 0.13

Prosocial behaviorf

236 – 0.12

(1.07)

233 – 0.20

(1.17)

0.08

(0.13)

0.05

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

PBTH group on total earnings, the proportion of family heads

assignment to the PBTH intervention. Of the eight outcomes

employed, or the number of quarters employed during the year.

on which statistically significant impacts in this comparison

were found, only three had statistically significant impacts

Summary of the CBRR-Versus-PBTH Comparison

detected in any other study comparison (psychological distress,

Across Domains

children’s behavior problems, and proportion of families

For a number of reasons, the CBRR-versus-PBTH comparison

who are food insecure). It seems surprising that outcomes on

offers a weaker test than do the other pairwise comparisons in

which no impacts were detected in comparisons with the SUB

the study. The number of families in this comparison sample

intervention (which had stronger contrasts in program use

is the lowest of the pairwise comparisons, and so it provides

and clear patterns of impacts) would show differentials in the

less statistical power than the other tests.116 In addition, takeup

CBRR-versus-PBTH comparison (which has a weaker contrast

rates for the programs offered in the assigned interventions

in program use). Therefore, the three effects on psychological

offered in the assigned groups—53 percent for PBTH families

distress, children’s behavior problems, and proportion of

and 55 percent for CBRR families—are somewhat lower than

families who are food insecure—all of which show favorable

for other comparisons.

impacts of assignment to the CBRR intervention relative to as-

The CBRR-versus-PBTH comparison yields a somewhat enig-

signment to the PBTH intervention—seem the least likely to be

matic pattern of effects across the five study domains. Of the

due to chance. Given the relatively few statistically significant

eight statistically significant effects found in this comparison,

results for this comparison, however, the study team hesitates

six show more favorable outcomes for assignment to the

to draw strong conclusions for this comparison.

CBRR intervention and two show more favorable outcomes for

116 The smaller comparison sample is, in large part, a result of the greater selectivity of PBTH programs, leading to the absence of the PBTH intervention from the randomization sets of 356 families. See Gubits et al. (2013), Exhibit 3-5, for more information on the relative selectivity of SUB, CBRR, and PBTH programs.

3-Year Impacts of Housing and Services Interventions for Homeless Families

95

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-26. CBRR Versus PBTH: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

75

34.7

(47.9)

79

30.9

(48.1)

3.8

(8.6)

0.06

Child care or preschool absences in past monthd

24

0.45

(0.93)

29

0.74

(0.82)

– 0.29

(0.36) – 0.26

Positive child care or preschool experiencese

24

0.99

(0.00)

31

0.75

(0.51)

0.23

(0.14)

0.49

Positive child care or preschool attitudesf

24

4.57

(0.76)

30

4.41

(1.10)

0.16

(0.32)

0.19

Child care or preschool conduct problemsg (%)

26

1.6

(0.0)

33

11.8

(36.4)

– 10.2

(10.3) – 0.37

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

59

69.6

(48.3)

69

59.2

(48.8)

10.4

(11.0)

0.18

Ages 3 years, 6 months to 7 years

Verbal abilityi

78 – 0.27

(1.13)

85 – 0.43

(0.97)

0.16

(0.22)

0.13

Math abilityj

75 – 0.41

(0.92)

85 – 0.46

(1.07)

0.05

(0.21)

0.04

Executive functioningk (self-regulation)

69

14.35 (16.45)

81

15.46 (15.29)

– 1.12

(2.47) – 0.05

Ages 5 to 17 yearsl

School enrollmentc (%)

171

98.8

(10.8)

169

94.2

(17.0)

4.6

(4.2)

0.24

School absences in past monthd,m

55

0.79

(0.91)

63

0.96

(0.96)

– 0.17

(0.18) – 0.13

Positive school experiencese,m

54

0.50

(0.57)

66

0.56

(0.61)

– 0.06

(0.15) – 0.07

Positive school attitudesf,m

55

4.08

(1.11)

66

4.14

(1.05)

– 0.05

(0.19) – 0.04

School conduct problemsg,m (%)

54

27.1

(45.2)

66

33.8

(47.5)

– 6.6

(9.1) – 0.11

Ages 8 to 17 years

Anxietyn

109

34.75

(7.13)

103

35.61

(7.45)

– 0.86

(1.17) – 0.08

Fearso

111

64.16 (14.83)

105

63.90 (14.75)

0.26

(1.62)

0.01

Substance usep (%)

108

4.74 (23.01)

100

8.84 (31.45)

– 4.09

(4.12) – 0.11

Goal-oriented thinkingq

105

22.39

(4.48)

100

22.84

(4.72)

– 0.45

(0.72) – 0.07

School effort in past monthr

109

2.42

(0.79)

104

2.69

(0.82)

– 0.27**

(0.12) – 0.25

Arrests or police involvement in past 6 monthss (%)

58

13.55 (32.86)

57

16.54 (38.37)

– 2.99

(7.34) – 0.08

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse.

See Chapter 2 and Appendix B for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

3-Year Impacts of Housing and Services Interventions for Homeless Families

96

FAMILY OPTIONS STUDY Chapter 6. Impacts of Permanent Housing Subsidy (SUB) Compared With Community-Based Rapid Re-Housing (CBRR), SUB Compared With Project-Based Transitional Housing (PBTH), and CBRR Compared With PBTH

Exhibit 6-27. CBRR Versus PBTH: Impacts on Self-Sufficiency at 37 Months

CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

180

36.6

(48.9) 184

40.4

(48.4)

– 3.8

(5.1) – 0.07

Any work for pay since 20-month surveyb (%)

166

59.9

(48.8) 169

59.5

(49.5)

0.3

(5.4)

0.01

Months worked for pay since 20-month surveyb,c

166

6.5

(7.5) 168

7.8

(8.6)

– 1.3

(0.9) – 0.16

Any work for pay since RA (%)

180

72.3

(43.4) 184

71.9

(45.4)

0.4

(4.5)

0.01

Months worked for pay since RAc

179

13.9

(13.2) 183

14.3

(14.2)

– 0.4

(1.3) – 0.03

Hours of work per week at current main jobd

180

12.4

(17.3) 184

12.7

(17.1)

– 0.3

(1.8) – 0.02

Income sources and amounts

Annualized current earnings ($)

176

6,808 (10,823) 182

7,535 (11,394)

– 727

(1,221) – 0.06

Total family income ($)

170 11,161 (9,110) 182 12,906 (10,608) – 1,745

(1,170) – 0.13

Anyone in family had earnings in past month (%)

180

50.8

(50.0) 184

54.4

(50.1)

– 3.6

(5.4) – 0.06

Anyone in family received TANF in past month (%)

180

26.2

(43.1) 184

23.3

(43.1)

2.9

(4.8)

0.06

Anyone in family received SSDI in past month (%)

180

9.6

(26.9) 184

7.5

(27.4)

2.1

(2.9)

0.06

Anyone in family received SSI in past month (%)

180

11.9

(32.2) 184

13.2

(33.2)

– 1.3

(3.4) – 0.03

Anyone in family received SNAP/Food Stamps in past month (%)

180

80.5

(36.9) 184

85.4

(37.5)

– 5.0

(4.0) – 0.11

Anyone in family received WIC in past month (%)

180

28.3

(46.2) 184

32.0

(46.4)

– 3.7

(5.2) – 0.07

Education and training

Participated in 2 weeks or more of any school or training since RA (%)

180

37.1

(48.9) 183

36.5

(48.5)

0.6

(5.3)

0.01

Number of weeks in school/training programs since RA

176

6.9

(16.6) 182

5.2

(10.7)

1.7

(1.7)

0.08

Participated in 2 weeks or more of school since RA (%)

179

8.6

(27.8) 183

9.6

(30.6)

– 1.0

(3.4) – 0.03

Participated in 2 weeks or more of basic education since RA (%)

179

1.8

(14.8) 183

3.3

(16.3)

– 1.5

(2.0) – 0.09

Participated in 2 weeks or more of vocational education since RA (%)

179

6.6

(24.1) 183

9.9

(30.6)

– 3.3

(3.3) – 0.08

Food security

Household is food insecure (%)

180

34.6

(47.3) 184

46.2

(50.1) – 11.6**

(5.5) – 0.20

Food insecurity scalee

180

1.46

(1.86) 182

1.76

(2.04) – 0.29

(0.21) – 0.12

Economic stressors

Economic stress scalef

178

– 0.14

(0.51) 183

– 0.13

(0.54) – 0.01

(0.06) – 0.02

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit 6-28. CBRR Versus PBTH: Earnings and Employment

CBRR

PBTH

ITT Impact

Outcome

Effect

sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Earnings in quarters 11 to 14 after RA (2015Q3$)

224

6,233 (10,781) 227

5,254

(9,233)

979

(949)

0.10

Any employment in quarters 11 to 14 after RA (%)

224

54.8

(49.9) 227

51.7

(50.1)

3.1

(4.8)

0.06

Number of quarters employed in quarters 11 to 14 after RA

224

1.6

(1.7) 227

1.5

(1.7)

0.1

(0.2)

0.04

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Note: See Chapter 2 and Appendix B for outcome definitions.

Source: Quarterly wage records from the National Directory of New Hires

3-Year Impacts of Housing and Services Interventions for Homeless Families

97

FAMILY OPTIONS STUDY

CHAPTER 7.

IMPACTS OF POOLED COMPARISONS

In addition to conducting the six pairwise comparisons, the One benefit of pooling the three interventions and usual care study team combined assignment groups in various ways

in impact comparisons is that it provides larger sample sizes

to examine additional comparisons. In the design phase of

for analysis. A family was included in a pooled comparison

the study, the study team identified four questions as being of

if its randomization set included at least one intervention on

interest to HUD.

each side of the impact comparison. For example, a family was

included in the SUB + PBTH comparison with CBRR if it had

1. What is the impact of having priority access to any kind of

the opportunity to be assigned to the CBRR group and to either

housing subsidy for homeless families (permanent housing

the SUB or PBTH group.117 Exhibit 7-1 shows the number of

subsidy [SUB] + community-based rapid re-housing [CBRR]

families who are included in the comparisons used to address

+ project-based transitional housing [PBTH]) compared with

the preceding questions.

the impact of the usual care offered in the community?

The study team’s examination of the impact results from the

2. What is the impact of having priority access to a housing

four pooled comparisons unexpectedly yielded little useful

subsidy with heavy services provided to homeless families

information on the questions posed. Instead, all the estimates

(PBTH) compared with the impact of having priority access

appear to be dominated by the relatively large effects of the per-

to housing subsidies with light or no services (SUB + CBRR)?

manent housing subsidies offered in the SUB assignment group

3. What is the impact of assignment to interventions that offer

when compared with any of the other assignment groups, no

programs that are more costly (SUB + PBTH) compared with

matter how the different randomly assigned interventions are

the impact of assignment to groups that offer a less costly

grouped. Therefore, the results of the pooled comparisons are

intervention (CBRR)?

not addressed here in the body of the report; rather, the results

are provided in Appendix F, with no additional discussion.

4. What is the impact of having priority access to a housing

subsidy that has no time limit (SUB) compared with the

impact of having priority access to housing subsidy pro-

grams that have time limits (CBRR + PBTH)?

Exhibit 7-1. Sample Sizes in the Four Pooled Comparisons

Sample Size in Pooled Comparisona

Assigned Intervention

SUB + CBRR + PBTH vs. UC

SUB + CBRR vs. PBTH

SUB + PBTH vs. CBRR

CBRR + PBTHvs. SUB

SUB

501

—

—

463

CBRR

434

—

382

—

PBTH

293

290

—

—

UC

556

—

—

—

395 (SUB+CBRR)

546 (SUB+PBTH)

491 (CBRR+PBTH)

Total

1,784

685

928

954

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

a Sample sizes are number of families who responded to the 37-month followup survey.

Source: Family Options Study 37-month followup survey

117 The randomization sets that provided the opportunity to be assigned to the CBRR group and to either the SUB or PBTH groups were {SUB, CBRR, PBTH, UC}, {SUB, CBRR, UC}, and {CBRR, PBTH, UC}.

3-Year Impacts of Housing and Services Interventions for Homeless Families

98

FAMILY OPTIONS STUDY

CHAPTER 8.

DO CERTAIN INTERVENTIONS WORK

BETTER WHEN OFFERED TO FAMILIES

WHO FACE GREATER DIFFICULTIES?

Previous chapters of this report have examined which This chapter explores whether, 37 months after random assign-interventions work best, on average, across all families

ment, relative impacts vary depending on these two types of

in the study. This chapter asks whether some inter-

family needs. The study team found no evidence of differential

ventions work better than others for families who have greater

intervention impacts for families who had different levels of

needs, measured at the outset of the study.

need 20 months after random assignment, but it is possible

that such impacts would take longer to emerge. To evaluate

The theory behind the interventions suggests that some types

this possibility, the study team created indices of psychosocial

of programs may be especially effective for families who have

challenges and housing barriers measured at the time of the

two types of needs: psychosocial challenges, such as domestic

baseline survey and examined whether the impact of the three

violence, psychological distress, or disability, and housing

interventions relative to each other and to usual care increases

barriers, such as lack of employment, lack of income to pay

as families’ scores on these indices increase. Because the

rent, or poor credit histories. Project-based transitional housing

interventions are presented to families as packages, the study

programs—with their extensive services—are designed to ad-

team considered differential effects across the array of outcomes

dress families’ psychosocial challenges. It is therefore plausible

preselected for inclusion in the executive summary.

that families facing more of those challenges may benefit, even

if families who have fewer psychosocial challenges do no better

Differential impact for any particular outcome could take more

than UC families. Because addressing these issues is thought to

than one form. It is possible that one intervention in a pairwise

lay the foundation for success in housing, differential benefits

comparison would be superior for that outcome for families

of project-based transitional housing for families who have

who have greater needs and the other would be superior for

more challenges may extend to other domains. The paucity of

families who have lower needs. This form may be most likely in

effects of assignment to the PBTH intervention, on average, may

a pair in which no average impact is found for that comparison.

mask differential effects for families who have greater needs.

As an alternative, one intervention in the pair may always be

superior for a particular outcome, but its superiority would

In a similar way, although the benefits of assignment to the

be less marked for families who have greater or lower needs.

SUB intervention are substantial across domains, it is possible

This form may be most likely for a pair in which a substantial

that not all families need long-term subsidies; families who

intervention impact is found, on average.

have relatively few housing barriers may do as well with the

less intensive CBRR programs or usual care. Because many

Patterns in measured impacts by level of need also require

PBTH programs provide job training and the SUB programs

subtle statistical interpretation. For each comparison (for ex-

provide housing despite low incomes caused by poor health or

ample, SUB versus PBTH) and each need index—psychosocial

disability, each of those types of programs may be especially

challenges, housing barriers—the study team conducted 18

successful with families who have both types of needs. Con-

statistical tests of whether impacts are different for families

versely, CBRR programs, with shorter subsidies and fewer ser-

who have higher or lower needs, 1 for each of the 18 primary

vices, may benefit families facing fewer psychosocial challenges

outcome measures featured in the analysis. With this number

or housing barriers but may be insufficient for families who

of tests, even if the indexes have no true relationship to

have higher levels of needs. The general form of this hypothesis

impacts, the chance of a statistically significant result on 1 or

is that the more intensive programs may have larger impacts

more tests is very high. A single test has a 10-percent chance

for families who have greater needs; the less intensive CBRR

of a “false positive” result of this kind. Among 18 statistically

programs may have larger impacts for families who have fewer

independent tests, the odds of at least one “false positive”

needs.

3-Year Impacts of Housing and Services Interventions for Homeless Families

99

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

when no true relationship is present goes up to 85 percent. To

to pay a security deposit or first and last month’s rent were almost

guard against overinterpreting findings in this circumstance,

universal. Four-fifths of families cited lack of current employ-

the study team does not credit apparently significant results as

ment as a barrier, three-fourths cited poor credit history, and

evidence of real differences in impact by need level unless 4 or

two-thirds cited lack of transportation to look for housing.

more of the 18 test results show significant findings at the .10

level—something that, with independent tests, has only a .098

(that is, very close to the .10 level used in the balance of the

Exhibit 8-1. Percentage of Adult Respondents Reporting

report) chance of occurring absent any true relationship. Thus,

Psychosocial Challenges at the Time of Study Enrollment

for a given need index and comparison, this report discusses

(for families interviewed at 37 months)

only the findings if significant results appear for 4 or more of

Psychosocial Challenges

Percent

the 18 tested outcomes. Given the positive dependence among

Domestic violence

50.0

Poor or fair health

30.9

the tests (because all are based on the same sample of families

In foster care or institution as child

26.9

for any given comparison, and outcomes are associated with

Severe psychological distress

22.1

one another), the study team cannot be sure that every such

Disability

21.6

case has an extremely low likelihood of reflecting only chance

Post-traumatic stress disorder (PTSD)

21.6

Drug abuse or alcohol dependency

21.2

differences,118 but, in exploratory analyses this approach seems

Child with disability

17.5

a reasonable standard for exercising caution in interpreting

Past felony

11.1

statistical results. If the number of statistically significant

Notes: Although psychosocial challenges were measured at baseline, percentages

results exceeds 4 out of 18 for a given comparison, the study

differ slightly from the short-term impacts report because a somewhat different group

of families responded to the 37-month survey. Sample reported is the 1,784 families

team then examined the patterns of results and whether they

who responded to the 37-month survey.

conform to the hypothesis that the more intensive intervention

Source: Family Options Study baseline survey

will have larger impacts for needier families—those with higher

levels of challenges or barriers.

Exhibit 8-2. Percentage of Adult Respondents Reporting

That a Condition Was a Big or Small Problem in Finding a

8.1. Descriptive Results for

Place To Live at the Time of Study Enrollment (for families

interviewed at 37 months)

Psychosocial Challenges and

Percent

Housing Barriers

Housing Barriers

Reporting

Big or Small

Problem

The psychosocial challenge index is a count of the number

Not enough income to pay rent

96.4

of nine potential psychosocial challenges reported by families

Inability to pay a security deposit or first/last month’s rent

93.3

at the baseline survey just before random assignment. As

Not currently employed

79.8

shown in Exhibit 8-1, the most common challenge, by far,

Poor credit history

73.5

was experiencing domestic violence, affecting one-half of

Lack of transportation to look for housing

65.5

No reference from past landlords

43.6

respondents (at some time in adulthood), with having current

Past eviction

39.4

poor or fair health and being in foster care or an institution

No rent history at all

38.3

as a child affecting more than one-fourth of families. Having

Recently moved to a community and no local rent history

32.2

psychological distress, a disability, drug or alcohol dependence,

Problems with past landlords

18.6

and a child with a disability each affected about one-fifth of the

Three or more children in the household

17.6

Racial discrimination

17.1

respondents. Overall, families experienced an average of 2.2

Past lease violations

15.9

challenges at study entry.

Someone in the household less than 21 years old

8.6

Teenagers in the household

5.9

The housing barriers index is a count of the number of 15

Notes: Although housing barriers were measured at baseline, percentages differ

potential barriers reported by families at the baseline survey.

slightly from the short-term impacts report because a somewhat different group of

Families reported an average of 6.5 housing barriers. As shown

families responded to the 37-month survey. Sample reported is the 1,784 families who

responded to the 37-month survey.

in Exhibit 8-2, both insufficient income to pay rent and inability

Source: Family Options Study baseline survey

118 Positive statistical dependence among the test results, as opposed to total statistical independence, raises the probability of 4 or more statistically significant results among 18 tests above .098 by some amount. Explicit adjustment of test findings for the multiple comparisons involved here would take this dependence into account, but here—and in other exploratory analyses in this report—that extent of protection against “false positive” conclusions is judged to be overly cautious and complex; only when judging impacts on the evaluation’s confirmatory outcome (at least 1 night homeless or doubled up in the past 6 months or in shelter in the past 12 months) does the study team apply such strict and explicit checks on the multiple comparison risk involved.

3-Year Impacts of Housing and Services Interventions for Homeless Families

100

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

The indices of challenges and barriers were positively correlated,119

8.2.2. CBRR Versus UC

indicating that families who have high levels of challenges were

The CBRR intervention, offering short-term subsidies and

also somewhat more likely to have high levels of barriers, and

low-intensity services focused on housing and self-sufficiency,

vice versa. The two indices reflect separate but related measures

may be insufficient for families who have high levels of

of the difficulties families face.

psychosocial challenges. It would be expected to work better

compared with usual care for families who have fewer challeng-

8.2. Differential Impacts Depending

es. Only 1 of 18 tests was statistically significant—slightly fewer

on Psychosocial Challenges

than would be expected by chance alone if no true impact

variation occurred. Previous chapters found little impact of

Exhibit 8-3 shows the estimated size of the impact of each

assignment to the CBRR intervention relative to usual care on

comparison (for example, SUB versus UC) for families who

average, and no evidence here indicates that assignment to the

have low and high levels of challenge. Low challenge is set at

CBRR intervention worked better for families who have a lower

the 20th percentile of challenge, and high challenge is set at

number of challenges.

the 80th percentile. The 20th percentile is one challenge (out

of nine challenges; that is, more than 20 percent of families

8.2.3. PBTH Versus UC

reported one or fewer challenges). The 80th percentile is four

As the intervention that is designed to address psychosocial

challenges. The asterisks in the exhibit reflect whether the

challenges directly through the provision of intensive services

variation in impact by the level of each psychosocial challenge

in a supervised facility, PBTH would be expected to have great-

is statistically significant.

er impact relative to usual care for families who have higher

For example, the first row considers impacts for the confir-

levels of challenges. Addressing these challenges is thought to

matory outcome of at least 1 night homeless or doubled up in

lay the foundation for success in housing and self-sufficiency.

the past 6 months or at least 1 night in shelter in the past 12

None of the 18 tests of differential impact, however, reached

months. In the first pair of columns, the impact of assignment

statistical significance. Previous chapters found little impact of

to the SUB intervention compared with usual care is estimated

assignment to the PBTH intervention relative to usual care, and

to be a reduction of 23.8 percentage points in this outcome for

no evidence indicates that impact differs depending on the level

families who have a low challenge level and 17.1 percentage

of psychosocial challenges families face.

points for families who have a high challenge level. The average

effect is very large and significant, as reported previously in

8.2.4. SUB Versus CBRR

Chapter 3, and the impact remains very large and significant

The SUB intervention, offering priority access to a permanent

for those with low and high challenges. The difference between

housing subsidy, would be expected to have greater impact

these two impacts, however, is not significant.

on housing stability than assignment to the CBRR intervention

for all families, but especially for those whose psychosocial

8.2.1. SUB Versus UC

challenges threaten that stability. To the extent that housing is

Assignment to the SUB intervention would be expected to

a platform for families to deal with other challenges they face,

have greater impact relative to usual care for families who have

the radiating impact of assignment to the SUB intervention

higher numbers of psychosocial challenges, even if the SUB

observed in earlier chapters may be especially marked for fam-

programs do not address these challenges directly. Families

ilies who have more challenges. With only 1 of 18 statistically

who have fewer challenges may be able to manage without

significant findings—slightly fewer than would be expected

long-term help; however, no evidence of differential impact

by chance alone—the study team concludes that the impacts

was indicated. None of the 18 tests for differences in impact

families experience from assignment to the SUB intervention

magnitude between low and high challenge levels reached sta-

compared with assignment to the CBRR intervention do not

tistical significance. The substantial average differences between

differ for families who have different levels of challenges.

assignment to the SUB intervention and usual care shown

in previous chapters held across both levels of psychosocial

challenges.

119 r = .23, p < .001 for the sample interviewed at 37 months.

3-Year Impacts of Housing and Services Interventions for Homeless Families

101

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

Exhibit 8-3. Impacts at 37 Months Moderated by Baseline Psychosocial Challenges Index Outcome

SUB vs. UC

CBRR vs. UC

PBTH vs. UC

SUB vs. CBRR

SUB vs. PBTH

CBRR vs. PBTH

Impact at Low Versus High

Challenge

Low

High

Low

High

Low High

Low

High

Low

High

Low

High

Housing stability

At least 1 night homeless or

– 23.8

– 17.1

3.1

– 0.6

5.7

– 4.2

– 20.7

– 21.5

– 29.2

– 12.8**

0.6

– 4.4

doubled up in past 6 months

or in shelter in past 12 months

(%) [confirmatory]

At least 1 night homeless or

– 19.6

– 16.4

3.2

– 1.9

6.2

– 5.3

– 15.2

– 16.9

– 28.7

– 8.8***

1.7

– 5.5

doubled up in past 6 months

(%)

Number of places lived in past 6 – 0.21

– 0.29

0.06 – 0.04

0.00 – 0.05

– 0.16

– 0.13

– 0.40

– 0.14

0.14

0.04

months

Any stay in emergency shelter in – 15.0

– 13.6

– 2.9

– 0.4

– 6.5

– 3.6

– 11.9

– 15.2

– 6.0

– 4.4

2.2

– 0.3

months 21 to 32 after random

assignment (%)

Family preservation

Family has had at least one child – 4.2

– 2.4

0.2

– 0.8

1.6

1.7

– 2.9

– 1.6

– 7.5

– 8.7

0.1

– 8.8

separated in past 6 monthsa (%)

Spouse/partner separated in

10.3

17.4

9.1

7.9

6.7

18.3

– 13.5

7.7

– 8.9

25.3**

5.2

– 8.0

past 6 months, of those with

spouse/partner present at

RAb (%)

Family has no child reunified,

– 5.9

– 9.9

– 5.6

– 9.9

2.4

– 1.7

– 18.9

– 11.0

– 18.3

– 21.7

11.5

– 2.1

of those families with at least

one child absent at RAc (%)

[limited base] (%)

Adult well-being

Health in past 30 days was poor

0.7

8.1

– 2.2

8.2**

0.5

– 1.7

– 1.9

3.7

– 5.3

2.8

– 4.0

10.2*

or fair (%)

Psychological distressd

– 0.92

– 0.41

– 0.36 – 0.23

0.12 – 0.52

– 0.14

0.67

– 0.95

0.33

– 1.46 – 0.07

Alcohol dependence or drug

– 1.6

– 2.7

– 1.0

– 3.7

2.1

4.2

1.3

3.1

– 2.4

1.3

– 5.9 – 11.8

abusee (%)

Experienced intimate partner

– 3.7

– 4.7

– 3.2

1.3

– 0.6

– 2.0

2.2

– 8.6**

– 4.9

5.2*

– 3.7

7.4*

violence in past 6 months (%)

Child well-being

Number of schools attended

– 0.18

– 0.12

0.04

0.01

– 0.03

0.17

– 0.23

– 0.32

– 0.17

– 0.26

0.04

0.05

since RAf

School absences in past month – 0.14

0.01

– 0.18 – 0.12

– 0.07 – 0.36

0.15

0.00

– 0.21

– 0.10

– 0.29 – 0.05

(ages 5 to 17 years)g

Poor or fair health (%)

3.0

0.8

1.4

– 2.7

– 0.9

0.7

1.0

0.8

– 1.2

– 3.4

0.5

– 4.8

Behavior problemsh

– 0.24

– 0.18

– 0.14 – 0.15

0.01 – 0.15

0.02

0.01

– 0.22

– 0.14

– 0.24 – 0.06

Self-sufficiency

Work for pay in week before the

– 2.2

1.9

– 1.0

2.2

1.6

3.8

1.7

– 2.3

– 8.1

0.1

– 9.2

– 1.0

survey (%)

Total family income ($)

– 453

– 1,367

– 982

27

– 37

153

– 1,095

– 1,139

– 953

– 2,534

– 1,922

– 838

Household is food secure (%)

10.9

6.4

6.4

0.9

5.5

– 1.5

– 2.4

6.9

13.0

8.7

5.8

19.0

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

RA = random assignment.

*/**/*** Impact magnitude varies significantly with level of [psychological or housing barriers] challenge at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Measures the percentage of families in which a child who was with the family at baseline was separated from the family in the 6 months before the 37-month survey.

b Measures the percentage of families in which a spouse or partner who was with the family at baseline was separated from the family in the 6 months before the 37-month survey.

c Percentage of families in which at least one child was separated from the family at baseline where no child was reunited with the family at the time of the 37-month survey.

d Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

e Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

f Number of schools outcome is topcoded at 4 or more schools.

g Absences outcome is defined as 0 = no absences in past month; 1 = one to two absences; 2 = three to five absences; 3 = six or more absences. This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

h Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

Notes: The low estimate is calculated at the 20th percentile of the moderator in the full sample and the high estimate is calculated at the 80th percentile of the moderator. Impact mean difference estimates are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B for outcome definitions.

Source: Family Options Study 37-Month Followup Survey

3-Year Impacts of Housing and Services Interventions for Homeless Families

102

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

8.2.5. SUB Versus PBTH

fewer challenges but reduced them by 25 percentage points for

families who have more challenges. Separation outcomes are

Because the PBTH intervention offers priority access to

correlated with baseline intimate partner violence overall, but

programs that address psychosocial challenges more directly

especially for families assigned to the SUB group.120 It is plausi-

than the SUB intervention does, proponents of project-based

ble that the subsidy intervention enabled some family heads to

transitional housing would expect it to be especially beneficial

leave abusive relationships who would not have done so with

for families who have higher numbers of these challenges.

assignment to the PBTH group. Because baseline interpersonal

Some evidence supports this proposition. Of the 18 tests for

violence is the most prevalent of the psychosocial challenges,

differential impacts by psychosocial challenge level, 4 reached

these women are more likely to be in the high challenge group.

statistical significance, and the direction of differential effects

is consistent: the benefit from assignment to the SUB group

A caveat here is that none of the differential effects for the SUB-

relative to assignment to the PBTH group is smaller for families

versus-PBTH comparison were found at 20 months. Further,

who have more challenges than it is for families who have

the theory behind project-based transitional housing would

fewer challenges. This degree of contrast suggests more than

lead one to expect this same pattern of findings for the PBTH-

chance variability in the data and signals that further attention

versus-UC comparison and the CBRR-versus-PBTH comparison

to the 4 specific findings involved is warranted.

as for the SUB-versus-PBTH comparison. None of the tests

for differential impacts in the housing or family preservation

In the housing stability domain, assignment to the SUB group

domain reach significance for these comparisons, and the

rather than to the PBTH group remained beneficial overall, but

nonsignificant results are in different directions. The lack of

the benefits were stronger for families who have fewer challenges.

replication across time points and comparisons suggests that

Assignment to the SUB group reduced the confirmatory outcome

findings may be due to chance. The finding that assignment to

(at least 1 night homeless or doubled up in the past 6 months

the PBTH intervention is more likely to reduce intimate partner

or in shelter in the past 12 months) by 13 percentage points for

violence for families facing more challenges is replicated in the

families who have more psychosocial challenges and by 29 per-

CBRR-versus-PBTH comparison (at 37 months only), however.

centage points for families who have fewer challenges. Results

All these factors led the study team to considerable caution

for the associated outcome of at least 1 night homeless or dou-

about these results.

bled up in the past 6 months were parallel: assignment to the

SUB group rather than to the PBTH group led to a reduction of

If the results in this comparison are accepted as reflecting true

9 percentage points for families facing more challenges and 29

differential impact rather than chance findings, what should the

percentage points for families who face fewer challenges.

conclusion be? Considering the set of 18 outcomes as a whole,

even among families facing high levels of challenges, one

In the adult well-being domain, families who have more challenges

would prefer the SUB intervention to the PBTH intervention

benefited more from assignment to the PBTH intervention than

regarding all the housing stability outcomes and regarding

from assignment to the SUB intervention, but families who

other dimensions such as child separations, number of schools

have fewer challenges experienced the opposite, benefiting

children attended, and food security for which the SUB

more from assignment to the SUB intervention than from

intervention had uniformly more positive impacts than the

assignment to the PBTH intervention. Assignment to the PBTH

PBTH intervention across levels of psychosocial challenges.

intervention rather than to the SUB intervention reduced

Regarding intimate partner violence, however, assignment to

experiences of intimate partner violence modestly (5 percentage

the PBTH intervention was superior to assignment to the SUB

points) for families who have high levels of challenges but

intervention for those families who have more psychosocial

increased them by about the same amount for families who

challenges. Assignment to the PBTH intervention also led to

have low levels.

fewer separations of spouses and partners for families who have

The finding of differential impact in the family preservation

more challenges.

domain is more difficult to interpret. Assignment to the PBTH

intervention rather than to the SUB intervention increased

8.2.6. CBRR Versus PBTH

recent separations of spouses and partners among the subset of

As the intervention offering intensive social services in a

families who had a partner with the family in shelter at the out-

supervised facility, PBTH would be expected to have greater

set of the study by 9 percentage points for families who have

impact relative to the CBRR intervention for families who have

120 r = .42 between interpersonal violence at baseline and separation at 37 months for SUB families.

3-Year Impacts of Housing and Services Interventions for Homeless Families

103

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

more psychosocial challenges. Of 18 results here, 2 reached

would be expected to work best compared with usual care fam-

statistical significance. Both are in the same direction of less

ilies who have lower barriers to housing. Temporary subsidies

favorable impacts for families in the high challenge group who

may not be sufficient to overcome greater housing barriers. Of

were assigned to the CBRR group rather than to the PBTH

18 tests, 2 were statistically significant, both in the predicted

group. One of these effects is a greater reduction in intimate

direction, but because 2 does not exceed the number of statis-

partner violence for families who have more challenges who

tically significant findings expected by chance alone, the study

are assigned to the PBTH group rather than to the CBRR group.

team cannot conclude that the impacts of assignment to the

Because the number of effects is not more than would be

CBRR intervention versus usual care varies with the number of

expected by chance alone, however, the study team concludes

housing barriers.

that the size of impacts families experience from assignment to

the CBRR intervention compared with assignment to the PBTH

8.3.3. PBTH Versus UC

intervention does not appear to vary by the level of psychoso-

The services that PBTH programs provide may directly address

cial challenges.

housing barriers by helping with job training or restoration of

credit. They more generally address issues such as substance

8.3. Differential Impacts Depending

abuse or psychological distress that may interfere with em-

on Housing Barriers

ployment or create problems with landlords. Thus, assignment

to the PBTH intervention would be expected to have greater

Exhibit 8-4 illustrates possible differential impacts of interven-

impact relative to usual care for families who have more

tions based on the number of housing barriers families reported

housing barriers. Only 2 of the 18 tests conducted reached

at study entry. It shows the estimated size of the impact for a

significance—not more than would be expected by chance

given policy contrast (for example, the SUB intervention versus

alone if no true impact variation occurred—and they are in

usual care) for families who have low and high barriers, again

opposite directions. Previous chapters found little impact of

using cutoffs at the 20th and 80th percentiles of barriers. For

assignment to the PBTH intervention relative to usual care, and

housing barriers, the 20th percentile is 4.0 barriers and the

no evidence indicates that impact differs depending on the level

80th percentile 8.6 barriers. As in the previous section on psy-

of housing barriers families face.

chosocial challenges, the asterisks reflect whether the variation

in impact by level of housing barriers is statistically significant.

8.3.4. SUB Versus CBRR

Also, as in that section, the study team considers both the

The permanent and often deeper housing subsidies that the

number of statistically significant findings and their patterns in

SUB programs provide would be expected to have a greater

interpreting whether results show real evidence.

impact than the temporary and shallower subsidies of the

CBRR programs for families who have more housing barriers.

8.3.1. SUB Versus UC

Of the 18 tests, 3 reached statistical significance—less than

By providing priority access to permanent housing subsidies,

the threshold of 4 for interpretation—and the direction of the

assignment to the SUB intervention would be expected to have

effects was not consistent. The study team therefore cannot

greater impact relative to usual care for families who have

conclude that the size of impacts families experience from

higher housing barriers, such as lack of income to pay rent or

assignment to the SUB intervention compared with assignment

security deposits, lack of employment, or poor credit. Such

to the CBRR intervention differs for families who have different

families may not otherwise be able to access stable housing.

numbers of housing barriers.

No evidence, however, supported this hypothesis. Of the 18

comparisons, 2 were statistically significant—not more than

8.3.5. SUB Versus PBTH

would be expected by chance alone—and they were in oppo-

Because housing subsidies overcome many barriers to housing,

site directions. The substantial average differences between

proponents of the permanent housing subsidy would expect it

assignment to the SUB intervention and usual care shown in

to be especially beneficial for families who have more of these

previous chapters hold across families facing different numbers

barriers. Proponents of project-based transitional housing make

of housing barriers.

the opposite prediction. The 1 statistically significant result out

of 18—fewer than would be expected by chance alone—fails to

8.3.2. CBRR Versus UC

confirm either proposition and is best interpreted as chance.

The CBRR programs offered to families assigned to the CBRR

intervention, as the least intensive of the active interventions,

3-Year Impacts of Housing and Services Interventions for Homeless Families

104

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

Exhibit 8-4. Impacts at 37 Months Moderated by Baseline Housing Barriers Index

Outcome

SUB vs. UC

CBRR vs. UC

PBTH vs. UC

SUB vs. CBRR

SUB vs. PBTH

CBRR vs. PBTH

Impact at Low Versus

High Challenge

Low

High

Low

High

Low

High

Low

High

Low

High

Low

High

Housing stability

At least 1 night homeless or

– 19.3 – 23.1

4.8

– 1.8

– 2.5

5.1

– 19.4 – 21.9

– 22.8

– 23.5

– 0.4

– 2.3

doubled up in past 6 months or

in shelter in past 12 months (%)

[confirmatory]

At least 1 night homeless or doubled – 16.3 – 20.4

4.7

– 2.7

– 4.7

6.8

– 15.7 – 15.4

– 19.1

– 23.1

4.1

– 6.9

up in past 6 months (%)

Number of places lived in past 6

– 0.17 – 0.32

0.10 – 0.07

– 0.06

0.01

– 0.16 – 0.12

– 0.39

– 0.21

0.13

0.05

months

Any stay in emergency shelter in

– 12.0 – 16.7

– 0.2

– 3.5

– 2.8

– 7.5

– 11.1 – 15.4

– 7.6

– 2.9

– 2.2

5.5

months 21 to 32 after random

assignment (%)

Family preservation

Family has had at least one child

– 6.6

– 1.3

– 1.5

0.8

– 5.5

7.9**

– 2.8

– 2.0

– 13.8

– 3.5

– 0.7

– 6.8

separated in past 6 monthsa (%)

Spouse/partner separated in past

7.6

19.0

7.8

5.1

6.9

13.7

– 10.1

– 4.7

– 16.1

16.7*

2.6

– 1.2

6 months, of those with spouse/

partner present at RAb (%)

Family has no child reunified, of

– 7.3

– 9.8

– 7.3

– 7.2

– 4.4

1.6

– 12.9 – 19.7

– 22.0

– 19.3

4.6

1.8

those families with at least one

child absent at RAc (%) [limited

base] (%)

Adult well-being

Health in past 30 days was poor or

2.1

4.9

– 3.7

7.5*

– 4.5

2.8

4.8

– 3.9

– 0.2

– 4.8

– 0.4

2.1

fair (%)

Psychological distressd

– 1.58

0.02**

– 1.13

0.36**

– 0.29 – 0.15

0.59 – 0.16

– 1.22

0.09

– 0.88

– 1.61

Alcohol dependence or drug abusee

– 2.6

– 2.1

0.5

– 5.2

3.1

2.1

– 2.2

5.2

– 0.2

– 2.0

– 8.1

– 8.9

(%)

Experienced intimate partner

– 6.0

– 2.8

1.3

– 4.9

0.3

– 2.8

– 6.5

2.0**

– 4.0

1.5

– 1.8

2.1

violence in past 6 months (%)

Child well-being

Number of schools attended since

0.05 – 0.32***

0.06 – 0.01

0.04

0.04

– 0.04 – 0.44*** – 0.13

– 0.29

0.20

– 0.09

RAf

School absences in past month

0.08 – 0.15

– 0.22 – 0.14

– 0.07 – 0.25

0.11

0.06

– 0.15

– 0.15

– 0.54

0.08**

(ages 5 to 17 years)g

Poor or fair health (%)

0.0

3.4

0.1

– 1.1

1.5

– 1.6

0.4

0.7

– 5.6

– 0.1

– 5.2

0.3*

Behavior problemsh

– 0.25 – 0.22

– 0.24 – 0.15

– 0.05 – 0.07

0.11 – 0.03

– 0.27

– 0.19

– 0.20

– 0.24

Self-sufficiency

Work for pay in week before the

1.0

– 1.8

2.6

– 0.8

1.6

3.8

– 2.9

1.4

– 8.1

– 1.2

– 4.2

– 5.6

survey (%)

Total family income ($)

– 2,039

196

– 1,400

368

– 1,795 1,841* – 2,695

– 86*

– 952 – 2,053

– 1,381 – 1,663

Household is food secure (%)

10.6

8.3

6.0

2.9

1.1

3.5

2.7

0.6

20.0

6.1

10.1

12.8

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

RA = random assignment.

*/**/*** Impact magnitude varies significantly with level of [psychological or housing barriers] challenge at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Measures the percentage of families in which a child who was with the family at baseline was separated from the family in the 6 months before the 37-month survey.

b Measures the percentage of families in which a spouse or partner who was with the family at baseline was separated from the family in the 6 months before the 37-month survey.

c Percentage of families in which at least one child was separated from the family at baseline where no child was reunited with the family at the time of the 37-month survey.

d Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress.

e Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

f Number of schools outcome is topcoded at 4 or more schools.

g Absences outcome is defined as 0 = no absences in past month; 1 = one to two absences; 2 = three to five absences; 3 = six or more absences. This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

h Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

Notes: The low estimate is calculated at the 20th percentile of the moderator in the full sample and the high estimate is calculated at the 80th percentile of the moderator. Impact mean difference estimates are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B for outcome definitions.

Source: Family Options Study 37-Month Followup Survey

3-Year Impacts of Housing and Services Interventions for Homeless Families

105

FAMILY OPTIONS STUDY

Chapter 8. Do Certain Interventions Work Better When Offered to Families Who Face Greater Difficulties?

8.3.6. CBRR Versus PBTH

families who have more psychosocial challenges than it is for

families who have fewer challenges. In particular, the ability of

As the intervention offering more intensive services that address

assignment to the SUB intervention to produce greater long-run

housing barriers directly and indirectly, PBTH would be ex-

housing stability than assignment to the PBTH intervention

pected to have greater impact relative to the CBRR intervention

was less pronounced (though still present) for families who

for families who have higher housing barriers. For families who

have a high level of psychosocial challenges at baseline than for

have fewer barriers, temporary subsidies may suffice. Of 18

families who have a low level of challenges. Consistent with the

tests here, 2 reach statistical significance, both in the predicted

theory behind project-based transitional housing, assignment

direction, but this is not more than would be expected by

to the PBTH intervention actually did more than assignment

chance alone. Thus, the study team cannot conclude that the

to the SUB intervention to reduce separations from spouses

size of the impacts families experience from assignment to the

and partners and intimate partner violence at 37-months for

CBRR intervention compared with assignment to the PBTH

families who have higher levels of psychosocial challenges.

intervention differ for families who have different numbers of

(An alternate interpretation is that assignment to the SUB

housing barriers.

intervention enabled respondents to leave abusive partners.)

Still, across the full range of outcomes examined at 37 months,

8.4. Summary

families who have high levels of psychosocial challenge at

baseline experienced many benefits of assignment to the SUB

On average, the families in this study reported high numbers of

intervention compared with assignment to the PBTH inter-

psychosocial challenges and even higher numbers of barriers to

vention regarding additional housing stability outcomes, child

housing at the time they entered the study after having spent at

separations, children’s school mobility and behavior problems,

least 7 days in an emergency shelter. At the same time, families

and household food security, where differential impacts by

varied in the number of challenges and barriers they reported.

challenge level did not emerge.

This chapter examined whether any of the interventions works

comparatively better for families depending on the difficulties

The study team cannot completely rule out the possibility

they face.

of further differential effects related to families’ difficulties at

baseline—doing so would require larger sample sizes than

With one exception, the study team found no statistically

are available in the study. At this point, however—with the

convincing evidence that any of the interventions works

exception of a small subset of outcomes in the SUB-versus-

comparatively better or worse for families who face more

PBTH comparison when considering families who have high

psychosocial challenges or housing barriers than for families

psychosocial challenge—the main results in previous chapters

who face fewer difficulties. The exception was that, for certain

about impacts across all families provide the study’s clearest

outcomes, the benefit from assignment to the SUB intervention

guidance for policy and practice.

relative to assignment to the PBTH intervention is smaller for

3-Year Impacts of Housing and Services Interventions for Homeless Families

106

FAMILY OPTIONS STUDY

CHAPTER 9.

INTERVENTION COSTS

This chapter documents the costs of providing the after random assignment, the Short-Term Impacts report (Gubits housing or shelter and supportive services in the pro-et al., 2015) also found that total program use for families

grams associated with the interventions—permanent

assigned to the SUB intervention cost about the same as total

housing subsidies (SUB), community-based rapid re-housing

program use for families assigned to usual care and slightly

(CBRR), project-based transitional housing (PBTH), and usual

more than for families assigned to the CBRR intervention. Costs

care (UC)—in the Family Options Study and the total costs

of program use for SUB families were clearly less than for fam-

incurred by families in each pairwise comparison. The goal of

ilies assigned to the PBTH group. The nearly equivalent cost of

the 3-year cost analysis was to determine the relative costs of

assignment to the SUB group compared with assignment to the

studied interventions taking into account all programs used in

UC group in the first 20 months was driven by both decreased

the 3 years since random assignment.

time in emergency shelter and decreased use of relatively more

costly PBTH programs for families assigned to the SUB group.

For decisionmakers who design and implement policy to

The costs of total program use for families assigned to the SUB

address homelessness among families, information about the

and CBRR interventions were similar because the greater use

relative costs of the active interventions and usual care in this

of SUB programs by families assigned to the SUB intervention

study is a critical complement to findings about their relative

was offset by the greater use of transitional housing, emergency

impacts. To assess the relative costs of the interventions, it

shelter, and other programs by CBRR families.

is crucial to understand both the cost per month for each

program to which families were given priority access and the

The study team anticipated that costs for families assigned to

overall cost to all providers of shelter and housing assistance of

the SUB intervention, which provides priority access to housing

giving families priority access to a particular type of program.

assistance that is not time limited, may begin to outpace costs

for families assigned to interventions that offered priority

This chapter begins by reviewing the concepts and methods

access to time-limited assistance. In Section 9.3, the study team

used to analyze and describe program costs.121 Then, Section

presents a finding that this is indeed the case. Average costs

9.2 presents estimates of per-family monthly costs for each

of all program use during the entire 3-year followup period

program type. To estimate the cost of the use of programs of all

are now about $4,000 more for families assigned to the SUB

types, the study team summed monthly cost estimates over ob-

intervention than for families assigned to each of the other in-

served program use in the 37-month followup period. Section

terventions. Assignment to the CBRR intervention now results

9.3 presents these estimates of the cost of all program use for

in average costs during the followup period that are about

families randomly assigned to groups that offered priority ac-

$4,000 less than assignment to usual care (and also to the SUB

cess to each program type, or assignment to usual care. Finally,

intervention) and about $10,000 less than assignment to the

Section 9.4 reports the monthly cost of observed program use

PBTH intervention.

at the time of the 37-month followup survey.

The Family Options Study: Short-Term Impacts of Housing and

9.1. Introduction and Review of

Services Interventions for Homeless Families (hereafter, the

Methodology

Short-Term Impacts report; Gubits et al., 2015) reported that

the per-family monthly cost of emergency shelter varied

The objective of the Family Options Study is to provide

considerably by program type. Per-family monthly costs were

evidence to support decisions of policymakers, planners,

highest for emergency shelter programs and lowest for rapid

and practitioners who address homelessness among families.

re-housing programs (the programs to which families assigned

Although much of the study is focused on estimating relative

to the CBRR intervention were referred). In the first 20 months

effects of three active interventions and usual care, such

121 Chapter 12 of the Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families (Gubits et al., 2015) provides additional details about the cost analysis.

3-Year Impacts of Housing and Services Interventions for Homeless Families

107

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

estimates are only one input into decisions about homelessness

use during the 37-month followup period, and (3) monthly

policy. Because of differences in the type of housing or shelter

cost of all program use at the time of the 37-month followup

provided, the duration of assistance, and the range and

survey.

intensity of supportive services offered, the programs associated

1. Per-family monthly program cost. Per-family monthly

with each active intervention vary in cost. The extent to which

program cost is the cost of all resources used to provide

families who are provided priority access to a particular type of

shelter or housing and services to a family during the course

program use that assistance or find their way to a different type

of a month because they are receiving assistance through a

of assistance (or no assistance of any type) is also fundamental

particular SUB, CBRR, PBTH, or emergency shelter program.

to establishing the relative cost of the interventions studied.

The study team developed these costs by cataloging and

This chapter reports on the costs of providing the housing and

valuing the housing or shelter dimension of each program

services in the programs associated with the three active inter-

(capital and operating costs or rent) and also the services

ventions: SUB, CBRR, and PBTH. First, the chapter presents

dimension (the personnel, space, and materials used to

the unit costs of the programs to which families were given

provide services) in each of 81 study programs at the time

priority access and the costs of emergency shelters from which

when study families initially received assistance.

families were enrolled in the study. These costs represent the

2. Cost of all program use during the followup period. The

ongoing costs of providing assistance to a family at SUB, CBRR,

second cost measure accounts for costs of all programs

PBTH, and emergency shelter programs that participated in the

that families used during the 37-month followup period.

study . The chapter also reports the cost of all use of shelter and

Families given priority access to a particular type of program

housing assistance programs, regardless of how families found

through random assignment nonetheless used multiple

their way to those programs, imputed using the unit costs of

programs—both the program type to which they were given

the study programs. These costs estimate the cost of the SUB,

priority access and other program types. This cost includes

CBRR, PBTH, and UC interventions in which families received

the expense of providing housing or shelter and associated

priority access to a particular program or usual care and then

assistance (services) to study families during the time

would proceed to access a variety of programs and program

between random assignment and the 37-month followup

types. For this cost concept, the study team imputes costs of

survey. This cost concept represents the cost to the homeless

all types of shelter or housing assistance reported in the study’s

services and housing assistance system of the SUB, CBRR,

Program Usage Data. That cost concept includes the costs of

and PBTH interventions—priority access to a particular

emergency shelter programs to provide information on the

program type—and the cost of program use for usual care.

cost of continued stays in emergency shelter following families’

enrollment in the study and also the costs of any subsequent

3. Monthly cost of all program use at the 37- month followup

returns to shelter during the period between random assign-

survey. The monthly cost of all program use at the 37-month

ment and the 37-month followup survey.

followup survey considers the average per-family monthly

program cost of programs from which families were receiv-

Throughout this chapter, the study team refers to SUB, CBRR,

ing assistance at the time of the followup survey. Receiving

PBTH, and UC interventions when referencing the fact that

priority access to one program type may make it more or

families were randomly assigned to receive priority access

less likely that the family will use other housing or homeless

to a particular program type (or in the case of usual care, no

assistance programs in the medium and long terms. As a

priority access). When referencing the cost of the programs to

result, giving families priority access to a particular program

which families are given priority access or the shelter programs

type today can change the cost of assistance they receive

where families were enrolled in the study, the study team refers

months and years into the future. This chapter reports the

to SUB, CBRR, PBTH and emergency shelter programs.122

averages of this point-in-time cost calculated for each of the

This chapter presents analyses for three concepts of cost:

study’s six pairwise comparisons 37 months after random

(1) per- family monthly program cost, (2) cost of all program

assignment.

122 The monthly cost of the programs that make up the three active interventions represents the cost of actually providing assistance to families who take up programs associated with the assigned intervention. The monthly cost of emergency shelter programs is not the cost of providing usual care, because usual care includes whatever housing subsidies or supportive services families were able to obtain without receiving priority access to one of the active interventions. Because it was not feasible to determine the extent and costs of any assistance beyond what was provided by the emergency shelter program for per-family monthly program costs, the study team reports costs associated with emergency shelter only rather than all costs associated with the usual care. For the cost of all program use during the 37-month followup period and the cost of program use at the time of followup, the study team does estimate the cost of other program use for families assigned to the UC group.

3-Year Impacts of Housing and Services Interventions for Homeless Families

108

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

9.1.1. Cost Data-Collection and Analysis

Exhibit 9-1. Programs Included in the Estimates of

Methodology

Monthly Costs

This section provides a brief review of the cost data-collection

Program Type

Number of Programs Used

and analysis methodology. Appendix G provides greater detail.

To Produce Cost Estimates

Permanent subsidies offered to the SUB

10 sites

To calculate the costs of the SUB, CBRR, PBTH, and emergency

group

(administrative data)

shelter programs, the study team attempted to include all

Rapid re-housing programs offered to the

12

CBRR group

resources that are used to provide the housing or shelter and

Project-based transitional housing programs

24

the services that are part of the programs. Thus, the study

offered to the PBTH group

team included services provided by partners that are not in the

Emergency shelter programs

45

programs’ budget, when those services are an integral part of

CBRR = priority access to community-based rapid re-housing. PBTH = priority access

to project-based transitional housing. SUB = priority access to permanent housing

the program—for example, because participants in the program

subsidy.

have preferential access to the services. The cost concept also

Notes: For the 10 sites with the SUB intervention, average costs are calculated from

household-level administrative data by site for families who received services from

includes the monetary equivalent of in-kind donations of

the SUB providers. The SUB intervention was not available to families in Atlanta or

Baltimore.

services and materials and includes capital costs incurred for

housing and shelter.123

The study team collected cost data from more than one-half

housing or shelter and services to a typical family served by the

of the programs that initially agreed to provide the study’s

program. The primary source for cost data for rapid re-housing,

interventions. Estimates of per-family monthly costs for each

transitional housing, and emergency shelter programs was

program type are calculated from these data. These estimates

audited expense statements. Program budgets, staffing lists,

are then used to estimate costs for all shelter and housing assis-

partner commitment letters, and program staff estimates of

tance programs that study families ultimately used (including

labor and material costs of any services not reflected in expense

for programs that did not participate in the study). The study

statements supplemented these statements. Cost data were

team had two aims in selecting programs to include in the cost

collected for the program fiscal year that overlapped most

analysis. First, programs that served the most study families

closely with the time in which study families actually received

were selected so that cost estimates would be more likely to

assistance from the programs. Because this initial period of

reflect assistance study families actually received. Second, costs

assistance was largely completed at the time of the 20-month

of programs of each intervention type offered at each site were

period, the study team continues to use per-family monthly

included, so that cost estimates would reflect variation in the

program cost estimates based on this initial data period for the

housing or shelter and services provided across programs.

cost estimates calculated in this chapter. These estimates are the

Exhibit 9-1 shows the counts of programs used to develop the

same as those reported in the Short-Term Impacts report (Gubits

per-family monthly cost estimates. The programs selected for

et al., 2015) , with adjustments for inflation, where appropriate.

the cost analysis represent more than 85 percent of study fami-

The study team developed program costs for permanent

lies who accepted a study referral to CBRR and PBTH programs

housing subsidies using HUD Public and Indian Housing Infor-

and more than 90 percent of families assigned to the UC group

mation Center (PIC) and Tenant Rental Assistance Certification

from shelter programs. The issue of selecting programs did not

System data covering Housing Assistance Payments (HAP) costs

arise for the SUB intervention because administrative data were

for all SUB families who received permanent housing subsidies

available for all SUB programs in the study.

and Financial Data Schedule data for the administrative

For CBRR, PBTH, and emergency shelter programs, the study

costs of the Housing Choice Voucher (HCV) program. For

collected costs at the program level and normalized the costs by

costs of a permanent housing subsidy, as in the Short-Term

the number of families served by the program (as opposed to

Impacts report (Gubits et al., 2015) , the study team used cost

tying particular housing units or shelter beds and specific sup-

estimates from HUD administrative data systems for families

portive services to study families directly). These program-level

who were assigned to the SUB intervention and took up the

costs can be thought of as the average cost of providing

permanent housing subsidies offered to SUB families. For this

123 This approach to estimating costs is different from the approaches in previous studies that calculate the costs of homelessness. Many studies in recent decades sought to compare the cost of supportive housing for chronically homeless individuals or families with mainstream healthcare and public safety costs of managing this population in the absence of supportive housing. An introduction to and overview of this literature are provided in Culhane et al. (2007). By contrast, this analysis focuses on the costs incurred, not by other systems or services, but by the programs providing shelter or housing and related services to homeless families. A complete cost-benefit analysis, which is not part of the study design, would include estimates of cost offsets to other systems and of all costs of services that study participants may have received from providers that were not involved with the study, and it would also include an attempt to monetize the benefits associated with differences in impacts.

3-Year Impacts of Housing and Services Interventions for Homeless Families

109

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

report, monthly costs are recalculated for the entire 37-month

services, and consumer goods donated to program clients.

period.124 Those costs are composed of household-level monthly

The importance of these costs varies widely from program to

HAPs and site-level costs of administering the HCV program.

program. When present, they typically are part of the cost of

The database for the cost calculation for this report includes

supportive services provided by a program. In some cases,

data from more recent PIC records that cover the period be-

however, housing or shelter costs include the costs of labor,

tween the 20- and 37-month followup surveys and also data for

such as handyman services, or of facilities used regularly for

months 1 to 20 used in the Short-Term Impacts report (Gubits

program activities that were provided in kind. In other cases,

et al., 2015). Costs are estimated using the data for the SUB

accounting, legal, or administrative services were provided

families who are also included in the reports of impacts (400),

in kind or by partners. In each case, the study team appor-

a slightly smaller sample than was available at 20 months (454).

tioned the cost to the appropriate category.

The cost analysis considered costs in two broad categories:

The study team estimated the cost of all program use since

random assignment by multiplying the average site-level

1. Housing or shelter costs refer to the rental cost—either ob-

per-family monthly program cost for each program type by

served or estimated—of the space used to provide housing

the number of months of assistance of each respective type

or shelter and program services and to any maintenance or

provided to each family between random assignment and the

other facility operation costs (including durable items such

family’s followup survey, as observed in the Program Usage

as furnishings). The rental cost is net of any rent payments

Data. The imputed cost of all program use during the followup

made by the family.

period for each family is then the sum of these monthly costs

2. Supportive services costs refer to the cost of any services

times months of assistance for each program type.

other than housing or shelter provided as an integral part of

Program Usage Data measured the number of months each

the program, including case management and any cash or

family received seven types of homeless or housing assistance

in-kind assistance (for example, meals provided in emergency

programs: (1) subsidy, (2) rapid re-housing, (3) transitional

shelters).

housing, (4) emergency shelter, (5) permanent supportive housing,

Two other categories of costs are measured and included in

(6) public housing, and (7) project-based housing assistance

the calculation of housing or shelter costs and of services costs.

(project-based vouchers or Section 8 projects). (See Chapters 3

Additional detail is shown for these two types of costs because

through 6 for more information on program use, particularly the

they provide information on the typical structure of homeless

respective “Program Use Since Random Assignment” exhibits.)

assistance programs:

Translating the number of months of assistance received

1. Administrative and overhead costs include management

into the cost of all housing and services programs used since

salaries; legal, accounting, and other professional services;

random assignment requires a few assumptions. First, many

and program support costs, such as insurance premiums and

families accessed shelter or housing and related services from

agency and association fees. Administrative and overhead

programs not included in the cost data collection. The study

costs are divided among supportive services and housing or

team valued all stays at programs that matched a “type” from

shelter costs according to the cost types’ relative share of total

a study at the site-level average of the per-family monthly pro-

costs so that they are included in the two broad categories.

gram cost. So, assistance from any rapid re-housing program

was valued at the sites’ average per-family monthly cost for

2. In-kind and partner costs include any costs of housing or

CBRR programs, and assistance from any transitional housing

shelter and supportive services provided to families because

program was valued at the site’s average per-family monthly

they participate in a program. These costs are not provided

cost for PBTH programs. In a similar way, site-level costs for

by the program itself, and, as a result, are not included in

the SUB intervention are used as a proxy for the costs of hous-

program financial statements. Common examples include

ing choice vouchers used by all families in the study, including

onsite health or mental health providers funded by an

those not assigned to the SUB intervention.125

outside agency, community volunteers providing a variety of

124 This recalculation was done to allow for the possibility that changing family incomes or rental market conditions may influence average housing assistance payments for vouchers during the 3-year followup period. The updated monthly program cost estimate for SUB programs ultimately increased by only $10. Monthly assistance costs for other program types may also have changed during the followup period, suggesting that our estimates for subsidies and the SUB intervention are more precise than for other program types and interventions.

125 Because Atlanta and Baltimore did not offer the SUB intervention, per-family monthly program costs were not calculated in these sites for SUB programs. An additional site (Minneapolis) did not have adequate takeup of project-based transitional housing to support cost data collection. In these sites, the study team uses study-level average per-family monthly program costs as a proxy to allow cost of all program use since random assignment estimates to include the families who found their way to these program types without study assistance.

3-Year Impacts of Housing and Services Interventions for Homeless Families

110

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

Three of the program types that families used—permanent

for a relatively long duration. Finally, emergency shelter

supportive housing, public housing, and project-based housing

programs often offer extensive supportive services and a place

assistance (project-based vouchers or Section 8 projects)—are

to stay, typically for a very limited time.

not associated with study interventions and were not included

Monthly costs of serving a typical family vary considerably

in the cost data collection. The study team did not collect cost

by program type. Exhibit 9-2 presents the average per-family

data from these types of programs. Instead, under the assump-

monthly program cost for each type of program. SUB pro-

tion that they have similar program and cost structures, the

grams, on average, cost slightly less than $1,200 per family

study team used site-level average PBTH per-family monthly

per month and consist wholly of the cost of housing, because

program costs as a proxy for the monthly cost of permanent

this intervention provides no supportive services. Unlike the

supportive housing. SUB program costs are used as a proxy for

other program types, the study team used newly available data

the costs of public housing and project-based assistance.

to update the costs for SUB programs, as detailed previously.

These estimated costs of all program use for each family are

The monthly cost (adjusted to 2013 dollars) did not change

then averaged across random assignment outcomes within

substantially in the interval between the 20- and 37-month

each of the study’s six pairwise comparisons. This chapter

followup surveys. The updated estimate is $10 higher than the

reports averages of this amount (calculated using the same

original. In no study site did costs for SUB programs change by

nonresponse weights used in the impact analyses) for each of

more than 3 percent.126

the study’s pairwise comparisons. Thus, this estimate provides

CBRR programs have the lowest per-family monthly program

a total cost of housing or shelter and services that reflects the

cost among the program types, with a program average of

different mixes of program types used that resulted from a fam-

slightly less than $900. Housing costs, on average, make up

ily’s being randomly assigned to a group that provided priority

72 percent of CBRR program costs. PBTH programs have an

access to a particular program type.

average per-family monthly program cost of slightly more than

For monthly costs of all program use at the followup survey,

$2,700, with supportive services, on average, constituting 42

the study team made the same assumptions to impute the

percent of PBTH program costs.

monthly cost of observed program use for that month. The

average of the cost of each of the various types of programs

Exhibit 9-2. Average Per-Family Monthly Cost of Suppor-

families were observed or reported using in the month in which

tive Services and Housing or Shelter Across Program Types

they participated at the time of the followup survey was calcu-

lated as the site-level per-family monthly program cost for that

$6,000

intervention. Again, averages were calculated over the study

$4,819

impact sample. This cost concept does not consider the length

$5,000

of program use, but rather averages per-family monthly pro-

gram costs by the observed point-in-time average frequencies

$4,000

of program use by study families in each intervention group.

Observed emergency shelter stays, in particular, may last less

$3,000

$2,706

than a month.

$2,000

9.2. Per-Family Monthly Costs

$1,172

Average monthly cost per family

$1,000

$880

This section summarizes per-family monthly costs for SUB,

CBRR, PBTH, and emergency shelter programs. The interven-

$0

SUB

CBRR

PBTH

ES

tions examined in the Family Options Study were intended to

Supportive services

Housing or shelter

vary in both intensity and duration. SUB programs provide a

deep housing subsidy of indefinite duration but no supportive

CBRR = rapid re-housing programs offered to the CBRR group. ES = emergency

shelter. PBTH = transitional housing programs offered to the PBTH group. SUB =

services. CBRR programs provide a short-term, typically shal-

permanent housing subsidies offered to the SUB group.

lower housing subsidy with some supportive services. PBTH

Sources: Family Options Study cost data (CBRR, PBTH, and ES); U.S. Department of

Housing and Urban Development, Public and Indian Housing Information Center, Tenant

programs provide a place to stay and extensive services support

Rental Assistance Certification System, and Financial Data Schedule records (SUB)

126 In Honolulu, where the SUB intervention consisted largely of public housing placement, cost estimates are based on Fair Market Rents (FMRs) and increased by 7 percent.

3-Year Impacts of Housing and Services Interventions for Homeless Families

111

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

The results for the costs of all program use during the followup

Variation in CBRR and SUB costs across programs is driven

period reported in the next section are greatly influenced by

largely by housing costs. For the 12 CBRR programs in the cost

the finding that emergency shelter programs are much more

analysis, per-family monthly program cost ranges from slightly

costly than other program types. Emergency shelter programs

more than $550 to slightly less than $1,400. Across the 10

have the highest per-family monthly program cost for both

sites with the SUB intervention, average per-family monthly

supportive services and housing or shelter, which, on average,

program cost ranges from $777 to $2,250, largely reflecting

total slightly more than $4,800, four times the per-family

differences in the local cost of rental housing.

monthly cost of SUB programs, more than five times the cost of

Differences in the nature of the CBRR, PBTH, SUB, and emergency

CBRR programs, and almost twice the cost of PBTH programs.

shelter programs are reflected in the differences in average costs

Supportive services make up 63 percent of emergency shelter

across the programs. Exhibit 9-4 reports summary statistics

costs, the highest share among the four program types. The

for the four program types. CBRR and SUB programs provide

higher monthly cost of housing or shelter for emergency

assistance for private market housing. In both cases, but particularly

shelter programs reflects both program structure and the

for SUB programs, the cost of housing assistance is driven by

approach to classifying costs. Emergency shelters tend to have

local housing market conditions, as measured by Fair Market

higher per-family levels of facility staffing and expenditure for

Rent (FMR). Even though the CBRR programs provide some

maintenance and materials than do PBTH programs or than

supportive services in the form of housing placement and limited

what is reflected by the rents CBRR and SUB programs pay.

case management assistance, CBRR program costs are lower

In addition, housing or shelter costs include the capital cost

than SUB program costs on a per-month average basis, because

value of all physical space provided by the program, including

CBRR program assistance is sometimes a fixed amount that is

facilities such as classrooms, case management offices, kitchens,

less than typical HAPs provided by vouchers in the same site,

and dedicated childcare centers.127

and, in many cases, the subsidy declines the longer the family

Within each study intervention, the study team also found

receives CBRR program assistance.

substantial variation in the costs of the individual programs.

PBTH programs and emergency shelters are similar to each

Exhibit 9-3 shows this variation among per-family monthly

other and distinct from CBRR and SUB programs in that they

program costs for each program type. PBTH and emergency

provide a mix of housing or shelter and supportive services. In

shelter programs have substantial variation, driven largely by

fact, many PBTH and emergency shelter programs that study

variation in supportive services costs but also by variation in

team members visited for cost data collection are operated by

capital costs and administrative expenses. For the 24 PBTH

the same agency; in a number of instances, PBTH and emergency

programs in the cost analysis, per-family monthly program

shelter programs are distinguished only by length of stay, and

cost ranges from slightly more than $1,260 to slightly less than

families in both programs receive the same supportive services and

$6,300. Per-family monthly program cost for the 45 emergency

live in the same facility. Other emergency shelters are distinct

shelter programs ranges from $1,900 to slightly more than

in providing congregate shelter or shared rooms for sleeping,

$9,000.

Exhibit 9-3. Summary Statistics of Per-Family Monthly Program Cost by Program Type Per-Family Monthly Program Cost Summary Statistic

Program Type

Programs

Enrolled

Mean

Min 25th pct Median 75th pct Max

Families

($)

($)

($)

($)

($)

($)

Permanent subsidies offered to the SUB group

10

400

1,172

777

833

1,101

1,392

2,250

Rapid re-housing programs offered to the CBRR group

12

268

880

563

713

847

977

1,388

Project-based transitional housing programs offered to the PBTH group

24

107

2,706 1,261

1,738

2,352

3,535

6,292

Emergency shelter programs

45

667

4,819 1,888

3,907

4,352

5,786

9,170

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

Notes: CBRR, PBTH, and emergency shelter statistics are calculated from program-level cost estimates, weighted by the number of study families who enrolled in the program; SUB statistics are calculated from household-level data. For CBRR, PBTH, and emergency shelter, the number of families is study families who, based on enrollment verification data, enrolled in the programs used to estimate monthly costs; for SUB, the number of families is study families assigned to SUB who are 37-month followup survey respondents and who also have administrative records used to calculate SUB costs.

Sources: Family Options Study cost data (CBRR, PBTH, and ES); U.S. Department of Housing and Urban Development, Public and Indian Housing Information Center, Tenant Rental Assistance Certification System, and Financial Data Schedule records (SUB)

127 This study’s finding of higher monthly costs for family shelter programs than for other homeless assistance program types is consistent with previous estimates reported in HUD’s Costs Associated With First-Time Homelessness for Families and Individuals (Spellman et al., 2010), which found emergency shelter monthly costs for families were higher than transitional housing costs and higher than the local Fair Market Rent (FMR) in three of four cities. By contrast, shelters serving individuals had costs that, on average, were equal to or substantially lower than transitional housing costs and the FMR.

3-Year Impacts of Housing and Services Interventions for Homeless Families

112

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

Exhibit 9-4. Comparison of Cost Summary Statistics Across Program Types

SUB

CBRR

PBTH

Emergency

Programs

Programs

Programs

Shelter

Programs

Housing or shelter shares (%)

100

72

58

38

Supportive services share (%)

0

28

42

62

Partner and in-kind share (included in shelter or supportive services cost as relevant) (%) 0

Two sites: 1, 3

8

15

Ten sites: 0

Administrative and overhead cost share (included in both housing and supportive services costs) (%) 9

11

14

16

Per-family monthly program cost ($)

1,172

880

2,706

4,819

CBRR = community-based rapid re-housing. PBTH = project-based transitional housing. SUB = permanent housing subsidy.

Notes: Housing and supportive services shares add to 100. Partner and in-kind share and administrative share are included in housing and supportive services.

Sources: Family Options Study cost data (CBRR, PBTH, and ES); U.S. Department of Housing and Urban Development, Public and Indian Housing Information Center, Tenant Rental Assistance Certification System, and Financial Data Schedule records (SUB)

whereas PBTH (and SUB and CBRR) programs largely provide

The study team examined these costs of all program use associated

families with private units. Partner and in-kind resources

with the combination of assistance that the families received

represent a greater share of costs, on average, for emergency

for each of the six pairwise comparisons (see Exhibit 1-3 in

shelter programs than for PBTH programs. In general, the study

Chapter 1 for an overview of the pairwise comparisons):

team finds that PBTH programs relied more than did emer-

• SUB versus UC.

gency shelter programs on partner organizations to provide

• CBRR versus UC.

professional services such as counseling or mentoring, whereas

emergency shelter programs were more likely to use volunteer

• PBTH versus UC.

and in-kind resources.

• SUB versus CBRR.

• SUB versus PBTH.

9.3. Cost of All Program Use During

• CBRR versus PBTH.

the Followup Period by Families

As reported in Chapters 3 through 6, different sets of families

Assigned to Each Intervention

took part in each of the study’s pairwise comparisons. Exhibit 9-5

Having presented costs per month in a given program type

presents the average cost of all program use during the follow-

in Section 9.2, this section presents estimates of the total cost

up period for the families in each comparison. Each bar is made up

during the 37-month followup period after being randomly

of segments that reflect the cost of the average use of different

assigned to an intervention that provided priority access to a

program types during the followup period. Each segment is the

particular program type. This total cost includes the cost of the

average cost of observed use of the program type by families

program to which priority access was given, if the family used

assigned to the intervention within the pairwise comparison.

that program, and to other programs of various types that fami-

Looking across all the pairwise comparisons, whenever families

lies accessed on their own. These estimates can be thought of as

are assigned to the SUB or PBTH interventions, costs of SUB or

the costs of achieving the relative impacts of the interventions

PBTH program use represent the highest share of the cost of all

reported in Chapters 3 through 6. Receiving priority access to

program use. Substantial emergency shelter costs remain even

a particular type of program both increased the rate at which

in these instances, however, and costs of emergency shelter

families used that program and affected the rate at which fami-

use represent the highest cost share whenever families are

lies used other types of shelter and housing assistance programs

not assigned to the SUB or PBTH interventions. This finding

during the followup period.

illustrates that, when families have been in shelter for 7 or

These estimates use the per-family monthly program costs,

more days, substantial shelter costs are associated with assisting

together with the observed patterns of program usage reported

all families up until the time they leave the emergency shelter

in Chapters 3 through 6, to construct estimates of total costs of

either to use a program to which they were given priority

the mix of homeless or housing assistance programs that served

access or to go somewhere else. Differences also exist across

study families assigned to each of the interventions in the

interventions in the incidence of return to shelter during the

period between random assignment and the followup survey.

followup period, the second factor determining the amount of

shelter costs within the cost of all program use.

3-Year Impacts of Housing and Services Interventions for Homeless Families

113

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

Exhibit 9-5. Cost of Program Use Since RA for Each Intervention Contrast

Program type

Other

SUB

CBRR

PBTH

ES

Panel A

SUB vs. UC

CBRR vs. UC

PBTH vs. UC

$50,000

$45,902

$45,000

$42,134

$42,167

$40,130

$40,000

$38,722

$38,144

$35,000

$30,000

$25,000

$20,000

$15,000

Cost of program use since RA

$10,000

$5,000

$0

SUB

UC

CBRR

UC

PBTH

UC

N = 501

N = 395

N = 434

N = 434

N = 293

N = 259

Assigned intervention and pairwise comparison

Panel B

SUB vs. CBRR

SUB vs. PBTH

CBRR vs. PBTH

$50,000

$45,668

$44,895

$45,000

$41,743

$40,793

$40,269

$40,000

$35,000

$30,479

$30,000

$25,000

$20,000

$15,000

Cost of program use since RA

$10,000

$5,000

$0

SUB

CBRR

SUB

PBTH

CBRR

PBTH

N = 362

N = 290

N = 215

N = 201

N = 180

N = 184

Assigned intervention and pairwise comparison

CBRR = priority access to community-based rapid re-housing. ES = emergency shelter. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

Notes: Averages are for all 37-month survey respondents in each pairwise comparison and are weighted for survey nonresponse to represent full comparison sample. Cost estimates assume a site-specific average cost per month based on the Family Options Study cost data and HUD administrative data. The other category refers to other permanent housing subsidies and includes permanent supportive housing, public housing, and project-based assistance (project-based vouchers or Section 8 projects).

Sources: Family Options Study cost data; U.S. Department of Housing and Urban Development, Public and Indian Housing Information Center, Tenant Rental Assistance Certification System, and Financial Data Schedule records (SUB); Family Options Study Program Usage Data 3-Year Impacts of Housing and Services Interventions for Homeless Families

114

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

Total costs for the average family in the SUB-versus-UC

average cost of all programs used for families assigned to the

pairwise comparison are shown in the far left set of stacked bar

PBTH intervention compared with assignment to other inter-

charts in panel A of Exhibit 9-5. The total cost for the average

ventions. As at 20 months, in each of the three comparisons

family assigned to the UC group in the study is $42,134. When

involving the CBRR intervention, families assigned to the CBRR

the cost includes the cost of all program use for each set of

intervention have the lower average cost of all programs used.

families—families assigned to the SUB group versus families

For example, in the CBRR-versus-UC comparison, families

assigned to the UC group—the total average cost per family

assigned to the CBRR group used less transitional housing, less

assigned to the SUB group is almost $3,800 more than for

emergency shelter than families assigned to the UC group. The

families assigned to the UC group. In the first 20 months after

largest difference is in the CBRR-versus-PBTH comparison, in

random assignment (see the Short-Term Impacts report), this

which the high cost and greater use of PBTH programs results

difference was only about $500. The Short-Term Impacts report

in a nearly $10,000 difference in average cost of all program

discussed the potential for cost of all program use for families

use for families assigned to the respective interventions. For

assigned to the SUB group to eventually outpace the costs for

the CBRR-versus-UC and SUB-versus-CBRR comparisons,

families assigned to the UC group, and this has happened, with

CBRR families have an average cost of all programs used that

SUB intervention costs now 9 percent higher than UC costs

is $4,000 less than the program costs of those assigned to the

during the entire followup period.

other interventions.

The SUB-versus-UC comparison reveals how assignment to

the SUB group compared with assignment to the UC group

9.4. Monthly Cost of All Program

altered the composition of housing assistance programs used

Use at the Time of the Followup

and their associated costs. The average emergency shelter cost

Survey by Families Assigned to

for families assigned to the UC group during the full 3-year

Each Intervention

period was, on average, 1.7 times that of families assigned to

the SUB group. UC families also frequently found their way to

Exhibit 9-6 shows the monthly costs of all program use in the

other permanent housing programs (mostly public housing in

month of the followup survey for each pairwise comparison.

this instance, which is assumed to have the same monthly cost

This analysis uses the per-family monthly program cost for

as SUB programs) and to relatively costly PBTH programs. The

each type of program and information about the mix of

decreased use of programs providing project-based transitional

program types families were using at the time of the 37-month

housing, rapid re-housing, and emergency shelter by families

followup survey. As discussed in Chapters 3 through 6, the

assigned to the SUB group outweighs much of the increased

mix of programs used during the month of the followup survey

use of SUB programs.

is different than the mix of program use observed during the

In the SUB-versus-CBRR comparison, the cost of program

entire followup period. For example, in the SUB-versus-UC

use since random assignment for families assigned to the SUB

comparison, among families assigned to the UC group, 23

group is about 9 percent higher, on average, than for families

percent used rapid re-housing during the followup period,

assigned to the CBRR group. Compared with the 20-month

but only 2 percent were using rapid re-housing at the time of

findings, the cost ordering changes for only one pairwise

the survey. In a similar way, 29 percent of UC families in the

comparison—the SUB-versus-PBTH comparison. During the

SUB-versus-UC comparison used transitional housing at some

3-year followup period, average costs of all program use for

time during the followup period, but only 4 percent were doing

families assigned to the SUB group are 10 percent higher than

so at the time of the survey. Altogether, 38 percent of families

for families assigned to the PBTH group, whereas total costs for

assigned to the UC group used a permanent housing subsidy

families assigned to the PBTH group were 10 percent higher at

during the followup period, and 31 percent were using a per-

the earlier followup point. Compared with costs of assignment

manent housing subsidy at the time of the survey. The monthly

to the PBTH intervention, costs of assignment to the SUB

costs at the time of the followup survey provide an indication

intervention continued to grow with time as families used per-

of how costs for the interventions may diverge in the future due

manent housing subsidies with no time limit at a much greater

to differing trends in use of the different program types.

rate, while costs associated with taking up PBTH programs

Exhibit 9-6 shows that, in contrasts involving the CBRR

came to an end without being fully offset by increases in shelter

intervention, costs of program use for families assigned to the

or other program use.

CBRR intervention are lower than for families assigned to the

In the other two contrasts containing the PBTH intervention,

SUB intervention (by $192), the UC intervention (by $152),

the high monthly cost of PBTH programs results in a higher

and the PBTH intervention (by $61). This finding reflects the

3-Year Impacts of Housing and Services Interventions for Homeless Families

115

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

Exhibit 9-6. Average Per-Family Monthly Costs for Program Use at Time of the Followup Survey, by Comparison Program type

Other

SUB

CBRR

PBTH

ES

Panel A

SUB vs. UC

CBRR vs. UC

PBTH vs. UC

$1,200

$978

$1,000

$842

$840

$778

$804

$800

$688

$600

$400

$200

Cost of program use in survey month

$0

SUB

UC

CBRR

UC

PBTH

UC

N = 501

N = 395

N = 434

N = 434

N = 293

N = 259

Assigned intervention and pairwise comparison

Panel B

SUB vs. CBRR

SUB vs. PBTH

CBRR vs. PBTH

$1,200

$998

$1,000

$950

$862

$800

$758

$691

$630

$600

$400

$200

Cost of program use in survey month

$0

SUB

CBRR

SUB

PBTH

CBRR

PBTH

N = 362

N = 290

N = 215

N = 201

N = 180

N = 184

Assigned intervention and pairwise comparison

CBRR = priority access to community-based rapid re-housing. ES = emergency shelter. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

Notes: Averages are for all 37-month survey respondents in each arm of each pairwise comparison and are weighted for survey nonresponse to represent full comparison sample.

Cost estimates assume a site-specific average cost per month based on the Family Options Study cost data and HUD administrative data. The other category refers to other permanent housing subsidies and includes permanent supportive housing, public housing, and project-based assistance (project-based vouchers or Section 8 projects).

Sources: Family Options Study cost data (CBRR, PBTH, and ES); U.S. Department of Housing and Urban Development, Public and Indian Housing Information Center, Tenant Rental Assistance Certification System, and Financial Data Schedule records (SUB); Family Options Study Program Usage Data differences in the prevalence of shelter use, transitional hous-incidence of use of programs providing permanent housing

ing, and permanent housing subsidies during the month of the

subsidies among families assigned to the SUB intervention

followup survey.128

outweighed the higher cost associated with more frequent use

of shelter and PBTH programs by families assigned to usual care.

In all contrasts involving the SUB intervention, the costs of pro-

The monthly costs of program use at the time of the followup

gram use during the month of the followup survey are higher

survey for families assigned to the SUB intervention were $136

for families assigned to the SUB intervention. Compared with

higher than for families assigned to the PBTH intervention

costs of program use for families assigned to usual care, costs

and were $192 higher than for families assigned to the CBRR

of program use for families assigned to the SUB intervention

intervention. In the month of the 20-month followup survey

($978) were $136 higher in the month of the followup survey

(see Exhibit 12-20 in the Short-Term Impacts report [Gubits et al.,

than for families assigned to usual care ($842). The greater

2015]), the average per-family monthly cost of program use for

128 As noted previously, the study team used the per-family monthly cost estimated for emergency shelters to approximate program costs at the time of the followup survey.

In reality, these emergency stays may be shorter or longer than 1 month.

3-Year Impacts of Housing and Services Interventions for Homeless Families

116

FAMILY OPTIONS STUDY

Chapter 9. Intervention Costs

families assigned to the SUB intervention was only $20 more

CBRR intervention, however, families assigned to the PBTH

than for families assigned to usual care ($1,086 for families

intervention have higher costs of program use in the month of

assigned to the SUB intervention and $1,066 for families assigned

the survey.

to usual care). This differential had grown to $136 at the time of

It is not clear how expected future costs of homeless or housing

the 37-month survey, as shown in Exhibit 9-6 ($978 for families

assistance will compare across the interventions. It is important

assigned to the SUB intervention and $842 for families assigned

to note that families receiving permanent housing assistance,

to the UC intervention). In the month of the 20-month survey,

mostly through HCV or public housing programs, will contin-

the cost of program use for families assigned to the SUB inter-

ue to accrue monthly costs indefinitely. Families assigned to

vention was $88 higher than for families assigned to the PBTH

usual care or the other interventions, however, may continue to

intervention and $102 higher than for families assigned to the

experience greater housing instability than their counterparts

CBRR intervention.

assigned to the SUB group. This instability could result in

Costs for families assigned to the PBTH intervention are $26

higher future costs from subsequent use of relatively more

a month less than costs for UC families in the month of the

expensive shelter and transitional housing programs.

followup survey. Compared with families assigned to the

3-Year Impacts of Housing and Services Interventions for Homeless Families

117

FAMILY OPTIONS STUDY

CHAPTER 10.

CONCLUSIONS

HUD launched the Family Options Study in 2008 to 2. At 3 years after random assignment, what are the relative fill a gap in knowledge about which housing and

effects of the three active interventions compared with usual

services interventions work best for families experi-

care and of the active interventions compared with each

encing homelessness. Based on the 3-year analysis, this report

other?

provides evidence about the relative effects of priority access to

3. What are the cumulative costs of the interventions during

permanent housing subsidies (SUB), community-based rapid

the 3-years following random assignment?

re-housing (CBRR), and project-based transitional housing

(PBTH) compared with one another and with usual care (UC)

Longer-term followup is often desirable in assessing the effects

(in which families had no priority access to any program but

of social policy interventions and is especially desirable in the

were left on their own to find their way out of shelter). After

case of the Family Options Study. To evaluate the effects of

spending at least 7 days in emergency shelter, nearly 2,300

priority access to temporary programs that can last up to 18

families in 12 sites across the country were randomly assigned

months (CBRR) or 24 months (PBTH), 20 months is not a long

to one of these four interventions. Random assignment

enough period. Some families may not have received a full dose

produced well-matched groups of families, with no systematic

of a temporary program by the time of the 20-month followup

differences in baseline characteristics.

analysis, and anxieties about the impending end of a program

or disruption from having recently moved at the time of the

Families were free to take up the programs to which they were

20-month survey could have depressed families’ outcomes in

given priority access or make other arrangements on their own,

the CBRR and PBTH interventions. To the extent that priority

so families in each group used a mix of programs. Nonetheless,

access to a particular type of program strengthens families or

the study generated substantial contrasts in program use during

sets the foundation for later success, as theorized by propo-

the followup period because the families’ program choices

nents of PBTH programs, new findings may emerge at 3 years.

were influenced strongly by the particular offer of priority

In a similar way, increases in incomes observed at 20 months

access they received from the study. Random assignment and

for CBRR families may set families on a positive trajectory of

the subsequent contrasts in program use provide a strong

sustained benefit from that intervention. On the other hand, to

basis for drawing conclusions about the relative impacts of the

the extent that the reduction in homelessness seen for PBTH

alternative interventions on several aspects of family well-being

families at 20 months was a temporary consequence of still

3 years after random assignment.

being in PBTH programs at that time, impacts may fade. This

This chapter begins by describing the questions that guide

3-year analysis enables the study team to examine outcomes

this 3-year analysis. Then for UC and each of the three active

well after families reach the time limits for these temporary

interventions, the chapter describes program use, family out-

programs. Longer-term analysis is also important to measure

comes (for UC) or impacts (for active interventions, compared

impacts that may take longer than 20 months to emerge, such

with UC and one another), and program costs. The chapter

as those on child well-being outcomes.

concludes with implications for theory and policy.

The Family Options Study: Short-Term Impacts of Housing and

Services Interventions for Homeless Families (hereafter, the Short-

10.1. Questions Addressed in the

Term Impacts report; Gubits et al., 2015) found that families

3-Year Analysis

assigned to all four interventions used a variety of programs.

The 3-year analysis updates information about the ways that

The 3-year analysis addresses three primary questions:

having priority access to particular interventions affected pat-

1. What programs do families who experience homelessness

terns of use. In the case of permanent housing subsidies, which

use during a 3-year period, and how does assignment to an

can last as long as families comply with program requirements,

intervention that offers priority access to a particular kind of

it was not clear at 20 months whether families would success-

program affect this program use?

fully renew leases and sustain tenancies. The 3-year analysis

3-Year Impacts of Housing and Services Interventions for Homeless Families

118

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

addresses whether families assigned to the SUB group are

The 3-year followup data for study participants tell what would

able to retain the assistance. Emergency shelters, transitional

happen if each of these ways of targeting offers and access were

housing programs, and rapid re-housing programs frequently

pursued as federal or local policy—for the families actually

attempt to enroll their families on waiting lists for permanent

studied in the target communities. The pairwise comparisons

housing. Longer-term followup can also show whether these

between active interventions show the impact of offering fami-

programs serve as way stations to permanent housing subsidies.

lies priority access to one type of program rather than another.

The data also allow for the comparison of each option with the

Finally, the Short-Term Impacts report (Gubits et al., 2015)

mix of programs that the homeless assistance systems provided

found that, after 20 months, the cost of all the programs used

at the time of the study (that is, the programs available to the

by families assigned to the SUB group was about the same as

UC group). The pairwise comparisons between active interven-

for families assigned to the UC group. Because the subsidies

tions and usual care show the impact of referring a family to a

offered in the SUB programs are permanent, whereas the CBRR

specific type of program compared with the impact of letting

and PBTH programs are time limited, there was good reason

families pursue any available assistance on their own.

to expect that the relative costs of interventions may change

during a longer followup period. The current report examines

The analysis in this report measures the impact of having been

these costs cumulatively over an average of 37 months after

offered a particular type of program regardless of whether the

random assignment.

family involved actually received the program assistance. The

findings reflect the real way in which the homeless assistance

10.2. Meaning of Intention-to-Treat

system interacts with families, in that families are offered assis-

tance rather than mandated to accept the assistance being of-

Impact Estimates

fered. Whether families participate in a program to which they

The inherent strength of the experimental research design

have gained priority access through their randomly assigned

employed in the Family Options Study is the assurance that

intervention reflects the relative desirability and accessibility

the groups that are created through the random assignment

of the programs for families within the context of the other

process will be similar to each other in all respects except for

options they may choose to pursue on their own.

their priority access to a particular type of homeless or housing

As the report shows, in the 3 years after random assignment,

assistance program. This assurance means that subsequent dif-

a substantial number of families did not use the program to

ferences in outcomes (beyond the bounds of chance sampling

which they were given priority access, and some used other

variability) reflect the relative impact of those interventions.

programs. The full experimental sample for a given intervention

The Family Options Study tests for the impacts of three

collectively shows how different forms of housing assistance are

different potential emphases in federal or local assistance policy

used when families are given priority access to one particular

to homeless families: (1) What impact would priority access

program type while simultaneously having the freedom to

to project-based transitional housing (offered to the PBTH

use other forms of assistance available in their communities.

group) have on families in shelter who are not able to resolve

Including all the families randomly assigned to the UC group

their episodes of homelessness quickly? (2) How does this

similarly reveals the range of programs used when no priority

policy compare with providing access to community-based

access is provided. The programs (including the programs

rapid re-housing (offered to the CBRR group)? (3) How does

offered by the interventions examined in this study) that UC

it compare to permanent housing subsidies (offered to the

families used exist in communities and would each continue

SUB group)? In each case, the corresponding policy question

to exist even with a stronger federal or local push for only one

is, “What impact would this policy emphasis have on the

of them. Thus, the full-sample comparisons between randomly

outcomes of families in shelter relative to usual care or another

assigned interventions—known as “intention-to-treat,” or ITT,

policy emphasis?”

impact estimates—provide the best guide to policymakers in a

messy, complex world and are reported here as the main study

findings.129 Exhibit 10-1 displays a summary of the impact find-

ings for the 18 outcomes included in the executive summary.

129 All this said, evidence of the effects of a particular program type on families who actually use that approach (for example, the effect of rapid re-housing on the families who use the CBRR programs) compared with equivalent families who do not use the approach would have high value to the homeless assistance field. Such information is important, not because any federal or local policy action could actually create such a contrast for the population of all shelter-housed families, but because efforts to improve a particular intervention model need to be based on knowledge of what participating in that model actually does for families compared with not participating. This report does not provide such information because some assumptions necessary to calculate these effects do not appear to hold true for the study sample.

3-Year Impacts of Housing and Services Interventions for Homeless Families

119

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

Exhibit 10-1. Summary of Statistically Significant Impact Results by Policy Comparison: Executive Summary Outcomes 37 Months After RA

Statistically Significant ITT Impact Estimates

Outcome

SUB

CBRR

PBTH

SUB

SUB

CBRR

vs. UC

vs. UC

vs. UC

vs. CBRR

vs. PBTH

vs. PBTH

Effect

Effect

Effect

Effect

Effect

Effect

Sizea

Sizea

Sizea

Sizea

Sizea

Sizea

Housing stability (intervention goal: lower values)

At least 1 night homelessb or doubled up in past 6 months or in shelter

ê – 0.37

ê – 0.36

ê – 0.43

in past 12 monthsc (%) [confirmatory]

At least 1 night homelessb or doubled up in past 6 months (%)

ê – 0.33

ê – 0.28

ê – 0.40

Number of places lived in past 6 months

ê – 0.20

ê – 0.12

ê – 0.25

Any stay in emergency shelter in months 7 to 18 after RA (%)

ê – 0.33

ê – 0.14

ê – 0.30

ê – 0.13

Family preservation (intervention goal: lower values)

Family has at least one child separated in past 6 monthsd (%)

ê – 0.19

Spouse/partner separated in past 6 months, of those with spouse/partner éf 0.23

present at RAe (%) [limited base]

Family has no child reunified, of those families with at least one child

absent at RAg (%) [limited base]

Adult well-being (intervention goal: lower values)

Health in past 30 days was poor or fair (%)

Psychological distressh

ê – 0.11

ê – 0.19

Alcohol dependence or drug abuse in past 6 monthsi (%)

ê – 0.19

Experienced intimate partner violence in past 6 months (%)

ê – 0.12

Child well-being (intervention goal: lower values)

Number of schools attended since RAj

ê – 0.13

ê – 0.22

ê – 0.18

School absences in past monthk (ages 5 to 17 years)

Poor or fair health (%)

Behavior problemsl

ê – 0.13

ê – 0.12

ê – 0.15

ê – 0.13

Self-sufficiency (intervention goal: higher values)

Work for pay in week before survey (%)

Total family income ($)

Household is food secure (%)

é 0.17

é 0.22

é 0.20

Number of families

895

868

551

652

414

363

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment.

a Effect size columns show standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing. Additional impacts on the use of transitional housing are provided in Appendix E.

c After adjustment for multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-UC, SUB-versus-CBRR, and SUB-versus-PBTH comparisons.

d Percentage of families in which a child who was with the family at baseline became separated from the family in the 6 months before the 37-month survey.

e Percentage of families in which a spouse or partner who was with the family at baseline became separated from the family in the 6 months before the 37-month survey.

f Assignment to SUB increased spouse/partner separations relative to UC. Separations may have allowed family heads to leave relationships in which they had experienced intimate partner violence.

g Percentage of families in which at least one child was separated from the family at baseline and no child was reunited with the family at the time of the 37-month survey.

h Psychological distress is measured with the Kessler Psychological Distress Scale (K6) and ranges from 0 to 24, with higher scores indicating greater distress. Impacts shown as standardized effect sizes. Effect sizes were standardized by dividing impacts by standard deviation for the UC group.

i Measures evidence of alcohol dependence or drug abuse using responses to the Rapid Alcohol Problems Screen (RAPS-4) and six items from the Drug Abuse Screening Test (DAST-10).

j Number of schools outcome is topcoded at 4 or more schools.

k Absences outcome is defined as 0 = no absences in past month; 1 = one to two absences; 2 = three to five absences; 3 = six or more absences. This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

l Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire, or SDQ.

Notes: This exhibit displays findings that are statistically significant at the .10 level or more. Blank cells indicate that no statistically significant impact was detected. Impact estimates are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

3-Year Impacts of Housing and Services Interventions for Homeless Families

120

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

10.3. Usual Care (UC)

permanent housing subsidies, they also retained the assistance.

That is, the study found that most families (82 percent) who

Emergency shelters in this study were the entry points into

obtained a permanent housing subsidy during the followup

homeless assistance in each site. Families randomly assigned to

period were still using that subsidy at the time of the followup

the UC group did not receive priority access to any program,

survey. Families assigned to the UC group also used other

although a range of supports were available to them if accessed

forms of assistance. Slightly less than one-third received tran-

by the family’s own initiative. In fact, UC families typically re-

sitional housing and 20 percent received rapid re-housing at

mained in emergency shelter for some additional time, seeking

some point during the followup period. At 3 years after random

whatever assistance was available in the community. Thus, the

assignment, 40 percent were still using some form of assistance.

experiences of UC families reflect how the homeless assistance

systems work in the 12 communities studied when families

At 3 years after random assignment, UC families had, on

in shelter were not given priority access to another homeless

average, made modest improvements in circumstances since

or housing assistance program. The study provides valuable

the 20-month followup point but still experienced substantial

information about what types of assistance families use without

housing instability, low incomes, and low rates of employ-

special offers of assistance and how families who have spent at

ment. Nearly one-fifth of UC families reported at least 1 night

least 7 days in shelter progress over time.

homeless in the 6 months before the 37-month survey. This

proportion is smaller than that at the 20-month followup point

UC families (that is, families to whom random assignment

but indicates that families were still experiencing substantial

did not give priority access to any active intervention) spent

instability. More than one-third (37 percent) of UC families

substantial periods of time in emergency shelter after random

were working in the week before the 37-month survey, a high-

assignment. UC families spent an average of 4 months in emer-

er proportion than at baseline or at the time of the 20-month

gency shelter in the 3 years following random assignment, almost

followup survey. Median annual cash income from all sources

all of it as part of their initial shelter stays. More than one-half

for the calendar year before the survey was about $12,000,

(60 percent) of UC families spent less than 3 months in emer-

less than two-thirds of the federal poverty threshold for a

gency shelter cumulatively, 19 percent spent 3 to 6 months,

three-person family in the study in the same year (U.S. Census

and 21 percent spent more than 6 months in emergency shelter

Bureau 2013).

during the followup period.

At the 37-month followup point, a substantial number of UC

Emergency shelters offered a range of supportive services.

families experienced poor outcomes in the studied domains.

The shelters studied provided a range of supportive services

Altogether, 17 percent had been separated from a child who

in primarily congregate settings (dorms or other group living

was with the family at the time of random assignment, and

situations). All the shelters offered comprehensive needs as-

3 per cent had a child in foster care. Of UC family heads, nearly

sessments, case management, supportive services, and referrals

one-third reported poor or fair health, 11 percent reported

to other programs. Shelters in some instances also offered

alcohol dependence or substance abuse, and 11 percent had

supportive services such as access to physical health care,

experienced intimate partner violence in the past 6 months.

employment training, child advocacy, life skills training, mental

Children had attended more than two schools in the past 3 years

health care, and parenting services.

and parents reported behavior problems well above national

UC families participated in homeless and housing assistance

averages. At the time of the survey, nearly one-half of UC

programs at fairly high rates and many were able to access

families (47 percent) were food insecure.

and retain permanent housing subsidies. In the 3-year

UC families incurred substantial costs. The study found that

followup period, some families assigned to the UC group did

the emergency shelter programs that the UC families used cost

not use any other form of homeless or housing assistance

slightly more than $4,800 per family per month. Of this total,

besides shelters, but most did. The study found that, even

63 percent was for supportive services. Altogether, costs of all

without having priority access to a permanent housing subsidy

the homeless and housing programs and associated services

during the 37-month follow-up period, more than one-third

that families assigned to the UC group accessed—whether

(37 percent) of UC families were able to obtain some form of

in a shelter or in active programs—were about $40,000 per

permanent subsidy130 and used it for an average of 19 months.

family during the 3-year followup period compared with about

Not only were families assigned to the UC group able to obtain

$30,000 in accumulated costs through 20 months. Further, the

130 These subsidies include the permanent housing subsidies offered to the SUB group, permanent supportive housing, public housing, and project-based assistance and Section 8 projects.

3-Year Impacts of Housing and Services Interventions for Homeless Families

121

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

cost of all program use for families in the UC group, particular-

permanent subsidy. Smaller proportions of families assigned

ly in the month of the 37-month followup survey, was a little

to the SUB group used rapid re-housing (11 percent) and tran-

more than $800, roughly 20 percent less than the monthly

sitional housing (7 percent) at some point during the 3 years,

cost of program use at the time of the 20-month survey. Thus,

with some overlap among the three groups.132

UC families continued to incur costs for housing assistance be-

Compared with the CBRR and PBTH interventions and with

tween months 21 through 37, but the rate of cost accumulation

usual care, assignment to the SUB intervention caused improve-

had slowed by the 3-year followup point.

ments in housing stability 3 years after random assignment.

Having priority access to permanent housing subsidies reduced

10.4. Permanent Housing Subsidy

the proportion of families with a stay in shelter or places not

(SUB)

meant for human habitation in the 6 months before the 37-month

survey by more than one-half when compared with assignment

In most cases, the families assigned to the SUB intervention

to the PBTH group or with assignment to the UC group. As-

were given priority access to a housing choice voucher, and

signment to the SUB group also led to notable improvements in

they may have been offered housing search assistance (they

other aspects of housing stability relative to assignment to the

were not offered ongoing social services). The permanent

UC group and both of the other groups, reducing the incidence

housing subsidies offered to SUB families are not generally

of doubling up, subsequent emergency shelter stays, housing

accessible to families while in emergency shelter unless families

crowding, and number of places families lived during the fol-

reach the top of waiting lists for subsidies during that period.

lowup period. Compared with assignment to the CBRR group,

Against that circumstance, what does the Family Options

however, the study team did not detect an effect of assignment

Study tell about offering homeless families priority access to

to the SUB group on the proportion of families who reported

permanent housing subsidies?

homelessness in the 6 months before the 37-month followup

When permanent housing subsidies are available to families

survey. The study did not find evidence that assignment to

in shelter, they take it up at high rates and continue to use

the SUB group caused changes in the quality of housing that

it for a sustained period. SUB programs were the least likely

families reported at the time of the 37-month followup survey

of the active interventions studied to exclude families because

compared with assignment to any of the other groups.

of eligibility rules. For example, only 2 percent of families in

The benefits of assignment to the SUB intervention relative

the study were disqualified from random assignment to the

to assignment to the PBTH intervention and to usual care ex-

SUB group because of answers to screening questions asked

tended beyond housing stability at the 3-year followup point.

by study staff. Of the families randomly assigned to the SUB

At 3 years after random assignment, the benefits of having

group, however, 11 percent were found to be ineligible after

priority access to permanent housing subsidies compared with

random assignment. Altogether during the 3-year followup

usual care extended beyond housing stability, with reductions

period, 83 percent of respondent families assigned to the SUB

in adult psychological distress and in intimate partner violence

group at some point used the permanent subsidy that was

(a reduction of one-third in this indicator). Assignment to

offered.131 Most families assigned to the SUB group who used

the SUB group increased separations of spouses and partners

their offered permanent subsidy continued using it to the end

relative to assignment to the UC group and reduced child sep-

of the followup period. Among those who ever used their

arations relative to assignment to the PBTH group. Assignment

offered permanent housing subsidy, the average duration was

to the SUB group reduced the number of schools attended

31 months. In addition to the 83 percent of families assigned to

by children relative to assignment to all the other groups.

the SUB group who used the permanent subsidy to which they

Compared with assignment to the UC group, assignment to

had priority access, some families assigned to the SUB group

the SUB group led to improvements in other areas of child

used other forms of permanent subsidy to which they did not

well-being, with reductions in behavior and sleep problems

have priority access, bringing the total who used any form of

and with improvements in prosocial behavior. Relative to

permanent subsidy to 88 percent. By the time of the 37-month

assignment to the PBTH group, assignment to the SUB group

followup survey, 73 percent were still receiving some form of

led to greater school grade completion. Some impacts of the

131 The takeup rate presented here is the proportion of all survey respondent families assigned to the SUB group who used the permanent housing subsidy assistance that was offered. A voucher “success rate” refers to the proportion of families who were issued a voucher who leased up with a unit. The Interim Report: Family Options Study (Gubits et al., 2013) reported a success rate of 94 percent, based on information about voucher issuance collected through the end of random assignment. This rate is higher than the 69-percent success rate found by Finkel and Buron (2001).

132 For example, the same family may have used permanent housing subsidies and project-based transitional housing at different points during the followup period.

3-Year Impacts of Housing and Services Interventions for Homeless Families

122

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

SUB intervention relative to usual care observed at 20 months

$4,000), on average, than assignment to the PBTH group and

(for example, reductions in adult alcohol dependence or drug

about 9 percent more than assignment to the CBRR or UC

abuse, reductions in separations and foster care placements of

groups during the study period. That is, the substantial gains

children, and increases in children’s school attendance) were

in housing stability and other outcomes associated with assign-

no longer evident at 37 months, perhaps because more UC

ment to the SUB intervention come at some additional cost. In

families had stabilized in housing. Even if these findings are

the month of the 37-month survey, the cost of program use for

short term, they should be considered among the benefits of

families assigned to the SUB group was higher than for families

assignment to the SUB group during the full 3-year period.

assigned to any of the other groups by 15 to 25 percent. This

differential in monthly cost of program use is greater than what

Compared with assignment to the UC group, assignment

was observed at the earlier, 20-month followup survey. In the

to the SUB group reduced labor market engagement in the

month of the 20-month survey, the cost of program use for

second half of the followup period. Compared with their

SUB families was 2 percent higher than that for UC families

counterparts assigned to the UC group, the heads of families

and 8 to 9 percent higher than that for PBTH families or CBRR

assigned to the SUB group worked less during the full 3-year

families.

followup period and in the second half of the followup period

(in the time from the 20- to the 37-month followup surveys).

In the UC group, 64 percent of family heads had worked for

10.5. Community-Based Rapid

pay at some point during the second half of the followup pe-

Re-housing (CBRR)

riod, but only 58 percent of those in the SUB group had done

so. These employment effects were not evident in comparisons

The CBRR intervention offered priority access to short-term

of the SUB intervention with the CBRR or PBTH intervention.

rental assistance lasting up to 18 months (median length of

In comparisons of the SUB intervention with the CBRR and

use was 8 months) to rent private-market housing. CBRR pro-

PBTH interventions and usual care, the study did not detect

grams also offered limited case management services focused

differences in the proportion of family heads in the SUB group

on housing and self-sufficiency. CBRR programs typically

who worked in the week before the survey or the proportion of

received funding from the Homelessness Prevention and Rapid

families who had earnings in the month before the survey.

Re-Housing Program, or HPRP. What do the findings from the

3-year assessment tell about this intervention?

Compared with assignment to the PBTH and UC groups, as-

signment to the SUB group improved food security. The study

Takeup of offered rapid re-housing was relatively low. Of

did not detect effects of assignment to the SUB group on annual

families randomly assigned to the CBRR group, 59 percent

family cash income relative to assignment to any of the other

used rapid re-housing rental assistance during the 37-month

interventions but showed that assignment to the SUB interven-

followup period, a rate that is much lower than the 83-percent

tion improved family food security relative to assignment to the

takeup rate for SUB programs in that random assignment group.

PBTH intervention and to usual care. Families assigned to the

Qualitative research suggested that the short duration of CBRR

SUB group also reported less economic stress in the 6 months

programs—or uncertainty about its duration—made some

before the 3-year followup survey compared with PBTH and

families reluctant to use CBRR programs (Fisher et al., 2014).

UC families.

Families assigned to the CBRR group were able to use and

During the 3-year followup period, the cost of all program use

retain permanent housing subsidies. Families assigned to

for families assigned to the SUB group exceeded that of fam-

the CBRR group used multiple forms of permanent housing

ilies assigned to the CBRR, PBTH, and UC groups by roughly

subsidies during the 3-year followup period (35 percent

9 to 10 percent. On average, SUB programs cost about $1,200

across all types of permanent subsidy). It does not appear that

per family per month, which is lower than the corresponding

assignment to the CBRR group led to greater use of permanent

monthly costs for emergency shelter and PBTH programs but

housing subsidies compared with assignment to the UC

higher than the monthly cost for CBRR programs. During the

group, but CBRR families began using the permanent housing

3-year period, however, SUB families used programs (usually

subsidies later in the followup period than their counterparts in

permanent housing subsidies) to a much greater extent than

the UC group. By the 37-month followup survey, 30 percent of

did the families assigned to the other interventions. As a conse-

CBRR families and 31 percent of UC families were using some

quence, compared with the average cost of all program use for

form of permanent subsidy. The rate of retention of permanent

each of the CBRR, PBTH, and UC groups, the average cost of all

housing subsidies among families who used them was similar

program use for families assigned to the SUB group was about

for all three groups. About one-fourth of the families assigned

$4,000 higher than for families in the other interventions. As-

to the CBRR group (23 percent) also used transitional housing.

signment to the SUB group costs about 10 percent more (about

3-Year Impacts of Housing and Services Interventions for Homeless Families

123

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

The CBRR intervention resulted in outcomes equivalent to

that took place before random assignment, and 18 percent of

those observed with usual care but was less effective than

those who passed and were assigned to the PBTH group were

the SUB intervention in preventing subsequent stays in

subsequently screened out as ineligible by the transitional

shelters and doubling up and in improving other aspects of

housing programs to which they were referred. Of the families

housing stability. The study found that having priority access

assigned to the PBTH group, 53 percent used some form of

to community-based rapid re-housing was equivalent to usual

transitional housing during the 3-year followup period. This

care—and was substantially less effective than having priority

low level of takeup reflects a combination of family choices and

access to a permanent housing subsidy—in reducing subse-

program eligibility restrictions, with some families deemed inel-

quent stays in shelters and doubling up and in improving other

igible by the programs to which they were offered priority ac-

aspects of housing stability. Having priority access to rapid

cess and some families choosing not to use the PBTH assistance

re-housing had little impact on outcomes in other domains

offered. Qualitative interviews suggest that the fixed location of

compared with usual care.

PBTH units may be a barrier to takeup when assigned locations

are not close to families’ schools, work, transportation, and

The CBRR intervention has the lowest monthly cost of the

support networks, or when families perceive the facilities to be

active interventions studied, and total costs of all programs

in bad neighborhoods (Fisher et al., 2014).

used during the 3-year followup period for those assigned to

the CBRR group were lower than those of families assigned

Families assigned to the PBTH group were able to use and re-

to the SUB, PBTH, and UC groups. CBRR programs had a

tain permanent housing subsidies even without having priority

lower per-family monthly cost than those of PBTH and SUB

access to that type of assistance. One-third of families assigned

programs, averaging about $900. Housing costs comprised,

to the PBTH group used some form of permanent housing

on average, 72 percent of these costs. The total cost of all pro-

subsidy during the followup period, despite not receiving

grams used by CBRR families during the 3 years after random

priority access to that form of assistance. More than one-fourth

assignment was, on average, about $4,000 less than the cost of

of families assigned to the PBTH group were still receiving

program use for SUB and UC families and about $10,000 less

some type of permanent subsidy at the time of the 37-month

than cost of program use for PBTH families.

followup survey. Families assigned to the PBTH group used

these other forms of permanent housing subsidies at about the

10.6. Project-Based Transitional

same rate as did families assigned to the UC group, but families

assigned to the PBTH group began using permanent housing

Housing (PBTH)

subsidies later in the followup period than did UC families.

The PBTH intervention offered priority access to housing for

Having priority access to transitional housing programs thus

up to 24 months, coupled with a wide array of social services.

did not lead to greater use of permanent housing subsidies

The study focused on transitional housing provided in agency-

than did usual care by the end of the followup period. Some

controlled settings (although some PBTH families were referred

families assigned to the PBTH group also used rapid re-housing

to programs with scattered-site units; in all cases, families were

programs (14 percent).

required to move from their units when assistance ended).

Assignment to the PBTH intervention reduced stays in

All the PBTH programs studied offered comprehensive case

emergency shelter in the final year of the followup period

management, assessed family needs, and offered direct services

compared with usual care but did not lead to other effects on

and dedicated referrals to outside providers to address those

housing stability and did not lead to effects on other aspects

needs. PBTH programs offered access to employment training,

of family well-being. Compared with usual care, assignment

life skills, mental health care, parenting skills, and physical

to the PBTH intervention reduced the proportion of families

health care. The scope of needs addressed in PBTH programs

who had stays in emergency shelter in months 21 to 32 after

was similar to that of emergency shelters. During the followup

random assignment. Program Usage Data show that families

period, 53 percent of families assigned to the PBTH group used

assigned to the PBTH group were more likely than UC families

that form of assistance for an average duration of 15 months.

to still be using PBTH programs in this period.) The study

What lessons does the Family Options Study provide about the

finds no evidence of other effects of assignment to the PBTH

PBTH intervention?

intervention on other indicators of housing stability, housing

Takeup of project-based transitional housing programs was

independence, or housing quality at the 3-year followup

relatively low. PBTH providers were more selective than either

survey. The study also does not find evidence that assignment

SUB or CBRR providers regarding the families they would

to the PBTH group led to effects on any of the other domains

serve. Nearly one-fourth (23 percent) of families considered for

examined compared with assignment to the UC group.

the study did not pass the initial screening for PBTH programs

3-Year Impacts of Housing and Services Interventions for Homeless Families

124

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

The PBTH programs cost less than emergency shelters on a

Study findings lend support for the underlying theoretical

per-family, per-month basis, and total costs for PBTH families

model for permanent housing subsidies. The striking impacts

during the 3-year period after random assignment are less

of assignment to the SUB group in reducing subsequent

than for families assigned to the SUB group and greater than

stays in shelters or places not meant for human habitation

for families assigned to the UC and CBRR groups. PBTH

provide support for the view that homelessness is for many

programs cost about $2,700 per family per month, with

families a housing affordability problem that can be remedied

supportive services constituting 42 percent of these costs. This

with per manent housing subsidies—without specialized

monthly cost is less than that of emergency shelter but is more

homeless-specific psychosocial services. The larger set of

than the monthly costs for SUB and CBRR programs. The cost

findings on this active intervention also provides some support

of all program use by PBTH families during the 3-year followup

for the theoretical proposition that resolving homelessness,

period was lower by $4,100 than the average total cost for SUB

when that impact can be achieved, has a radiating impact on

families, modestly higher by $1,400 than that for UC families,

adult and child well-being and food security compared with

and substantially higher by $9,800 than that for CBRR families.

usual care.

Assignment to the PBTH group did not lead to longer-term

Few study findings support the theoretical model underlying

effects on family well-being. The 3-year analysis does not pro-

project-based transitional housing. Project-based transitional

vide evidence of longer-term effects of assignment to the PBTH

housing is intended to address psychosocial challenges and

group. The study finds no evidence that, despite its inclusion

housing barriers by providing social services. The study does

of psychosocial services, assignment to the PBTH intervention

not provide evidence that this intervention accomplished that

led to improvements in family preservation, adult well-being,

result. Although assignment to the PBTH intervention reduced

or child well-being relative to usual care at 37 months. Among

homelessness at the 20-month followup point and reduced the

families eligible for both the PBTH and CBRR interventions,

number of emergency shelter stays through month 32, when

assignment to the PBTH intervention was less successful

compared with usual care, it did not produce effects in other

than assignment to the CBRR intervention in reducing adult

aspects of family well-being.

psychological distress, child behavior problems, and family

food insecurity.

10.9. Summary of Findings and

Implications for Policy

10.7. Impacts by Level of Family

Challenge

The Family Options Study’s random assignment design for

measuring intervention impacts is a stronger design than that of

Families participating in the Family Options Study experienced

other studies of programs for homeless families. Evidence from

numerous psychosocial challenges and housing barriers.

the study’s 3-year followup survey provides important new

The study yielded scant evidence that, during the 37-month

information about what happens to families who experience

followup period, any of the interventions studied works

homelessness in the absence of any special offers of assistance.

comparatively better for families who had more psychosocial

It also provides information about the impact of assignment to

challenges or housing barriers at baseline than for families who

three particular interventions: SUB, CBRR, and PBTH.

faced fewer difficulties. Nor did the reverse pattern occur. As

A clear finding from the study is that homelessness is expensive

a result, the study’s clearest guidance for policy for all types of

for families and communities. Even without priority access

families in the medium term consists of the main study results

to assistance, families in 12 communities used housing and

on overall impacts.

services programs costing about $40,000, on average, during

a period of a little more than 3 years. Despite this considerable

10.8. Implications for Theory

public (and in some cases private) investment, many families

who had been in shelter for at least 7 days at the outset of

In addition to findings on the effects of priority access to the

the study were still not faring well 3 years later. More than

three active interventions (relative to usual care and relative

one-third had been homeless or doubled up recently, nearly

to each other) and on intervention costs, the Family Options

one-half were food insecure, and incomes averaged less than

Study is also informative about the various theories underlying

two-thirds of the poverty threshold. The high cost of homeless

the active interventions. This section draws out implications in

services suggests that prevention efforts with low per-family

this area.

costs—if they were effective—would not need to be tightly

targeted to just the families who would otherwise experience

homelessness in order to save resources.

3-Year Impacts of Housing and Services Interventions for Homeless Families

125

FAMILY OPTIONS STUDY

Chapter 10. Conclusions

The longer-term evidence from the Family Options Study

group but at 9 percent lower costs, mainly because assignment

presented in this report indicates that having priority access to

to the CBRR intervention lowers the rate at which families use

deep permanent housing subsidies produces substantial ben-

costly transitional housing programs. Assignment to the PBTH

efits for families. More than one-third of families assigned to

intervention has few advantages over other types of assistance.

the CBRR, PBTH, or UC groups found their way to permanent

In addition, the study does not provide appreciable evidence

housing subsidies, but families given priority access to that as-

that intervention impacts differ according to the number of

sistance obtained subsidies sooner. Providing priority access to

families’ psychosocial challenges or housing barriers at baseline.

subsidies costs 9 percent more than not giving families any pri-

The Family Options Study suggests that families who expe-

ority offer during a 3-year followup period, and it suppressed

rience homelessness can successfully use and retain housing

work effort by about 6 percentage points during the second

vouchers, and that having priority access to deep permanent

half of that period; however, it had substantial benefits. Assign-

housing subsidies has considerable benefits at some additional

ment to the SUB group reduced by more than one-half most

cost. The homeless assistance system does not currently

forms of residential instability, improved multiple measures of

provide immediate access to such subsidies for most families in

adult and child well-being, and reduced food insecurity.

shelter, although more than one-third of families without pri-

The 3-year evidence shows that families randomly assigned to

ority access nevertheless obtained permanent housing subsidies

the CBRR group do about as well as families assigned to the UC

during a 3-year followup period.

3-Year Impacts of Housing and Services Interventions for Homeless Families

126

FAMILY OPTIONS STUDY

CHAPTER 11.

REFERENCES

Angrist, Joshua D., and Jören-Steffen Pischke. 2008. Mostly

———. 2014. National Center for Health Statistics.

Harmless Econometrics: An Empiricist’s Companion. Princeton,

VitalStats. http://www.cdc.gov/nchs/data_access/vitalstats/

NJ: Princeton University Press.

VitalStats_Births.htm.

Bassuk, Ellen L., and Stephanie Geller. 2006. “The Role of

Cherpitel, Cheryl J. 2000. “A Brief Screening Instrument for

Housing and Services in Ending Family Homelessness,”

Problem Drinking in the Emergency Room: The RAPS-4,”

Housing Policy Debate 17 (4): 781–806.

Journal of Studies on Alcohol 61 (3): 447–449.

Bloom, Barbara, Lindsey I. Jones, and Gulnur Freeman.

Culhane, Dennis P., Stephen Metraux, and Thomas Byrne.

2013. “Summary Health Statistics for U.S. Children: Na-

2011. “A Prevention-Centered Approach to Homelessness

tional Health Interview Survey, 2012.” National Center for

Assistance: A Paradigm Shift?” Housing Policy Debate 21 (2):

Health Statistics, Vital Health Stat 10 (258): 1–73.

295–315.

Burt, Martha R. 2006. Characteristics of Transitional Housing

Culhane, Dennis P., Stephen Metraux, Jun Min Park, Mary-

for Homeless Families: Final Report. Prepared for the U.S.

anne Schretzman, and Jesse Valente. 2007. “Testing a Ty-

Department of Housing and Urban Development, Office of

pology of Family Homelessness Based on Patterns of Public

Policy Development and Research. Washington, DC: Gov-

Shelter Utilization in Four U.S. Jurisdictions: Implications

ernment Printing Office.

for Policy and Program Planning,” Housing Policy Debate 18

(1): 1–28 .

———. 2010. Life After Transitional Housing for Homeless

Families. Prepared for the U.S. Department of Housing and

Dahl, Ronald E., and Alison G. Harvey. 2007. “Sleep in

Urban Development, Office of Policy Development and

Children and Adolescents with Behavioral and Emotional

Research. Washington, DC: Government Printing Office.

Disorders,” Sleep Medicine Clinics 2: 501–511.

Burt, Martha R., Laudan Y. Aron, and Edgar Lee, with Jesse

Finkel, Meryl, and Larry Buron. 2001. Study on Section 8

Valente. 2001. Helping America’s Homeless: Emergency Shelter

Voucher Success Rates: Vol. 1 Quantitative Study of Success Rates

or Affordable Housing? Washington, DC: Urban Institute

in Metropolitan Areas. Washington, DC: U.S. Department of

Press.

Housing and Urban Development.

Centers for Disease Control and Prevention (CDC).

Fisher, Benjamin W., Lindsay S. Mayberry, Marybeth Shinn,

2012a. Early Release of Selected Estimates Based on Data

and Jill Khadduri. 2014. “Leaving Homelessness Behind:

From the 2011 National Health Interview Survey. Available

Housing Decisions Among Families Exiting Shelter,” Housing

at http://www.cdc.gov/nchs/data/nhis/earlyrelease/

Policy Debate 24 (2): 364–386.

earlyrelease201206_13.pdf. Accessed July 2012.

Goodman, Robert N. 1997. “The Strengths and Difficulties

———. 2012b. 2011 National Health Interview Survey

Questionnaire: A Research Note,” Journal of Child Psychology

Instrument. ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/

and Psychiatry 38: 581–586.

Survey_Questionnaires/NHIS/2011/English/qadult.pdf.

Greene, William H. 2003. Econometric Analysis: Fifth Edition.

Accessed September 2016.

Upper Saddle River, NJ: Prentice-Hall.

———. 2012c. 2011 State and Local Youth Risk Behavior

Gubits, Daniel, Marybeth Shinn, Stephen Bell, Michelle

Survey. http://www.cdc.gov/healthyyouth/yrbs/pdf/

Wood, Samuel Dastrup, Claudia D. Solari, Scott R. Brown,

questionnaire/2011_hs_questionnaire.pdf.

Steven Brown, Lauren Dunton, Winston Lin, Debi McInnis,

Jason Rodriguez, Galen Savidge, and Brooke E. Spellman.

———. 2012d. CDC Youth Risk Behavioral Surveillance

2015. Family Options Study: Short-Term Impacts of Housing

Survey Results. http://www.cdc.gov/healthyyouth/data/yrbs/

and Services Interventions for Homeless Families. Washington,

results.htm.

DC: Government Printing Office.

3-Year Impacts of Housing and Services Interventions for Homeless Families

127

FAMILY OPTIONS STUDY

Chapter 11. References

Gubits, Daniel, Brooke Spellman, Lauren Dunton, Scott

Miller, Portia, Elizabeth Votruba-Drzal, and Claude M. Setodji.

Brown, and Michelle Wood. 2013. Interim Report: Family

2013. “Family Income and Early Achievement Across the

Options Study. Prepared for the U.S. Department of Housing

Urban-Rural Continuum,” Developmental Psychology 49 (8):

and Urban Development, Office of Policy Development and

1452–1465.

Research. Washington, DC: Government Printing Office.

Mills, Gregory, Daniel Gubits, Larry Orr, David Long, Judith

Gubits, Daniel, Brooke Spellman, Debi McInnis, Stephen

Feins, Bubul Kaul, Michelle Wood, Amy Jones, Cloudburst

Bell, and Michelle Wood. 2012. Revised Data Collection and

Consulting, and the QED Group. 2006. Effects of Housing

Analysis Plan: Family Options Study. Prepared for the U.S.

Vouchers on Welfare Families: Final Report. Prepared for the

Department of Housing and Urban Development, Office of

U.S. Department of Housing and Urban Development,

Policy Development and Research. Washington, DC: Gov-

Office of Policy Development and Research. Washington,

ernment Printing Office.

DC: Government Printing Office.

Harvard Medical School. 2012. “NCS–R Twelve-Month Prev-

National Research Council and Institute of Medicine. 2010.

alence Estimates: Table 2, 12–Month Prevalence of DSM-IV/

Student Mobility: Exploring the Impact of Frequent Moves on

WMH-CIDI Disorders by Sex and Cohort 1 (n=9282).” Na-

Achievement: Summary of a Workshop. Committee on the Im-

tional Comorbidity Survey (NCS) and National Comorbidity

pact of Mobility and Change on the Lives of Young Children,

Survey Replication (NCS–R). Cambridge, MA: Harvard Uni-

Schools, and Neighborhoods. Board on Children, Youth,

versity Press. http://www.hcp.med.harvard.edu/ncs/ ftpdir/

and Families, Division of Behavioral and Social Sciences and

NCS-R_12-month_Prevalence_Estimates.pdf.

Education. Washington, DC: The National Academies Press.

Huber, Peter J. 1967. “The Behavior of Maximum Likelihood

Nord, Mark, Margaret Andrews, and Steven Carlsen. 2005.

Estimates Under Nonstandard Conditions.” In Proceedings

Household Food Security in the United States, 2004. ERS

of the Fifth Berkeley Symposium on Mathematical Statistics and

Food Assistance and Nutrition Report No. ERR11. Washington,

Probability, edited by Lucien M. Le Cam, Jerzy Neyman, and

DC: Government Printing Office.

Elizabeth L. Scott. Berkeley, CA: University of California

Papay, James, and John Hedl, Jr. 1978. “Pyschometric Char-

Press, Vol. 1: 221–233.

acteristics and Norms for Disadvantaged Third and Fourth

Kann, Laura, Steve Kinchen, Shari L. Shanklin, Katherine

Grade Children on State-Trait Anxiety Inventory for Children,”

H. Flint, MA, Joseph Hawkins, William A. Harris, Richard

Journal of Abnormal Child Psychology 6 (1): 115–120.

Lowry, Emily O’Malley Olsen, Tim McManus, David Chyen,

Pearlin, Leonard, and Carmi Schooler. 1978. “The Struc-

Lisa Whittle, Eboni Taylor, Zewditu Demissie, Nancy Brener,

ture of Coping,” Journal of Health and Social Behavior 19 (1):

Jemekia Thornton, John Moore, PhD, Stephanie Zaza.

2–21.

(2014). Youth Risk Behavior Surveillance – United States, 2013.

MMWR. 63(ss-4):1-168. Accessed April 4, 2016. http://

Ponitz, Claire C., Megan M. McClelland, Jamaal S. Mat-

www.cdc.gov/mmwr/pdf/ss/ss6304.pdf.

thews, and Frederick J. Morrison. (2009) “A Structured

Observation of Behavioral Self-Regulation and Its Contribu-

Kessler, Ronald C., Peggy R. Barker, Lisa J. Colpe, Joel

tion to Kindergarten Outcomes,” Developmental Psychology

F. Epstein, Joe C. Gfroerer, Eva Hiripi, Mary J. Howes,

45 (3): 605–619.

Sharon-Lise T. Normand, Ronald W. Manderscheid, Ellen

E. Walters, and Alan M. Zaslavsky. 2003. “Screening for

Ponitz, Claire E. Cameron, Megan E. McClelland, Abigail

Serious Mental Illness in the General Population,” Archives of

M. Jewkes, Carol McDonald Connor, Carrie L. Farris, and

General Psychiatry 60 (2): 184–189.

Fredrick J. Morrison. 2007. “Touch Your Toes! Developing

a Direct Measure of Behavioral Regulation in Early Child-

Layzer, Carolyn. 2014. Personal communication. Associate,

hood,” Early Childhood Research Quarterly 23: 141–158.

Abt Associates.

Ramirez, Mary Louise, Ann S. Masten, and D. M. Samsa.

Little, Roderick J.A. 1986. “Survey Non-Response Adjust-

1991. Fears in Homeless Children. Paper presented at the

ments for Estimates of Means,” International Statistical Review

Biennial Meeting for the Society for Research in Child Devel-

54 (2): 139–157.

opment, Seattle, WA.

McGrew, Kevin S., Fredrick A. Shrank, and Richard W. Wood-

cock. 2007. “Technical Manual.” Woodcock-Johnson Norma-

tive Update. Rolling Meadows, IL: Riverside Publishing.

3-Year Impacts of Housing and Services Interventions for Homeless Families

128

FAMILY OPTIONS STUDY

Chapter 11. References

Rog, Debra J., and John C. Buckner. 2007. Toward Under-

———. 2014. School Enrollment: CPS October 2014—

standing Homelessness: The 2007 National Symposium on

Detailed Tables. https://www.census.gov/hhes/school/data/

Homelessness Research. Washington, DC: U.S. Department of

cps/2014/tables.html#NAV_1460391502_6.

Health and Human Services, Assistant Secretary for Planning

———. 2013. U.S. Rental Housing Finance Survey.

and Evaluation; U.S. Department of Housing and Urban

huduser.gov/portal/datasets/rhfs/home.html#about.

Development, Office of Policy Development and Research.

U.S Department of Housing and Urban Development.

Schochet, Peter. 2009. “An Approach for Addressing the

Forthcoming. Annual Homeless Assessment Report (AHAR)

Multiple Testing Problem in Social Policy Impact Evalua-

to Congress: Part 2 – Estimates of Homelessness in the U.S.

tions,” Evaluation Review 33 (6): 539–567.

Washington, DC: U.S. Department of Housing and Urban

Skinner, Harvey A. 1982. “The Drug Abuse Screening Test,”

Development.

Addictive Behavior 7 (4): 363–371.

U.S. Department of Housing and Urban Development,

Snyder, C. Richard, Betsy Hoza, William E. Pelham, Michael

Office of Public and Indian Housing (HUD-PIH). 2004.

Rapoff, Leanne Ware, Michael Danovsky, Lori Highberger,

“Form HUD-50058 Instruction Booklet.” http://www.hud.

Howard Ribenstein, and Kandy J. Stahl. 1997. “The Devel-

gov/offices/pih/systems/pic/50058/pubs/ib/ib_final_0601.

opment and Validation of the Children’s Hope Scale,” Journal

pdf accessed 10/5/2016.

of Pediatric Psychology 22 (3): 399–421.

United States Interagency Council on Homelessness.

Snyder, C. Richard, Susie C. Sympson, Florence C. Ybasco,

(2015). Opening Doors: Federal Strategic Plan to Prevent and

Tyrone F. Borders, Michael A. Babyak, and Raymond L.

End Homelessness. Accessed September 15, 2016. https://

Higgins. 1996. “Development and Validation of the State

www.usich.gov/resources/uploads/asset_library/USICH_

Hope Scale,” Journal of Personality and Social Psychology 70:

OpeningDoors_Amendment2015_FINAL.pdf.

321–335.

White, Halbert. 1980. “A Heteroskedasticity-Consistent Co-

Solari, Claudia D., and Jill Khadduri. Forthcoming. Family

variance Matrix Estimator and a Direct Test for Heteroske-

Options Study: How Families Use Housing Choice Vouch-

dasticity,” Econometrica 48: 817–830.

ers. Short paper. Washington, DC: U.S. Housing and Urban

———. 1984. Asymptotic Theory for Econometricians.

Development.

Orlando, FL: Academic Press.

Spellman, Brooke, Jill Khadduri, Brian Sokol, and Joshua

Wood, Michelle, Marybeth Shinn, Scott R. Brown, Douglas

Leopold. 2010. Costs Associated With First-Time Homelessness

Walton and Daniel Gubits. Forthcoming. Community-Based

for Families and Individuals. Prepared for the U.S. Depart-

Rapid Re-housing. Short paper. US Department of Housing

ment of Housing and Urban Development, Office of Policy

and Urban Development.

Development and Research. Washington, DC: Government

Printing Office.

Woodcock, Richard W., Kevin S. McGrew, and Nancy Mather.

2001. Woodcock-Johnson III Tests of Achievement. Rolling

Spielberger, Charles D., Richard L. Gorsuch, Robert E.

Meadows, IL: Riverside.

Lushene, Peter R. Vagg, and G. A. Jacobs. 1973. State-Trait

Anxiety Inventory for Children. Palo Alto, CA: Mind Garden.

World Health Organization (WHO). 2011. “International

Statistical Classification of Diseases and Related Health

Squires, Jane, Diane Bricker, Elizabeth Twombly, LaWanda

Problems.” 10th Revision. Vol. 2. Instruction Manual. http://

Potter. (2009). ASQ-3 User’s Guide. Baltimore, MD: Paul H.

www.who.int/classifications/icd/ICD10Volume2_en_2010.

Brookes Publishing Co., Inc.

pdf?ua=1.

United States Census Bureau. 2013. Poverty Thresholds:

Youth in Mind. 2012. “Information for Researchers and

Preliminary Estimate of Weighted Average Poverty Thresholds

Professionals About the Strengths & Difficulties Question-

for 2015. https://www.census.gov/hhes/www/poverty/data/

naires.” http://www.sdqinfo.com/.

threshld/. Accessed April 4, 2016.

3-Year Impacts of Housing and Services Interventions for Homeless Families

129

FAMILY OPTIONS STUDY

CHAPTER 12.

ADDITIONAL READING

Adam, Emma K. 2004. “Beyond Quality: Parental and

Beatty, Alexandra S. 2010. Student Mobility Exchange: Explor-

Residential Stability and Children’s Adjustment,” Current

ing the Impact of Frequent Moves on Achievement: Summary of a

Directions in Psychological Science 13: 210–213.

Workshop. Washington, DC: National Research Council and

Institute of Medicine, Committee on the Impact of Mobility

Allison, Paul D. 2002. Missing Data. University Paper No.

and Change on the Lives of Young Children, School, and

136. Thousand Oaks, CA: SAGE Publications.

Neighborhoods.

Angrist, Joshua D., Guido W. Imbens, and Donald B. Rubin.

Buckner, John C. 2008. “Understanding the Impact of

1996. “Identification of Causal Effects Using Instrumental

Homelessness on Children: Challenges and Future Research

Variables,” Journal of the American Statistical Association 91:

Directions,” American Behavioral Scientist 51: 721–736.

444–472.

Buckner, John C., and Ellen L. Bassuk. 1997. “Mental Disor-

Angrist, Joshua D., and Alan B. Krueger. 1992. “The Effect

ders and Service Utilizations Among Youths From Homeless

of Age at School Entry on Educational Attainment: An

and Low-Income Housed Families,” Journal of American

Application of Instrumental Variables With Moments From

Academy of Child and Adolescent Psychiatry 36 (7): 890–900.

Two Samples,” Journal of the American Statistical Association

87 (418): 328–337.

Buckner, John C., Ellen L. Bassuk, Linda Weinreb, and Mar-

garet Brooks. 1999. “Homelessness and Its Relation to the

———. 1995. “Split-Sample Instrumental Variables Esti-

Mental Health and Behavior of Low-Income School Aged

mates of the Return to Schooling,” Journal of Business and

Children,” Developmental Psychology 35: 246–257.

Economic Statistics 13 (2): 225–235.

Buckner, John C., William R. Beardslee, and Ellen L. Bassuk.

Bassuk, Ellen L., John C. Buckner, Linda F. Weinreb, Angela

2004. “Exposure to Violence and Low-Income Children’s

Browne, Shari S. Bassuk, Ree Dawson, and Jennifer N.

Mental Health: Direct, Moderated, and Mediated Relations,”

Perloff. 1997. “Homelessness in Female-Headed Families:

American Journal of Orthopsychiatry 74: 413–423.

Childhood and Adult Risk and Protective Factors,” American

Journal of Public Health 87 (2): 241–248.

Burt, Martha R., Laudan Y. Aron, Toby Douglas, Jesse Valente,

Edgar Lee, and Britta Iwen. 1999. Homelessness: Programs

Bassuk, Ellen L., Carmela J. DeCandia, Alexander

and the People They Serve. Findings of the National Survey

Tsertsvadze, and Molly K. Richard. 2014. “The Effectiveness

of Homeless Assistance Providers and Clients. Prepared for

of Housing Interventions and Housing and Service Inter-

the Interagency Council on Homelessness. Washington, DC:

ventions on Ending Family Homelessness: A Systematic Re-

Government Printing Office.

view,” American Journal of Orthopsychiatry 84 (5): 457–474.

Burt, Martha R., Carol L. Pearson, and Ann Elizabeth

Bassuk, Ellen L., Jennifer N. Perloff, and Ree Dawson. 2001.

Montgomery. 2005. Strategies for Preventing Homelessness.

“Multiply Homeless Families: The Insidious Impact of Vio-

Prepared for the U.S. Department of Housing and Urban

lence,” Housing Policy Debate 12 (2): 299–320.

Development, Office of Policy Development and Research.

Bassuk, Ellen L., Lenore L. Rubin, and Alison M. Lauriat.

Washington, DC: Government Printing Office.

1986. “Characteristics of Sheltered Homeless Families,”

Burt, Martha R., Dave Pollack, Abby Sosland, Kelly S.

American Journal of Public Health 76 (9): 1097–1101.

Mikelson, Elizabeth Drappa, Kristy Greenwalt, and Patrick

Bassuk, Ellen L., Linda F. Weinreb, John C. Buckner, Angela

Sharkey. 2002. Evaluation of Continuums of Care for Homeless

Browne, Amy Salomon, and Shari S. Bassuk. 1996. “The

People. Washington DC: Urban Institute; Fairfax, VA: ICF

Characteristics and Needs of Sheltered Homeless and Low-

Consulting.

Income Housed Mothers,” Journal of the American Medical

Association 276 (8): 640–646.

3-Year Impacts of Housing and Services Interventions for Homeless Families

130

FAMILY OPTIONS STUDY

Chapter 12. Additional Reading

Burt, Martha R., and Brooke Spellman. 2007. Changing

Fantuzzo, John W., Whitney A. LeBoeuf, Chin-Chin Chen,

Homeless and Mainstream Service Systems: Essential Approaches

Heather L. Rouse, and Dennis P. Culhane. 2012. “The

to Ending Homelessness. Developed for the National Sympo-

Unique and Combined Effects of Homelessness and School

sium on Homelessness Research. Washington, DC: Govern-

Mobility on the Educational Outcomes of Young Children,”

ment Printing Office.

Educational Researcher 41 (9): 393–402.

Chen, Edith. 2004. “Why Socioeconomic Status Affects the

Finkel, Meryl, Meghan Henry, Natalie Matthews, Brooke

Health of Children: A Psychosocial Perspective,” Current

Spellman, and Dennis Culhane. Forthcoming. Evaluation of

Directions in Psychological Science 13 (3): 112–115.

the Rapid Re-housing Demonstration Program: Final Report.

Prepared for the U.S. Department of Housing and Urban

Chetty, Raj, and Nathanial Hendren. 2015. The Impacts of

Development, Office of Policy Development and Research.

Neighborhoods on Intergenerational Mobility: Childhood

Washington, DC: Government Printing Office.

Exposure Effects and County-Level Estimates. Unpublished

paper.

Glazerman, Steven, Dan Levy, and David Myers. 2003.

Nonexperimental Versus Experimental Estimates of Earnings

Chetty, Raj, Nathanial Hendren, and Lawrence F. Katz.

Impacts. Unpublished paper submitted to the Annals of the

2015. The Effects of Exposure to Better Neighborhoods on

Academy of Political and Social Sciences.

Children: New Evidence from the Moving to Opportunity

Experiment. NBER Working Paper No. 21156. Cambridge,

Hayes, Maureen A., Megan Zonneville, and Ellen L. Bassuk.

MA: National Bureau of Economic Research.

2013. The SHIFT Study: Final Report. Waltham, MA: The

National Center on Family Homelessness.

Chetty, Raj, Nathaniel Hendren, and Lawrence Katz. Forth-

Herbers, Janelle E., J.J. Cutuli, Laura M. Supkoff, David Heistad,

coming. The Effects of Exposure to Better Neighborhoods on

Chi-Keug Chan, Elizabeth Hinz, and Ann S. Masten. 2012.

Children: New Evidence From the Moving to Opportunity

“Early Reading Skills and Academic Achievement Trajectories

Experiment. American Economic Review.

of Students Facing Poverty, Homelessness and High Residen-

Coley, Rebekah Levine, Tama Leventhal, Alicia Doyle Lynch,

tial Mobility,” Educational Researcher 41 (9): 366–374.

and Melissa Kull. 2013. “Relations Between Housing Char-

Huntington, Nicholas, John C. Buckner, and Ellen L. Bassuk.

acteristics and the Well-Being of Low-Income Children and

2008. “Adaptation in Homeless Children: An Empirical

Adolescents,” Developmental Psychology 49 (9): 1775–1789.

Examination Using Cluster Analysis,” American Behavioral

Cowal, Kristin, Marybeth Shinn, Beth C. Weitzman, Daniela

Scientist 51 (6): 737–755.

Stojanovic, and Larissa Labay. 2002. “Mother-Child Sepa-

Institute for Children and Poverty. 2010. “National Survey

rations Among Homeless and Housed Families Receiving

of Programs and Services for Homeless Families.” http://

Public Assistance in New York City,” American Journal of

www.icphusa.org/PDF/reports/ICP_Massachusetts_Brief.pdf.

Community Psychology 30 (5): 711–730.

Jacob, Brian, Max Kapustin, and Jens Ludwig. 2014. Human

Culhane, Dennis P. 1992. “The Quandaries of Shelter Re-

Capital Effects of Anti-Poverty Programs: Evidence from

form: An Appraisal of Efforts to ‘Manage’ Homelessness,”

a Randomized Housing Voucher Lottery. NBER Working

Social Service Review 66: 428–440.

Paper 20164. http://www.nber.org/papers/w20164.

Culhane, Dennis P., Wayne D. Parker, Barbara Poppe,

Jacob, Brian A., and Jens Ludwig. 2012. “The Effects of Hous-

Kennen S. Gross, and Ezra Sykes. 2007. Accountability, Cost-

ing Assistance on Labor Supply: Evidence from a Voucher

Effectiveness, and Program Performance: Progress Since 1998.

Lottery,” American Economic Review 102 (1): 272–304.

Developed for the National Symposium on Homelessness

Khadduri, Jill. 2008. Housing Vouchers Are Critical for Ending

Research. Washington, DC: Government Printing Office.

Family Homelessness. Washington, DC: Homelessness Re-

Eggers, Fredrick J., and Fouad Moumen. 2013. Analysis

search Institute.

of Trends in Household Composition Using American Housing

Kish, Leslie. 1965. Survey Sampling. New York: Wiley.

Survey Data. Prepared for the U.S. Department of Housing

and Urban Development, Office of Policy Development and

Locke, Gretchen, Jill Khadduri, and Ann O’Hara. 2007.

Research. Washington, DC: Government Printing Office.

“Housing Models.” In Toward Ending Homelessness: The 2007

National Symposium on Homelessness Research, edited by

Evans, Gary. 2004. “The Environment of Child Poverty,”

Locke, Gretchen, Jill Khadduri, and Ann O’Hara. Washing-

American Psychologist 55: 77–92.

ton, D.C.: U.S. Department of Housing and Urban Develop-

ment Office of Policy Development and Research.

3-Year Impacts of Housing and Services Interventions for Homeless Families

131

FAMILY OPTIONS STUDY

Chapter 12. Additional Reading

Lowenstein, Amy E., Noemi Altman, Patricia M. Chou, Kristen

Northwest Institute for Children and Families. 2007. Evalu-

Faucetta, Adam Greeney, Daniel Gubits, Jorgen Harris, JoAnn

ation of the Sound Families Initiative: Final Findings Summary:

Hsueg, Erika Lundquist, Charles Michalopoulos, and Vinh Q.

A Closer Look at Families’ Lives During and After Supportive

Nguyen. 2014. A Family-Strengthening Program for Low-Income

Transitional Housing. Seattle: University of Washington

Families: Final Impacts from the Supportive Healthy Marriage

School of Social Work.

Evaluation, Technical Supplement. OPRE Report 2014-09B.

Obradović, Jolena, Jeffrey D. Long, J.J. Cutuli, Chi-Keung

Washington, DC: U.S. Department of Health and Human

Chan, Elizabeth Hinz, David Heistad, and Ann S. Masten.

Services, Administration of Children and Families, Office of

2009. “Academic Achievement of Homeless and Highly

Planning, Research and Evaluation.

Mobile Children in an Urban School District: Longitudinal

Masten, Ann S., Janette E. Herbers, Christopher D. Desjardins,

Evidence on Risk, Growth, and Resilience,” Development and

J.J. Cutuli, Christopher M. McCormick, Julianna K. Sapi-

Psychopathology 21 (02): 493–518.

enza, Jeffrey D. Long, and Philip D. Zelazo. 2012. “Executive

O’Flaherty, Brendan. 2009. Homelessness as Bad Luck: Implica-

Function Skills and School Success in Young Children

tions for Research and Policy. New York: Columbia University.

Experiencing Homelessness,” Educational Researcher 41 (9):

375–384.

Orr, Larry L. 1999. Social Experiments: Evaluating Public Pro-

grams With Experimental Methods. Thousand Oaks, CA: Sage.

Masten, Ann S., Donna Miliotis, Sandra A. Graham-Bermann,

Mary Louise Ramirez, and Jennifer Neemann. 1993. “Children

Park, Jung M., Stephen Metraux, Gabriel Broadbar, and Den-

in Homeless Families: Risks to Mental Health and Devel-

nis P. Culhane. 2004. “Child Welfare Involvement Among

op ment,” Journal of Consulting and Clinical Psychology 61:

Children in Homeless Families,” Child Welfare 83: 423–436.

335–343.

Pribesh, Shana, and Douglas B. Downey. 1999. “Why Are

Matsudaira, Jordan D., and Rebecca M. Blank. 2013. “The

Residential and School Moves Associated with Poor School

Impact of Earnings Disregards on the Behavior of Low-Income

Performance?” Demography 36 (4): 521–534.

Families,” Journal of Policy Analysis and Management 33 (1):

7–35.

Puma, Michael J., Robert B. Olsen, Stephen H. Bell, and

Cristofer Price. 2009. “What To Do When Data Are Missing

Mayberry, Lindsay S., Marybeth Shinn, Jessica Gibbons

in Group Randomized Controlled Trials.” NCEE 2009-0049.

Benton, and Jasmine Wise. 2014. “Families Experiencing

Washington, DC: U.S. Department of Education, Institute of

Housing Instability: The Effects of Housing Programs on

Education Sciences, National Center for Education Evalua-

Family Routines and Rituals,” American Journal of Orthopsy-

tion and Regional Assistance.

chiatry 84 (1): 95–109.

Rafferty, Yvonne, and Marybeth Shinn. 1991. “The Impact

Mayfield, Jim, Callie Black, and Barbara E.M. Felver. 2012.

of Homelessness on Children,” American Psychologist 46:

Employment Outcomes Associated with Rapid Re-Housing Assis-

1170–1179.

tance for Homeless DSHS Clients in Washington State. Seattle:

Washington State Department of Social and Health Services.

Rafferty, Yvonne, Maybeth Shinn, and Beth C. Weitzman.

2004. “Academic Achievement Among Formerly Homeless

McLoyd, Vonnie C. 1990. “The Impact of Economic Hard-

Adolescents and Their Continuously Housed Peers,” Journal

ship on Black Families and Children: Psychological Distress,

of School Psychology 42 (3): 179–199.

Parenting, and Socioemotional Development,” Child Devel-

opment 61 (2): 311–346.

Rodriguez, Jason. 2013. Homelessness Recurrence in Georgia.

Atlanta: Georgia Department of Community Affairs, State

McLoyd, Vonnie C. 1998. “Socioeconomic Disadvantage and

Housing Trust Fund for the Homeless.

Child Development,” American Psychologist 53: 185–204.

Rog, Debra J., Kimberly McCombs-Thornton, Ariana M.

Mehana, Majida, and Arthur J. Reynolds. 2004. “School

Gilbert-Mongelli, Consuelo Brito, and C. Scott Holupka.

Mobility and Achievement: A Meta-Analysis,” Children and

1995. “Implementation of the Homeless Families Program:

Youth Services Review 26: 93–119.

2. Characteristics, Strengths, and Needs of Participant Fami-

lies,” American Journal of Orthopsychiatry 65 (4): 514–528.

National Alliance to End Homelessness (NAEH). 2012. “Rap-

id Re-Housing Successes.” http://www.endhomelessness.org/

library/entry/rapid-re-housing-successes.

3-Year Impacts of Housing and Services Interventions for Homeless Families

132

FAMILY OPTIONS STUDY

Chapter 12. Additional Reading

Rog, Debra J., and Marjorie Gutman. 1997. “The Homeless

Tsai, Jack, Wesley Kasprow, and Robert Rosenheck. 2011.

Families Program: A Summary of Key Findings.” In To Im-

“Exiting Homelessness Without a Voucher: A Comparison

prove Health and Health Care: The Robert Wood Johnson Foun-

of Independently Housed and Other Homeless Veterans,”

dation Anthology, edited by Stephen L. Isaacs and James R.

Psychological Services 8 (2): 114–122.

Knickman. San Francisco: Jossey-Bass Publishers: 209–231.

United States Interagency Council on Homelessness

Rog, Debra J., and Frances L. Randolph. 2002. “A Multisite

(USICH). 2010. “Opening Doors: Homelessness Among

Evaluation of Supported Housing: Lessons Learned from

Families.” http://www.usich.gov/usich_resources/fact_sheets/

Cross-Site Collaboration,” New Directions for Evaluation 94:

opening_doors_homelessness_among_families_fact_sheet/.

61–72.

Accessed September 2015.

Shinn, Marybeth, Andrew L. Greer, Jay Bainbridge, Jonathan

Voight, Adam, Marybeth Shinn, and Maury Nation. 2012.

Kwon, and Sara Zuiderveen. 2013. “Efficient Targeting of

“The Longitudinal Effects of Residential Mobility on the

Homelessness Prevention Services for Families,” American

Academic Achievement of Urban Elementary and Middle

Journal of Public Health 103 (S2): S324–S330.

School Students,” Educational Researcher 41 (9): 385–392.

Shinn, Marybeth, Judith S. Schteingart, Nathaniel Chioke

Waldfogel, Jane. 2001. “International Policies Toward Paren-

Williams, Jennifer Carlin-Mathis, Nancy Bialo-Karagis, Rachel

tal Leave and Child Care,” The Future of Children 11 (1): 99–111.

Becker-Klein, and Beth C. Weitzman. 2008. “Long-Term

Weinreb, Linda, Robert Goldberg, Ellen L. Bassuk, and Jen-

Associations of Homelessness with Children’s Well-Being,”

nifer Perloff. 1998. “Determinants of Health and Service Use

American Behavioral Scientist 51: 789–810.

Patterns in Homeless and Low-Income Housed Children,”

Shinn, Marybeth, Beth C. Weitzman, Daniela Stojanovic,

Pediatrics 102: 562.

James R. Knickman, Lueila Jimenez, Lisa Duchon, Susan

Weitzman, Beth C. 1989. “Pregnancy and Childbirth: Risk

James, and David H. Krantz. 1998. “Predictors of Home-

Factors for Homelessness?” Family Planning Perspectives 21

lessness Among Families in New York City: From Shelter

(4): 175–178.

Request to Housing Stability,” American Journal of Public

Health 88 (11): 1651–1657.

Weitzman, Beth C., and Carolyn Berry. 1994. Formerly

Homeless Families and the Transition to Permanent Housing:

Shinn, Marybeth. 2014. Most Homeless Families Just Need

High-Risk Families and the Role of Intensive Case Management

Affordable Housing. Washington, DC: National Alliance to

Services. Final Report to the Edna McConnell Clark Foun-

End Homelessness.

dation. New York: New York University, Robert F. Wagner

Snyder, C. Richard, Lori M. Irving, and John R. Anderson.

Graduate School of Public Service, Health Research Program.

1991. “Hope and Health: Measuring the Will and the Ways.”

Westfall, Peter H., Randall Tobias, and Russell Wolfinger.

In The Handbook of Social and Clinical Psychology: The Health

2011. Multiple Comparisons and Multiple Tests Using SAS.

Perspective, edited by C. Richard Snyder and Donelson R.

Cary, NC: SAS Institute.

Forsyth. Elmsford, New York: Pergamon Press: 285–307.

Wong, Yin-Ling, Dennis P. Culhane, and Randall Kuhn. 1997.

Snyder, Thomas D., and Sally A. Dillow. 2013. Digest of

“Predictors of Exit and Reentry Among Family Shelter Users

Education Statistics 2012. NCES 2014-05. U.S. Department

in New York City,” Social Service Review 71 (3): 441–462.

of Education, Institute of Education Sciences, National Cen-

ter for Education Statistics. Washington, DC: Government

Wood, Michelle, Jennifer Turnham, and Gregory Mills.

Printing Office.

2008. “Housing Affordability and Family Well-Being: Re-

sults from the Housing Voucher Evaluation,” Housing Policy

Steffen, Barry L., Shaun Bucholtz, Marge Martin, David A.

Debate 19 (2): 367–412.

Vandenbroucke, and Yunn-Gann David Yao. 2013. Worst

Yoshikawa, Hirokazv, J. Lawrence Aber, and William R. Beard-

Case Housing Needs 2011: Report to Congress. Prepared for

slee. 2012. “The Effects of Poverty on the Mental, Emotional,

the U.S. Department of Housing and Urban Development,

and Behavioral Health of Children and Youth: Implications

Office of Policy Development and Research. Washington,

for Prevention,” American Psychologist 67: 272–284.

DC: Government Printing Office.

Yudko, Errol, Olga Lozhkina, and Adriana Fouts. 2007. “A

Thompson, Dianne T. 2007. “Evaluating Length of Stay in

Comprehensive Review of the Pyschometric Properties of

Assisted Housing Programs: A Methodological Note,” City-

the Drug Abuse Screening Test,” Journal of Substance Abuse

scape 9 (1): 217–238.

Treatment 32: 189–198.

3-Year Impacts of Housing and Services Interventions for Homeless Families

133

FAMILY OPTIONS STUDY

APPENDIX A.

DATA SOURCES AND DATA SET

CONSTRUCTION

This appendix describes the data sources, data collec- study uses data from study families, intervention providers, and tion procedures, completion rates, and data process-administrative data systems (see Exhibit A-1).

ing procedures used in the Family Options Study. The

Exhibit A-1. Data Sources Used in the Study

Data Source

Collection Process

Data Source Collects or Measures…

From study implementation

Random assignment

• Recorded in web-based enrollment and random

• Name, date of birth, and Social Security number of family

enrollment data (n = 2,282)

assignment tool, based on information entered by field

head and spouse or partner

interviewer and point-in-time intervention availability

• Eligibility screening responses

• Intervention availability at random assignment

• Random assignment result

From study families

Baseline survey (n = 2,282)

• In-person survey (40 minutes) conducted immediately

• Demographic characteristics

before random assignment

• Preshelter housing

• Completed for the full sample of families randomly

• Housing barriers

assigned

• Homelessness history

• Employment

• Family composition

• Income and income sources

• Family head: physical health

• Family head: mental health, experiences of trauma, and

other psychosocial challenges

6-, 12-, and 27- month

• Telephone survey (10 minutes) conducted 6, 12, and 27 • Family composition

tracking surveys (6-month

months after random assignment

• Current housing status

n = 1,671; 12-month

• Use of homeless and housing programs

n = 1,632; 27-month

n = 1,159)

20-month and 37-month

• In-person or telephone survey (60 minutes) conducted

• Current housing status

followup adult surveys

at least 18 months after random assignment (July 2012

• Experience of homelessness

(20-month n = 1,857;

to October 2013)

• Use of homeless and housing programs

37-month n = 1,784)

• In-person or telephone survey (60 minutes) conducted

• Housing quality and affordability of current unit

at least 32 months after random assignment (March

• Employment and earnings

2014 to December 2014)

• Income and income sources

• Material hardship

• Family composition and preservation

• Adult well-being

• Child well-being (for up to two focal children)

• Receipt of services

20-month and

• In-person child assessments (50 minutes) conducted

• Verbal ability (Woodcock-Johnson III letter-word identification

37-month followup child

for focal children who were ages 3 years, 6 months to

test; 20-month n = 876; 37-month n = 832)

assessments

7 years, 11 months

• Math ability (Woodcock-Johnson III applied problems test;

• Collection attempted only if family head responded to

20-month n = 846; 37-month n = 833)

followup adult survey

• Self-regulation (Head Toes Knees Shoulders assessment;

20-month n = 780; 37-month n = 798)

20-month and 37-month

• In-person or telephone survey (30 minutes) conducted

• Mental health

followup child survey

for focal children who were ages 8 to 17 years

• Experiences of traumatic events

(20-month n = 930;

• Collection attempted only if family head responded to

• Substance use

37-month n = 1,058)

followup adult survey

• School effort

• Arrests or police involvement

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-1. Data Sources Used in the Study (continued)

Data Source

Collection Process

Data Source Collects or Measures…

From study intervention providers

Enrollment verification

• Study team verified (by telephone and e-mail) whether

• Use of assigned intervention program

data

families enrolled in the programs to which they were

referred

• Conducted from September 2010 to September 2012

Program information

• Study team conducted site visits and staff interviews

• Provider information

• Conducted from June 2011 to April 2012

• Characteristics of housing assistance

• Characteristics of services

Program cost information

• Study team conducted site visits and staff interviews

• Overhead costs

• Collected audited expense statements, program

• Rental assistance costs

budgets, staffing lists, partner commitment letters,

• Facility operations costs

and program staff estimates of costs not reflected in

• Supportive services costs

expense statements

• Capital costs

• Conducted from November 2012 to August 2013

From administrative data systems

Homeless Management

• Individual-level records collected from community and

• Participation in homeless assistance programs covered

Information System (HMIS)

government administrators of the HMIS (one or more

in HMIS at the site where families enrolled (including

per site)

emergency shelter, rapid re-housing, transitional housing,

and permanent supportive housing)

HUD Public and Indian

• Individual-level data collected from HUD

• Receipt of housing assistance through HUD’s Housing

Housing Information

Choice Voucher and public housing programs

Center (PIC)

HUD Tenant Rental

• Individual-level data collected from HUD

• Receipt of housing assistance through project-based

Assistance Certification

Section 8 programs

System (TRACS)

State and local child

• Individual-level data collected from state and local

• Formal foster care placements and adoptions

welfare agency records

child welfare agencies (use Adoption and Foster Care

Analysis and Reporting System definitions)

National Directory of New

• Individual-level data collected from centralized system

• Quarterly earnings records

Hires

administered by the U.S. Department of Health and

Human Services Office of Child Support Enforcement

(OCSE) through agreement between HUD and OCSE

From combination of sources

Program Usage Data

• Combines data from nine sources: enrollment

• Participation in seven types of homeless and housing

verification; 6-, 12-, 20-, 27- and 37-month surveys;

assistance programs (by calendar month after random

HMIS; HUD PIC; and TRACS

assignment)

Notes: All surveys conducted with family head collected or updated family contact information for tracking purposes. Additional information about program cost data collection provided in Appendix G. Child assessment and child survey sample sizes are number of nonmissing observations in analysis data and exclude collected data that were not usable for analysis.

A.1. Random Assignment Data

• Document that the baseline survey was complete.

The study team created a secure website to support the

• Randomly assign the family to available intervention groups.

enrollment and random assignment of families into the Family

The Interim Report: Family Options Study provides additional

Options Study. Local site interviewers used the random

details about the enrollment process.

assignment website to—

• Document that the adult respondent provided informed

Family Options Study Sample

consent.

The study enrolled 2,282 families across 12 sites between Sep-

• Enter the personal identifiers for the adult respondent and a

tember 2010 and January 2012. Exhibit A-2 shows the timing

spouse/partner, if applicable.

of sample enrollment and the enrollment numbers by site and

intervention group. Of the 2,282 families who enrolled in the

• Check that intervention providers had openings available

study, 1,784 completed the 37-month adult survey.

in their programs, making it possible to conduct random

assignment.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-2. Sample Enrollment Period and Number of Families Enrolled by Intervention and Site

Families Randomly Assigned, by Intervention (N)

Site

Enrollment Period

Total (N)

CBRR

PBTH

SUB

UC

Alameda County

Sep 2010–Jan 2012

56

49

76

77

258

Atlanta

Oct 2010–Jan 2012

73

41

—

75

189

Baltimore

Mar 2011–Jan 2012

20

17

—

21

58

Boston

Feb 2011–Jan 2012

53

—

64

64

181

Connecticut*

Oct 2010–Dec 2011

73

18

47

76

214

Denver

Jan 2011–Jan 2012

8

23

76

65

172

Honolulu

Oct 2010–Jan 2012

44

66

43

65

218

Kansas City

Oct 2010–Jan 2012

30

42

53

50

175

Louisville

Apr 2011–Jan 2012

18

24

32

35

109

Minneapolis

Nov 2010–Jan 2012

52

4

62

63

181

Phoenix

Oct 2010–Dec 2011

62

65

71

81

279

Salt Lake City

Sep 2010–Oct 2011

80

19

75

74

248

Total

569

368

599

746

2,282

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

* Includes the cities of New Haven and Bridgeport, Connecticut.

Source: Random assignment records

Eligibility Determination

conducted enrollment. The interviewer informed families about

the study. If the family consented to participate in the study,

To maximize the likelihood that families would be accepted by

the interviewer then asked eligibility screening questions for

the assigned intervention program, the study team conducted

programs that had available program slots. If eligible for avail-

screening before random assignment. The study team collected

able interventions,2 the interviewer then administered the base-

each program’s eligibility requirements and developed eligi-

line survey using Computer-Assisted Personal Interviewing,

bility screening questions. The study team administered the

or CAPI, software. On average, it took families 40 minutes to

eligibility screening questions to families after informed consent

complete the baseline survey. The baseline survey covered fam-

but before random assignment. For each family, the study

ily composition, demographic characteristics, housing stability,

team asked only the eligibility screening questions relevant to

history of homelessness, employment, income, and health. The

the programs in that site that had openings available. A family

study team collected baseline survey data in the shelter where

was eligible for random assignment to an intervention if the

the family was staying at the time of random assignment.

adult respondent’s answers to the screener questions met the

eligibility requirements for at least one participating provider

In families with only one adult present, that individual was

of that intervention with an available program slot at the time

interviewed. For families headed by couples, the study team

of random assignment. The screener questions improved the

requested to interview the woman. Two reasons explain this

likelihood that families would be eligible for the assigned

preference: (1) some homeless assistance programs exclude

intervention.1 The study team retained data on the eligibility

men, and in cases of family separations the children are more

screening response. The Interim Report provides additional

likely to remain with the mother; and (2) some outcome meas-

details about eligibility determination.

ures such as psychological distress have different distributions

for men and women in the population at large, so this prefer-

A.2. Baseline Data Collection

ence results in having greater homogeneity in the sample.3

The covariates, discussed in Appendix C.1, were derived from

Study enrollment took place in the emergency shelters where

the baseline survey responses. The Interim Report provides

the families were staying. Enrollment began in September 2010

further details about the baseline survey and data collection.

and was completed in January 2012. Local field interviewers

1 After random assignment and referral to a program, families were required to complete the program’s regular eligibility determination process, including, in some cases, criminal background checks, drug testing, and income verification.

2 Initially, families had to eligible for available program slots in at least two active interventions in order to proceed with random assignment. In August 2011, this rule was changed so that families needed only to be eligible for an available program slot in only one active intervention in order to proceed with random assignment.

3 In the full study sample of 2,282 families, 524 adult baseline respondents of the 626 families headed by couples (84 percent) were women. In the 37-month respondent sample of 1,784 families, 398 baseline adult respondents of the 471 families headed by couples at baseline (85 percent) were women.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

A.3. Enrollment Verification Data

A.5. 20- and 37-Month Followup

Adult Surveys

The study team collected information from study programs to

document enrollment in the assigned intervention. The study

The 20- and 37-month followup data collection efforts each

team contacted study programs regularly (weekly or monthly)

included an adult survey, child survey, and child assessments.

to inquire about the status of families who had been referred

The 20- and 37-month adult surveys collected information on

to their programs after random assignment. This information is

these topics about the adults.

referred to as the enrollment verification data. The study team

• Housing situation.

collected the following information.

• Housing quality and affordability.

• Whether the study family made contact with the program to

• Employment.

which they were referred.

• Income source and total family income.

• Whether the family was accepted by the program (enrolled).

• Education and training.

• Whether the family actually moved into a housing unit using

• Economic hardship.

that assistance.

• Food security.

• For families who were accepted by the program but did not

• Family composition.

move in, the reason for not using the assistance.

• Family separation and reunification.

The calls were made throughout the enrollment period—

• Physical health.

September 2011 to January 2012—and continued through

• Behavioral health.

September 30, 2012, 9 months after the last family enrolled.

These enrollment verification data were used in the Program

• Substance use.

Usage Data file, discussed in Section A.13. The Interim Report

• Service receipt.

provides additional details about the enrollment verification

process.

The adult survey also asked several questions about the focal

children in a parent-on-child module. The parent-on-child

module asked about these topics—

A.4. 6-, 12-, and 27-Month Tracking

• Child education.

Surveys

• Child health.

During the followup period the study team conducted brief

• Child behavior.

tracking surveys 6, 12, and 27 months after enrollment.

• Family routines.

These surveys lasted an average of 10 minutes and collected

updated contact information for the adult respondent and

The study attempted to complete the 37-month adult survey

secondary contacts. The tracking surveys also collected data

with all 2,282 family heads, whether or not they completed a

on the current living situation, receipt of housing assistance,

survey at 20 months. For families headed by couples, the same

and family composition for each family. Local site interviewers

adult interviewed at baseline was interviewed at followup. At

administered the tracking surveys using CAPI technology. In

both 20- and 37-months, the adult survey took an average of

most sites, the interviewer was the same person who admin-

60 minutes to complete.

istered the baseline survey. Because the tracking surveys were

For the 37-month data collection, a minimum of 31.5 months

relatively short, most participants opted to complete the survey

elapsed between the date of random assignment and the date

by telephone rather than in person.

of the followup survey. The analysis period, during which all

impacts were estimated, was thus between 31.5 and 50 months

after random assignment for most families (Exhibit A-3).

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-3. Length of Time From Random Assignment to

additional focal children (for families that did not respond to

the 37-Month Followup Survey

the 20-month survey) at the time of the 37-month survey. Of

the 3,001 focal children selected in the study, 2,794 (93 per-

Duration (months)

Number of

Families

Percent

cent) were selected at the time of the 20-month survey and 207

31 to 31.99

8

0.5

(7 percent) were selected at the time of the 37-month survey.

32 to 32.99

39

2.2

33 to 33.99

99

5.6

The focal child selection process oversampled children who were

34 to 34.99

236

13.2

ages 3 to 17, and with the family at both baseline and followup,

35 to 35.99

247

13.9

in order to maximize the number of children from whom data

36 to 36.99

210

11.8

37 to 37.99

276

15.5

were directly collected (in the child assessments and child survey).

38 to 38.99

220

12.3

The oversampling criterion of being with the family at baseline

39 to 39.99

125

7.0

was included so that oversampled children would be directly

40 to 40.99

130

7.3

affected by the study’s random assignment. Children needed

41 to 41.99

92

5.2

42 to 42.99

43

2.4

to be with the family at followup for the study team to attempt

43 to 43.99

27

1.5

collection of child assessments or the child survey. (The study

44 to 44.99

8

0.5

did not attempt to locate children separated from the family.)

45 to 45.99

11

0.6

46 to 46.99

5

0.3

Two types of children were considered for focal child selection.

47 to 47.99

5

0.3

First, all children identified at baseline—those in shelter with the

48 to 48.99

1

0.1

adult at random assignment and those who were “part of the

49 to 49.99

2

0.1

Median: 37.2 months (1,130.5 days)

family” but not in shelter with the adult at enrollment—were

Mean: 37.4 months (1,137.8 days)

eligible for focal child selection. If focal child sample selection

Minimum: 31.6 months (961 days)

had been restricted to children identified at baseline, the focal

Maximum: 49.9 months (1,518 days)

child sample could have been defined before the start of the

Notes: N = 1,784. Percentages are unweighted. Month length is assumed to be

followup data collection. The study team expanded the focal

365/12 = 30.42 days. Duration is from random assignment to survey end date.

Source: Family Options Study 37-month followup data

child selection criteria, however, to include children who were

born after random assignment. The study team referred to these

A.6. Focal Child Selection

children as “newborns.” Because newborns could not be iden-

tified before the start of the followup data collection, the study

During the 20-month adult survey,4 the study team randomly

team administered the focal child selection screener during the

selected up to two focal children per family. For families who

followup survey. To ensure that all newborns had a chance to

completed both the 20- and 37-month adult surveys, the

be selected as focal children, the study team generated a ran-

study team attempted the 37-month child data collection with

domly ordered list of all the children identified at baseline plus

the focal children selected at the 20-month survey. For the

two slots for up to two newborn children. During the screener

163 37-month respondent families that did not complete the

section of the survey, the children were screened in the random

20-month survey, the survey team selected up to two focal

order for focal child selection.

children at the beginning of the 37-month adult survey. This

To be selected as a focal child, each child had to first meet

section discusses the focal child selection process and the

these two criteria.

number of focal children in the 37-month analysis sample.

1. The child was one of the following—

Focal Child Selection

a. Listed as a child on the household roster from the

To analyze impacts on child well-being, the study team selected

baseline survey.5

up to two focal children for each family who completed the

adult survey. This section describes the process for selecting

b. Identified as a newborn, by the adult respondent in the

focal children at the time of the 20-month survey and

focal child screener section.6

4 Gubits et al. (2015) analyzed short-term impacts of the interventions. The study team attempted to contact families for the study’s first followup survey beginning in the 18th month after random assignment. The median time from random assignment to the followup survey was 20 months. The followup period reported in Gubits et al.

(2015) is thus 20 months, but the followup survey is sometimes referred to as the 18-month survey.

5 On the baseline survey, the team collected children’s ages but not dates of birth, and all children on the household roster were age 17 or younger. The roster included children who the adult respondent thought were part of the family, even if they were not in the shelter with the respondent. All randomly assigned families had at least one child age 15 or younger.

6 Screener question 1: “Between [random assignment date] and [6 months before today’s date] have you (given birth to/fathered) a child?”

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

2. For those children selected at the 20-month survey, the

• Type 2: Met the minimal criteria, but did not meet the

child had to be at least 12 months of age but younger than

additional criteria.

18 years of age, as confirmed in the focal child screener. For

• Type 3: Did not meet the minimal criteria.

those children selected at the 37-month survey, the child

had to be at least 30 months of age but younger than 18

The focal child selection process worked as follows.

years of age.

• If the parent had any Type 1 children, the study team

After the potential child was confirmed eligible for selection

randomly selected one as “Focal Child A.” Next, if the parent

based on the first two criteria, the screener determined if the

had any other Type 1 or Type 2 children, the study team

adult respondent was knowledgeable enough about the child’s

randomly selected one as “Focal Child B.”

activities in the past 30 days to answer the parent-on-child

module. The screener made this determination using the next

• If the parent had no Type 1 children but did have at least

series of questions, indicating the third criterion for selection

one Type 2 child, then the study team did not select a “Focal

eligibility.

Child A” but randomly selected one Type 2 child as “Focal

Child B.” Next, if the parent had any other Type 2 children,

3. The child was one of the following—

the study team randomly selected one as “Focal Child C.”

a. Living in the same household as the adult respondent “at

• If the parent had only Type 3 children, a focal child was not

least half of the time” or “all of the time” at the followup

selected.

point.7

Focal Child A criteria excluded newborns and focal children

b. The parent spent time with the child frequently and was

who were not living with the respondent at least half of the

at least somewhat familiar with the child’s activities.8,9

time. This exclusion helped to maximize the number of families

in which focal child selection included at least one focal child

The preceding criteria constitute the minimal selection criteria.

for whom direct child data collection (child assessments or

If possible, the first focal child would also meet these additional

child survey) was possible.

criteria.

The focal child screening (confirmation/collection of date

4. The child’s age at the followup survey was greater than or

of birth and collection of information for other criteria) was

equal to 3 years, 6 months.

performed for each child in turn, following the randomly

5. The child was living with the parent in the shelter at baseline.

ordered list, until two focal children were selected. After two

focal children were selected, the focal child screening ceased.

6. The child was living in the same household as the parent

Therefore, collection of information for screening criteria other

“at least half of the time” or “all of the time” at the followup

than date of birth was not performed for every child in the

survey.10

respondent study families.

Potential focal children were then classified into one of three

types.

Focal Children Sample Sizes

• Type 1: Met all the minimal criteria; was at least 3 years,

At the 20-month survey, 1,857 families completed the adult

6 months of age but younger than 18 years of age; was living

survey. Of these families, 1,744 had at least one focal child

in the shelter with the adult respondent at enrollment; and

selected, resulting in a total of 2,784 focal children.11 About

lived with the respondent “at least half of the time” or “all of

530 children screened for selection as focal children were

the time” at the time of the followup survey.

living with the family head less than half of the time (out of

7 The point-in-time question to the parent was, “Do you currently live in the same household as [child name] ...?” It was not a question about the entire period between baseline and followup.

8 Two criteria had to be satisfied. First, the parent spent “1 or more hours a day” with the child at least a few times a week during the month before the followup survey.

Second, during that month, the parent “always,” “usually,” or “sometimes” knew at least two of the following: (1) how the child spent time when not in school or childcare, (2) which other kids the child spent time with, (3) whether the child had finished her/his schoolwork or studying, and (4) which TV programs the child watched.

9 Although children who lived with the family less than half of the time were eligible to be selected as focal children if the parent spent time with the child frequently and the parent was at least somewhat familiar with the child’s activities, only a few such children were in the sample. Only 60 of the 2,784 focal children selected at the 20-month survey and 15 of the 207 children selected at the 37-month survey were with the family less than half of the time.

10 This criterion is the same as criterion 3a. The difference is that the minimal criteria accept either 3a or 3b, whereas the additional criteria require 3a.

11 No focal child was selected in 130 families, mainly because children were no longer residing with the respondent and the respondent did not know enough about the child’s activities during the previous 30 days to respond properly and also because the children aged out of the age range by the time of the followup survey.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

about 4,200 total children screened). Of those children, the

focal children from the impact analysis because they were not

family head was knowledgeable about only 60 of the children.

living with the family head at least half of the time when the

In accordance with the focal child selection protocol, those

adult completed the survey.12 As a result, the 37-month child

60 children were selected as focal children (along with 2,724

impact results also generalize only to children living with the

other selected focal children who were living with the family

family head half of the time or more at the time of the adult

head at least half of the time). During analysis, however, it was

survey.

decided that such a small number of children would not allow

At the 37-month followup survey, the study team selected focal

for estimates to generalize to the whole group of largely absent

children between ages 18 years and 19 years, 5 months. Parents

children. Therefore, the 60 children were not included in

were asked questions in the parent on child module about

impact analyses. As a result, the 20-month child impact results

the transition to adulthood for these older focal children. The

generalize only to children living with the family head half of

adult respondents reported on whether the focal children had

the time or more at the time of the adult survey.

children of their own, were employed, as well as their marital

All together 2,665 focal children were selected for the 1,784

status. Exhibit A-4 shows the focal child sample sizes by age

adult respondents to the 37-month survey. Of those, 207 were

group and intervention group for the 2,665 focal children at

newly selected at the 37-month survey because the family had

the 37-month followup. Details on variations in focal child

not completed the 20-month survey and 2,458 were selected

selection and corresponding child weights are included in

during the 20-month survey. The study team excluded 156

Appendix C.3.

Exhibit A-4. Focal Child Sample Distribution, by Site and Intervention Group at 37 Months Focal Child Age (N)

Total

Focal Child

2 Years to

3 Years, 6 Months 5 Years, 7 Months

8 Years to

Site Name

Intervention

18 Years to

Group

Sample

3 Years,

to 5 Years,

to 7 Years,

17 Years,

20 Yearsa

(N)

5 Months

6 Months

11 Months

11 Months

Alameda County

CBRR

7

14

12

19

2

54

PBTH

6

11

18

20

1

56

SUB

8

26

23

36

2

95

UC

5

12

17

39

1

74

Total

26

63

70

114

6

279

Atlanta

CBRR

2

16

15

46

3

82

PBTH

3

8

11

33

1

55

UC

5

13

14

47

6

85

Total

10

37

40

125

10

222

Baltimore

CBRR

1

2

7

21

0

31

PBTH

2

4

2

8

0

16

UC

2

5

4

15

0

26

Total

5

11

13

44

0

73

Boston

CBRR

5

13

17

26

5

66

SUB

13

15

17

37

4

86

UC

7

21

22

30

2

82

Total

25

49

56

93

11

234

Connecticutb

CBRR

6

20

7

37

6

76

PBTH

3

3

6

9

1

22

SUB

5

14

7

28

3

57

UC

5

16

17

38

2

78

Total

19

53

37

112

12

233

12 At the time of the 37-month survey, children in families who had not been 20-month respondents were screened for focal child selection. Focal children in families who had been 20-month respondents (and so had been previously selected) were screened to determine whether data collection should be attempted. Data collection was attempted if children were either currently living with the family at least half of the time or if the parent spent time with the child frequently and was at least somewhat familiar with the child’s activities. Of 286 children newly screened for focal child selection, 85 were living with the family less than half of the time, and 15 of them were selected as focal children (because the parent was sufficiently knowledgeable about their daily activities). In addition, 141 of the 2,458 focal children previously selected were with their families less than half of the time at 37 months. Among these children, the parent was sufficiently knowledgeable about their daily activities to be asked to provide a parent report. Therefore, of the 156 children excluded for being with the family less than half of the time, some data are available for 56 children (15 newly selected children plus 41 previously selected children).

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

7

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-4. Focal Child Sample Distribution, by Site and Intervention Group at 37 Months (continued) Focal Child Age (N)

Total

Focal Child

2 Years to

3 Years, 6 Months 5 Years, 7 Months

8 Years to

Site Name

Intervention

18 Years to

Group

Sample

3 Years,

to 5 Years,

to 7 Years,

17 Years,

20 Yearsa

(N)

5 Months

6 Months

11 Months

11 Months

Denver

CBRR

0

3

3

4

0

10

PBTH

4

7

5

10

0

26

SUB

5

20

24

42

4

95

UC

3

21

10

34

1

69

Total

12

51

42

90

5

200

Honolulu

CBRR

6

17

12

22

3

60

PBTH

8

15

25

36

2

86

SUB

8

15

15

19

2

59

UC

9

20

14

36

4

83

Total

31

67

66

113

11

288

Kansas City

CBRR

2

5

2

19

1

29

PBTH

6

6

5

29

2

48

SUB

5

16

11

34

2

68

UC

7

8

8

23

1

47

Total

20

35

26

105

6

192

Louisville

CBRR

2

4

2

9

0

17

PBTH

0

10

7

10

0

27

SUB

5

7

10

14

1

37

UC

4

9

12

20

2

47

Total

11

30

31

53

3

128

Minneapolis

CBRR

2

16

20

29

3

70

PBTH

0

1

1

4

0

6

SUB

7

18

24

27

1

77

UC

7

15

20

33

2

77

Total

16

50

65

93

6

230

Phoenix

CBRR

4

19

15

33

1

72

PBTH

7

11

14

46

5

83

SUB

5

13

22

46

5

91

UC

8

11

14

41

4

78

Total

24

54

65

166

15

324

Salt Lake City

CBRR

5

18

12

43

6

84

PBTH

2

4

5

10

0

21

SUB

5

9

19

49

4

87

UC

6

14

14

33

3

70

Total

18

45

50

135

13

262

Overall

CBRR

42

147

124

308

30

651

PBTH

41

80

99

214

12

446

SUB

66

153

172

332

29

752

UC

68

165

166

389

28

816

Grand total

217

545

561

1,243

99

2,665

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

a Includes three respondents older than 19 years, 5 months. The respondents were ages 19 years, 6 months; 19 years, 7 months; and 20 years, 0 months. No data collection was conducted on the three respondents over age 19 years, 5 months.

b Includes the cities of New Haven and Bridgeport, Connecticut.

Sources: Family Options Study baseline survey; 37-month followup survey

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

8

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Parent-on-Child Module in the 37-Month

A.8. 20- and 37-Month Followup

Adult Survey

Child Assessments

In the parent-on-child module, the adult respondent provided

The study team administered the 20-month and 37-month

information about school attendance, academic performance,

child assessment tests to focal children who were between the

behavior, health, and family routines. All focal children who

ages of 3 years, 6 months and 7 years, 11 months at the time

were the subject of parent reports at 37 months were between

of the corresponding parent survey. These assessments were

the ages of 24 months and 19 years, 5 months at the time of the

the Woodcock-Johnson III (WJ III) and the Head Toes Knees

37-month parent survey. If a focal child’s CAPI-calculated age

Shoulders (HTKS) assessments.

was 12 months to 5 years, 6 months, the interviewer admin istered

the Ages and Stages Questionnaire (ASQ-3) to the adult as part of

The WJ III assessment consisted of two tests. The first was the

the parent-on-child module. If a focal child’s CAPI-calculated

Letter-Word Identification test and the second was the Applied

age was 3 years to 17 years, 11 months, the interviewer admin-

Problems test (McGrew, Shrank, and Woodcock, 2007). These

istered the Strengths and Difficulties Questionnaire (SDQ).

tests are subtests of educational achievement from the broader

WJIII battery of tests measuring verbal and quantitative/analytic

The ASQ-3 is a family of questionnaires that assess gross and

skills. The WJ III tests were administered in person and took an

fine motor skills, social development, communication, and

average of 30 minutes to complete per child. Details on scoring

problem solving as observed by parents (Squires and Bricker,

the WJ III tests are in Appendix B.5.

2009).13 The ASQ-3 took an average of 10 minutes to complete.

The questionnaire was self-administered for in-person adult

The HTKS assessment (Pontiz et al., 2007) measures self-

surveys. For surveys conducted by phone, the questionnaire

regulation, in which children must remember rules and inhibit

was administered to the adult by phone. Details on the compo-

incorrect responses. HTKS was conducted in person and

nent questions of the ASQ-3 and scoring are in Appendix B.4.

separately for each focal child. The HTKS took an average of 15

minutes per child to administer. Details on scoring HTKS are in

The SDQ is a behavioral and personality assessment. The question-

Appendix B.5.

naire addresses child emotional symptoms, conduct problems,

hyperactivity, peer problems, and prosocial behavior. Adult

respondents completed this questionnaire for focal children

A.9. 20- and 37-Month Followup

between ages 3 years and 17 years, 11 months. Details on the

HOME Inventories

component questions of the SDQ and scoring are in Appendix B.5.

When 20- and 37-month followup surveys were conducted in

A.7. 20- and 37-Month Followup

person in the family’s home, interviewers on the study team

completed an observation form based on a subset of questions

Child Surveys

from the Home Observation for Measurement of the Envi-

ronment (HOME) inventory. The HOME inventory questions

The study team administered the 20-month and 37-month followup

used were based on observation-only items about parent-child

child surveys to focal children who were between ages 8 years

interactions.14 The questions on the HOME inventory form

and 17 years, 11 months at the time of the corresponding par-

asked about interactions between the adult taking the adult

ent survey. Interviewers administered the survey using CAPI, with

survey and each focal child.15 For purposes of the HOME

surveys lasting 30 minutes on average. Surveys were conducted

inventory tool, the interviewer was instructed to explicitly

either in person or by telephone. The child survey asked ques-

praise each child during the adult survey and observe the

tions about anxiety using the A-Trait scale from the State-Trait

adult respondent’s reaction. The interviewer praised each focal

Anxiety Inventory for Children, or STAIC (Spielberger et al.,

child up to four times throughout the survey until the adult

1973); fears (Ramirez, Masten, and Samsa, 1991); substance

expressed reaction. The HOME inventory data were not used in

use (CDC, 2012); school attendance, effort, and disciplinary

analyses in this report.

problems; and goal-oriented thinking using a modified version

of the Children’s Hope Scale (Snyder et al., 1997). Details on

the component questions and scoring are in Appendix B.

13 The interviewers used the ASQ-3 online age calculator to determine which questionnaire to administer.

14 The full HOME inventory contains three types of items: (1) items asked about during a survey, (2) items either asked about during a survey or observed, and (3) items based only on observation. The form was based on observation-only HOME inventory items that were related to parent-child interactions.

15 In most cases for in-person surveys, the focal children were present, but, in some cases, they were not and interactions may not have been observed.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

9

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

A.10. Qualitative Surveys

instruments, the household members supplying different data

components, and implementation strategies for the 37-month

In 2011, the study team conducted indepth surveys with 80

followup data collection. This section also discusses the study

families (20 families assigned to each intervention) in four sites:

team’s efforts to maximize response rates, incentives to partici-

Alameda County, California; Bridgeport and New Haven, Con-

pate in the study, and a summary of overall response rates.

necticut; Kansas City, Missouri; and Phoenix, Arizona. Surveys

were administered in person 3 to 10 months after random as-

Implementation of 37-Month Data

signment (6.4 months on average), usually in the respondent’s

Collection

place of residence. The qualitative data collection was designed

to collect information to answer the following questions.

The 37-month followup data collection began in March 2014

and concluded in December 2014. Field interviewers conducted

1. How do families make housing decisions?

the followup surveys. Field managers recontacted 10 percent

2. What are families’ experiences (challenges) navigating the

of all respondents and administered a brief “validation ques-

housing service system?

tionnaire” to assure that the survey was done with the correct

respondent and that the interviewer followed proper protocols.

3. What explains separations of parents from children and

partners from each other?

The 37-month followup adult survey data collection process

included—

4. How do housing situations influence family processes?

1. Locating—reviewing contact history.

A team of two interviewers met with families to conduct

indepth surveys, lasting an average of 2 hours. The surveys

2. Adult informed consent—renewing consent for the adult

covered the following topics.

respondent.

1. Current housing situation, satisfaction with current situ-

3. Adult survey administration—including focal child selection.

ation, housing payments, and number of addresses since

• If no focal child selected, data collection concluded here.

study enrollment.

• If at least one focal child selected then data collection

2. Subsidy use.

continued.

3. Eligibility and takeup of assigned intervention, including

4. Consent to release information

reasons for not using the assigned intervention assistance.

Focal child data collection steps included—

4. Service receipt and satisfaction.

1. Parental permission—required before interviewers could

5. Household composition—child and spouse separations and

contact focal children.

reunifications.

2. Child assent—if focal child was ages 8 years to 17 years,

6. Family processes and rituals.

11 months.

The study team audio recorded the surveys and prepared

3. Child data collection.

handwritten notes. The team transcribed the survey summaries

and coded them using NVivo software. The qualitative data

• Parent completes ASQ-3 and SDQ (focal child ages

identified factors that influenced the family’s housing decisions

12 months to 5 years, 6 months).

as they left shelter (Fisher et al., 2014). These qualitative data

• Child assessments with focal child ages 3 years, 6 months

were used to interpret impact findings presented in Family

to 7 years, 11 months.

Options Study: Short-Term Impacts of Housing and Services Inter-

ventions for Homeless Families.

• Child survey with focal child ages 8 years to 17 years,

11 months.

A.11. Additional Details About

4. HOME inventory by the interviewer.

Surveys and Data Collection

At the time of the survey, interviewers first renewed consent with

the adult sample member. Study participants completed a par-

This section provides additional details on the surveys and data

ticipation agreement when they enrolled in the study, providing

collection process and results. The following section summa-

their informed consent to participate in the research study. The

rizes the topics covered in each of the aforementioned survey

team renewed consent with participating families at 37 months

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

10

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

to remind them of the voluntary nature of participation, the

random assignment) to a mailing (at 9,15, and 21 months

study requirements, and risks of participation. The renewed

after random assignment) to a more intensive tracking survey

consent form introduced the child data collection component.

(at 6, 12, and 27 months after random assignment).

The interviewer also asked adult respondents for permission to

send their and their child’s personally identifying information

Incentives

(PII) back to HUD, of which 1,590 adults consented to send

All respondents received an incentive payment in appreciation

their own PII and 1,468 consented to send their child’s PII.

for their time spent to complete the data collection. Adults who

When consent was renewed, interviewers administered the

completed the baseline survey received a $35 money order.

adult survey to the respondent. Although the survey instrument

Each time the adult participant responded to a tracking effort,

was designed to be conducted in person, 797 (44.7 percent) of

they received a $15 money order. Adults who completed the

adult respondents chose to do the survey by phone. In-person

20-month followup survey received a $50 money order. Adults

surveys were conducted in a variety of locations, both inside

also received a $15 money order on behalf of each child who

and outside the respondent’s residence. Local interviewers

completed the child assessments or the child survey. Adults

completed all adult surveys using laptops equipped with CAPI

who completed the 37-month adult survey received a $50

technology.

money order, they also received a $25 money order on behalf

of each child who completed the child assessments or child

If at least one focal child was selected for the study, interviewers

survey at 37 months.

reviewed the parental permission form with adult respondents

after completing the adult survey. Adult respondents could

Completion Rates

decline study participation for focal children independently of

whether they granted permission for another focal child in the

Exhibit A-5 shows the overall completion rates for each partic-

family. If the focal child was 12 months to 5 years, 6 months

ipant data collection effort. The completion rate represents the

of age, the adult respondent was asked to complete the ASQ-3

number of completed surveys as a percentage of the total cases

and all items in the parent-on-child module of the adult survey

attempted. The analytic response rate is applicable only to the

pertaining to children in that age range.

focal child data collection components (child assessment and

child survey components).

If a focal child was age 3 years, 6 months to 7 years, 11 months,

the interviewer made an appointment to meet with the adult re-

The final enrollment for the study was 2,282 families, which

spondent and the focal child to conduct the child assessments.

was the sample base for all the data collection efforts. The

After obtaining adult permission, interviewers also requested

37-month followup adult survey achieved a 78.2 percent

focal child permission to conduct the child assessments. In

completion rate. During the 37-month followup period, 7 adult

total, 44 attempts to administer the child assessments were

respondents were confirmed deceased, with 11 others deter-

refused by either the adult or focal child. All child assessments

mined deceased prior to the 37-month data collection release.

were completed in person.

Child data collection could be done only after an adult survey

Interviewers directly contacted focal children ages 8 years to

was completed because focal child selection occurred as part

17 years, 11 months to make an appointment to administer the

of the adult survey. Further, the adult respondent had to give

child survey. Before beginning the child survey, interviewers

parental permission before the child data collection could

reviewed the child assent form with the respondent and, if

commence. The child completion rates are based on the

assent was granted, proceeded to conduct the survey. Not all

number of completed child components as a percentage of the

focal children decided to participate. In total, 83 attempts to

focal children selected in households with a completed adult

administer the child survey were refused by either the parent

survey. Because an adult survey was completed with only 81

or focal child. Like the adult survey, a substantial percentage

percent of the adult sample at 20-months and 78 percent of the

of the older focal children (45.8 percent) preferred to do the

adult sample at 37-months, the child data collection analytic

survey by telephone.

response rates are lower, after adjusting for the households

without completed adult surveys.

Efforts To Improve Response Rates

Exhibits A-6, A-7, and A-8 show the number and percentage of

The study team used a variety of methods to maintain current

families who responded to study surveys. Among sample fami-

contact information on study families, with an effort to minimize

lies, nearly one-third (32.1 percent) responded to all six survey

participant burden. Study families were contacted quarterly.

efforts. Of enrolled families, 71.0 percent of families (1,621

The contacts ranged from a call (3 months and 24 months after

families) completed the 20-month and 37-month surveys.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

11

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-5. Overall Family Options Study Survey Completion Rates

Sample Released

Cases Completed

Completion Rate

Analytic Response

(N)

(N)

(%)

Rate (%)

Baseline

2,282

2,282

100

100

6-month tracking

2,282

1,671

73.2

73.2

12-month tracking

2,282

1,632

71.5

71.5

20-month adult

2,282

1,857

81.4

81.4

20-month ASQ-3

577

549

95.1

77.4

20-month HTKS

1,079

780

72.3

58.8

20-month WJ III Letter-Word

1,079

876

81.2

66.1

20-month WJ III Applied Problems

1,079

846

78.4

63.8

20-month child survey

1,128

945

83.8

68.2

27-month tracking

2,282

1,149

50.4

50.4

37-month adult

2,282

1,784

78.2

78.2

37-month ASQ-3

762

672

88.2

68.9

37-month HTKS

1,106

798

72.2

56.4

37-month WJ III Letter-Word

1,106

832

75.2

58.8

37-month WJ III Applied Problems

1,106

833

75.3

58.9

37-month child survey

1,243

1,083

87.1

68.1

ASQ-3 = Ages and Stages Questionnaire. HTKS = Head Toes Knees Shoulders. WJ III = Woodcock-Johnson III.

Sources: Family Options Study baseline survey; 6-month tracking survey; 12-month tracking survey; 20-month adult survey; 20-month child assessments; 20-month child survey; 27-month tracking survey; 37-month adult survey; 37-month child assessments; 37-month child survey Exhibit A-6. Survey Response Status for Family Options Study Baseline and Followup Surveys Families

Baseline Survey

20-Month Adult

37-Month Adult

Survey

Survey

Percent

(N)

1

0

0

262

11.5

1

0

1

163

7.1

1

1

0

236

10.3

1

1

1

1,621

71.0

TOTAL

2,282

100

Notes: 1 = completed. 0 = not completed.

Sources: Family Options Study baseline survey; 20-month followup survey; 37-month followup survey Exhibit A-7. Total Number of Families Assigned to Each Intervention and Number of Followup Survey Respondents 20-Month Survey

37-Month Survey

Both 20- and 37-Month

Surveys

Intervention

Families

Assigned

Number of

Response

Number of

Response

Number of

Response

Respondents

Rate (%)

Respondents

Rate (%)

Respondents

Rate (%)

SUB

599

530

88.5

501

83.6

467

78.0

CBRR

569

455

80.0

434

76.3

406

71.4

PBTH

368

294

79.9

293

79.6

259

70.4

UC

746

578

77.5

556

74.5

489

65.5

Total

2,282

1,857

81.4

1,784

78.2

1,621

71.0

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

Sources: Random assignment records; Family Options Study 20- and 37-month followup surveys A

3-Year Impacts of Housing and Services Interventions for Homeless Families

12

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-8. Survey Response Status for Family Options Study Baseline, Followup, and Tracking Surveys Baseline

6-Month Tracking

12-Month

20-Month Adult 27-Month Track- 37-Month Adult

Survey

Survey

Tracking

Survey

Survey

ing Survey

Survey

Families

Response

Rate (%)

1

0

0

0

0

0

129

5.7

1

0

0

0

0

1

41

1.8

1

0

0

0

1

0

1

0.0

1

0

0

0

1

1

8

0.4

1

0

0

1

0

0

30

1.3

1

0

0

1

0

1

42

1.8

1

0

0

1

1

0

4

0.2

1

0

0

1

1

1

88

3.9

1

0

1

0

0

0

24

1.1

1

0

1

0

0

1

15

0.7

1

0

1

0

1

0

1

0.0

1

0

1

0

1

1

6

0.3

1

0

1

1

0

0

37

1.6

1

0

1

1

0

1

52

2.3

1

0

1

1

1

0

6

0.3

1

0

1

1

1

1

127

5.6

1

1

0

0

0

0

56

2.5

1

1

0

0

0

1

28

1.2

1

1

0

0

1

0

1

0.0

1

1

0

0

1

1

10

0.4

1

1

0

1

0

0

27

1.2

1

1

0

1

0

1

64

2.8

1

1

0

1

1

0

10

0.4

1

1

0

1

1

1

111

4.9

1

1

1

0

0

0

49

2.2

1

1

1

0

0

1

37

1.6

1

1

1

0

1

0

1

0.0

1

1

1

0

1

1

18

0.8

1

1

1

1

0

0

94

4.1

1

1

1

1

0

1

398

17.4

1

1

1

1

1

0

28

1.2

1

1

1

1

1

1

739

32.4

Total

2,282

100

Notes: 1 = completed. 0 = not completed.

Sources: Family Options Study baseline survey; 6-month tracking survey; 12-month tracking survey; 20-month followup survey; 27-month tracking survey; 37-month followup survey

A.12. Administrative Data

Agencies collect information directly from people they serve

and enter the data into their CoC’s HMIS.

The Family Options Study used two main sources of adminis-

trative data: (1) the Homeless Management Information System

Exhibit A-9 shows the HMIS participation rates for the CoCs

(HMIS) data from each study site and (2) HUD’s Public and

containing our sample sites based on information reported

Indian Housing Information Center ( PIC) and Tenant Rental

by communities to HUD in the spring of 2011. HMIS bed

Assistance Certification System (TRACS) data. Each source of

participation refers to the percentage of beds that are covered

administrative data is described in the following sections.

in the HMIS. Thus, for example, data on clients staying in

83 percent of the beds designated for families in emergency

HMIS Data

shelters in Alameda County that are participating in the study

are included in HMIS.

An HMIS16 is the electronic information system designated by

the local Continuum of Care (CoC) program to record data on

The study team used HMIS records to measure use of

all people served within a CoC’s shelter, housing, and service

emergency shelter, rapid re-housing, transitional housing, and

system for individuals and families experiencing homelessness.

permanent supportive housing (PSH) and the length of time

families spent in these housing programs.

16 See http://www.hudexchange.info/hmis for more information on HMIS.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

13

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Exhibit A-9. HMIS Participation Rates for Emergency Shelter and Transitional Housing Providers in the Study Sites, 2011

HMIS Bed Participation Rates for

HMIS Bed Participation Rates

Study Site

All Providers in the CoC (%)

for Providers in the Study (%)

ES

TH

ES

TH

Alameda County

53

93

83

93

Atlanta

86

87

100

85

Baltimore

88

98

100

98

Boston

91

96

86

NA

Connecticut*

94–100

78–100

100

100

Denver

100

100

100

100

Honolulu

96

92

100

100

Kansas City

100

100

100

100

Louisville

80

100

100

100

Minneapolis

84

81

100

68

Phoenix

96

95

100

89

Salt Lake City

100

96

100

96

CoC = Continuum of Care. ES = emergency shelter. HMIS = Homeless Management Information System. NA = not available. TH = transitional housing.

* This study site comprises four CoCs in the New Haven/Bridgeport, Connecticut area; therefore, the figures reported for CoC coverage represent the range of coverage levels in these four CoCs.

Note: All “beds” (that is, program slots) enumerated in this table are considered “beds for families” by the CoCs.

Source: HUD Homeless Data Exchange, or HDX

HMIS data elements supplied by sites were—

Exhibit A-10. HMIS Match Rates With the Family Options

• Project entry date.

Sample by Site Through 37 Months

Sample

Sample

• Project exit date.

Site

Original

Sample (N)

Families

Families

in HMIS (N)

in HMIS (%)

• Project name.

Alameda

258

228

88.4

Atlanta

189

180

95.2

• Project type.

Baltimore

58

54

93.1

Boston

181

181

100.0

Providers enter a new HMIS record for every new entry into a

Connecticut*

214

191

89.3

program. Thus, for people who receive more than one episode

Denver

172

148

86.0

of assistance, HMIS contains multiple records per person. Pro-

Honolulu

218

213

97.7

viders ask clients entering programs to provide PII, but clients

Kansas City

175

170

97.1

Louisville

109

102

93.6

are not required to comply to receive services. Exhibit A-10

Minneapolis

181

176

97.2

shows the number and percentage of families in the sample

Phoenix

279

275

98.6

that were identified in the site’s HMIS. A family is considered

Salt Lake City

248

202

81.5

matched in the HMIS if at least one program record was found

All sites

2,282

2,120

92.9

for the head of household in the HMIS data received and time

HMIS = Homeless Management Information System.

of program use occurred after the random assignment date.

*Includes the cities of New Haven and Bridgeport, Connecticut.

Source: HMIS

The study team gathered supplementary shelter program-use

data for Minneapolis, Minnesota, and Boston, Massachusetts.

PIC and TRACS Data Files

Hennepin County provided records for emergency shelters that

The study team used extracts from two HUD administrative

were not covered in the Minneapolis HMIS. The State of Massa-

data systems, PIC and TRACS. PIC data were used to measure

chusetts Office of Community Development provided records

sample members’ receipt of housing assistance from one of

of Emergency Assistance program use that were not covered in

three programs—public housing, the Housing Choice Voucher

the Boston HMIS. In addition, the Connecticut HMIS data were

program, and project-based voucher assistance.17 TRACS data

provided by the Bridgeport and New Haven CoCs separately.

were used to track information regarding program entry and

exit for project-based Section 8 programs.18 PIC and TRACS

data measure use of the permanent housing subsidies offered to

families assigned to the SUB intervention.

17 HUD Form 50058 describes the full list of variables available in the PIC data and is accessible on line at http://portal.hud.gov/hudportal/documents/huddoc?id=50058.pdf.

18 Documentation on the TRACS data is accessible on line at http://portal.hud.gov/hudportal/HUD?src=/program_offices/housing/mfh/trx/trxdocs.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

14

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

HUD provided the data in 26 PIC and 26 TRACS quarterly

of random assignment.20 Therefore, for action types other than

extracts. The quarterly extracts cover the period from March

new admission, the date of program admission was consistently

2009 through June 2015 and cover the effective date period

used as the date of lease up. Exhibit A-11 summarizes the

from January 1, 2008, until June 30, 2015. Together, the 52

number of sample families who were matched in PIC/TRACS

quarterly extracts contain information on 919 study families.

and the number assigned to the SUB intervention matched in

PIC/TRACS.

For families using vouchers, the study team used PIC data

to identify the date on which the household began to receive

NDNH Data

rental assistance, referred to as the lease-up date. The program

admission date, effective date of the action, program type code,

HUD’s Office of Policy Development and Research (PD&R)

and program action code were the major variables used from

collected de-identified quarterly wage records from the Nation-

the PIC extracts to determine the timing of new admissions

al Directory of New Hires (NDNH), maintained by the Office of

(versus annual reexaminations, interim reexaminations, or

Child Support Enforcement (OCSE) within the Administration

other actions).19

for Children and Families at the U.S. Department of Health and

Human Services. The NDNH maintains up to eight quarters of

Depending on the type of action recorded in PIC, the date

quarterly wage data from state workforce agencies and federal

of program admission is either the same as or earlier than

agencies.21 Between June 2015 and March 2016, the study

the effective date of action. For new admissions, the date of

team sent quarterly matching files with family head name and

admission and the effective date of action are normally the

Social Security Number to OCSE. OCSE matched these files to

same. When they differ, the effective date is considered the

the quarterly wage records, then stripped the PII and replaced

better indicator of lease up, because the effective date refers to

with a randomly generated person identification code. In June

either the signing of the lease or the actual occupancy of the

2016, the study team sent a pass-through file with several data

unit (as opposed to, for instance, the issuance of the voucher

elements to OCSE, which again removed PII and replaced

to the participant). Among records of actions other than new

with the randomly generated person identification code. All

admissions, some effective dates fell up to a year after the date

data sets were then transferred to PD&R. The study team did

Exhibit A-11. Sample Families in PIC/TRACS Data and Those Assigned to the SUB Intervention in PIC/TRACS Data by Site Through 37 Months

All Families Assigned

Percent SUB-Assigned

Sample Families in

All Families Assigned

PIC/TRACS (N)

to SUB (N)

to SUB in

Families in

PIC/TRACS (N)

PIC/TRACS

Alameda County

108

76

70

92.1

Atlanta

35

0

0

NA

Baltimore

22

0

0

NA

Boston

122

64

58

90.6

Connecticut*

90

47

42

89.4

Denver

95

76

63

82.9

Honolulu

72

43

30

69.8

Kansas City

78

53

38

71.7

Louisville

45

32

21

65.6

Minneapolis

75

62

45

72.6

Phoenix

89

71

63

88.7

Salt Lake City

88

75

54

72.0

All sites

919

599

484

80.8

NA = not available. PIC = Public and Indian Housing Information Center. SUB = priority access to permanent housing subsidy. TRACS = Tenant Rental Assistance Certification System.

*Includes the cities of New Haven and Bridgeport, Connecticut

Source: PIC/TRACS data

19 An action is the administrative transaction that triggers the completion of the HUD Form 50058 that is submitted to PIC. The 50058 includes 14 action codes: new admission, annual reexamination, interim reexamination, portability move-in, portability move-out, end of participation, other change of unit, Family Self-Sufficiency/Welfare to Work addendum, annual reexamination, issuance of voucher, expiration of voucher, flat rent annual update, annual Housing Quality Standards inspection, and historical adjustment.

20 This lapse in time might occur, for instance, if the housing authority simply failed to record the new admission but recorded a subsequent action.

21 Established in accordance with the Personal Responsibility and Work Opportunity Reconciliation Act of 1996, the NDNH database was developed to assist state child support agencies in locating parents and enforcing child support orders. The NDNH contains quarterly wage, new hire, and unemployment information. The Family Options Study used only quarterly wage information.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

15

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

not have direct access to the data; instead, we sent statistical

The data obtained from these five agencies used AFCARS re-

analysis programs to staff at PD&R, who ran the programs on

porting definitions to ensure the information collected was—to

their servers and produced results. Summary results were then

the extent possible—defined consistently across the five sites.

transferred to the study team for analysis and reporting.

The minimum information requested from each site was—

OSCE provided HUD with four extracts of de-identified quar-

• Start and end dates of each removal or placement.

terly wage records. Although we anticipated that each extract

• Reason for discharge or placement end.

would contain eight calendar quarters of data, we found that

only seven quarters in each extract appeared to have complete

• Date of termination of parental rights.

data. Each extract contained various other quarters with re-

• Child date of birth.

ported wages, but the number of observations in these quarters

was significantly lower than the number of observations in the

Because child welfare data were obtained from only the state—

seven complete quarters. Due to these limitations, the study

or, in the case of Alameda County, the county—in which an

team decided to use only the seven quarters with complete

individual was randomly assigned, individuals with children in

data from each of the extracts. Across all extracts received from

out-of-home care under the care of another child welfare agen-

OCSE, complete data for the study sample was collected for 10

cy are not considered in the analysis. It is therefore possible

calendar quarters, from 2013Q2 to 2015Q3. Details about out-

that the records collected understate the overall level of formal

comes constructed with these data are provided in Appendix B.

separations because they miss separations occurring outside the

state or county the parent was randomly assigned.

Child Welfare Agency Administrative Data

To understand what data are available from child welfare

A.13. Program Usage Data File

agencies, it is helpful to understand the federal reporting

requirements of child welfare programs. For children under the

The study used several types of information to document

care and supervision of the child welfare agency (or adopted

the types of assistance families received during the followup

under the auspices of the agency), states are required to submit

period. Enrollment verification data, tracking and followup

semi-annually de-identified, case-level data to the Adoption

surveys, and administrative data, each described previously in

and Foster Care Analysis and Reporting System (AFCARS),

Appendix A, were combined to form a Program Usage Data file,

maintained by the Children’s Bureau of the Administration of

with information about program use for every month starting

Children and Families. The de-identified nature of the AFCARS

at the date of random assignment through the date of the adult

data makes it impossible to link these records to the study

survey. This section describes how the study team created

sample. Therefore, the study looked to state and local agencies

the Program Usage Data file, which contains data on the full

to obtain information on formal out-of-home placements.

sample of 2,282 families enrolled in the study.22

Out of the 12 study sites, the study team sought to acquire

The study team gathered family-level information on program

child welfare data from eight agencies where it was determined

entry dates, program exit dates, and program types using eight

feasible. Among those, the following five agencies provided

data sources collected by the Family Options Study. The data

child welfare data.

sources are—

• Alameda County Department of Social Services (Alameda

1. Enrollment verification data.

County site).

2. 6-month tracking survey.

• Arizona Department of Child Safety (Phoenix site).

3. 12-month tracking survey.

• Maryland Department of Human Resources, Social Services

4. 20-month followup adult survey.

Administration (Baltimore site).

5. 27-month tracking survey.

• Minnesota Department of Human Services, Child Safety and

Permanency Division (Minneapolis site).

6. 37-month followup adult survey.

7. HMIS.

• Missouri Department of Social Services, Children’s Division

(Kansas City site).

8. PIC/TRACS.

22 For the Short-Term Impacts report, the study team prepared program usage data for the 1,857 families who completed the 20-month followup survey. For the current report, the study team entirely recreated program usage data for all 2,282 study families, using data sources available at the time of the earlier report plus a second round of HMIS data collection, six more quarterly extracts of PIC and TRACS records, the 27-month tracking survey, and the 37-month followup survey.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

16

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

Each data source has information about program use since

To resolve conflicting information across data sources, the

the date of random assignment. The data sources vary in the

study team devised a system of decision rules. The fundamental

amount of time they cover. For example, the administrative

rule for cleaning the data was that two instances of program use

data and the 37-month survey cover the full analysis period,

could not overlap, forcing the study team to clean dates that

but the tracking surveys and enrollment verification cover only

indicated the family was in two or more programs simultaneously.

part of the period.

The study team ranked the data sources in the order believed

to contain the most to least reliable program use information.

The study team considered data from all eight sources when

Perceived reliability of the data sources varied by data item—

compiling family histories of program use. In many cases, the

program entry date, exit date, and type. Exhibit A-13 summa-

same instance of program use by a family was recorded in more

rizes the reliability ratings.

than one data source. In some of these cases, the multiple data

sources were in complete agreement. In other cases, the data

The study team considered the program entry date from the

sources had discrepant information about entry dates, exit

enrollment verification data most reliable because the team

dates, and/or program type.

collected these data directly from the participating program

Exhibit A-12. Program Types and Their Data Sources in the Program Usage Data

Program Type

Data Sources

Emergency shelter

• HMIS recordsa

• 6-, 12-, 20-, 27-, and 37-month surveys

Permanent housing subsidies offered to the SUB groupb

• HUD PIC and TRACS records

• Enrollment verification records (for referred program)

• 6-, 12-, 20-, 27-, and 37-month surveys

Rapid re-housing

• HMIS records

• Enrollment verification records (for referred program)

• 6-, 12-, 20-, 27-, and 37-month surveys

Transitional housingc

• HMIS records

• Enrollment verification records (for referred program)

• 6-, 12-, 20-, 27-, and 37-month surveys

Permanent supportive housing

• HMIS records

• 6-, 12-, 20-, 27-, and 37-month surveys

Public housing

• HUD PIC and TRACS records

• 6-, 12-, 20-, 27-, and 37-month surveys

Project-based vouchers or Section 8 projects

• HUD PIC and TRACS records

• 6-, 12-, 20-, 27- and 37-month surveys

SUB = priority access to permanent housing subsidy.

HMIS = Homeless Management Information System. HUD = U.S. Department of Housing and Urban Development. PIC = Public and Indian Housing Information Center.

TRACS = Tenant Rental Assistance Certification System.

a The study team collected HMIS records covering homeless assistance programs in the 12 study communities. The HMIS data thus cover program use only in the 12 communities and do not cover program use outside the site jurisdiction determined at baseline. Altogether, 12.3 percent of families (219 families) interviewed for the 37-month interview were residing in a different state from the address indicated at baseline. The proportion of families interviewed in different states varied by site from 5.2 percent in Boston to 28.5 percent in Kansas City.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii. The site-specific non-housing choice voucher programs were public housing in Honolulu, Hawaii, and project-based vouchers in Bridgeport, Connecticut. In other sites, these programs are coded separately.

c The transitional housing program type represents both project-based and scattered-site varieties of transitional housing, including transition-in-place units.

Exhibit A-13. Data Source Reliability by Program Use Data Item

Program Use Data Item

Higher Reliability

Lower Reliability

Program entry date

Enrollment verification

20- and 37-month followup surveys; tracking surveys

HMIS; PIC/TRACS

Program exit date

HMIS; PIC/TRACS

20- and 37-month followup surveys; tracking surveys;

tracking surveys; enrollment verification

Program type

Enrollment verification

20- and 37-month followup surveys; tracking surveys

HMIS; PIC/TRACS

HMIS = Homeless Management Information System. HUD = U.S. Department of Housing and Urban Development. PIC = Public and Indian Housing Information Center.

TRACS = Tenant Rental Assistance Certification System.

Sources: Enrollment verification data; 6-month tracking survey; 12-month tracking survey; 20-month followup survey; 27-month tracking survey, 37-month followup survey, HMIS; PIC/TRACS

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

17

FAMILY OPTIONS STUDY

Appendix A. Data Sources and Data Set Construction

specifically about the study families. The administrative data—

per family. The Program Usage Data file contained a series of

HMIS and PIC/TRACS—were also treated as highly reliable,

monthly indicator binary variables reflecting the period from

second to the enrollment verification. The administrative data

the month of random assignment through either the month of

are maintained by communities and HUD. Program entry date

the 37-month survey date (for 37-month survey respondents)

information from the tracking surveys and 20- and 37-month

or through the month of the 1,129th day after the date of

followup surveys was considered less reliable because of

random assignment (for 37-month survey nonrespondents).23

recall error. If entry date information was available only in the

The study team prepared a separate set of indicator variables

surveys, the study team considered program entry dates closest

for several program types: emergency shelter, transitional hous-

to survey dates as more reliable than other older entry dates

ing, rapid re-housing, permanent housing subsidies offered to

because they would have lower recall error.

the SUB group, PSH, public housing, and Section 8 projects/

project-based vouchers. An indicator variable was set equal to 1

The study team considered the program exit date from the

to indicate that the study family used a particular program type

administrative data to be most reliable. The exit date from

at least 1 day during the month, or it was set equal to 0 to in-

the enrollment verification data was considered least reliable

dicate no use of that program type in that month. The Program

because data were not collected for a long enough period to re-

Usage Data file was constructed to complement the outcomes

cord an exit date. The tracking surveys also contained missing

reported in the 20- and 37-month followup adult surveys.

exit date information if the family was in a housing program

Therefore, information provided by administrative data beyond

at the time of those surveys and could suffer from recall error.

the month of the 37-month followup survey response was not

The 37-month followup information on exit dates covered the

incorporated into the file.

full study period but could still suffer from recall error.

These data are known to miss at least some program use. The

The study team considered the program type data from the

baseline stay in emergency shelter does not appear in the data

enrollment verification as the most reliable because these pro-

for 18.3 percent of survey respondent families. The missing da-

viders were involved in the study to represent an intervention

ta rate for subsequent stays in emergency shelter is unknown.

program type. Program type data from administrative sources

The study team expects that HMIS records on community-

were considered to also be highly reliable, second to the en-

based rapid re-housing, transitional housing, and permanent

rollment verification data. The study team worked closely with

housing to be at least as complete as the baseline emergency

the HMIS administrators to accurately code programs. Data

shelter records (at least 82 percent). Because the data on these

from PIC/TRACS were also considered highly reliable because

three program types also rely on the program usage files com-

data are maintained by HUD. Program type information in

piled from multiple sources including enrollment verification

the 20- and 37-month followup and tracking surveys were

(for the referred program) and up to five self-reports (three

considered to be least reliable because of recall error and likely

tracking interviews and two followup interviews), the study

lack of knowledge of the program type beyond the name of the

team expects the vast majority of program spells of these types

program.

to be captured in the data.

Basing its analysis on these and other site-specific rules,

The data on use of subsidy, public housing, and project-based

the study team manually determined which records and

vouchers or Section 8 projects should be essentially complete

information were preserved that most accurately reflected the

because they are based on HUD administrative records. Addi-

program use history of a family. These data were converted

tional detail about the construction of outcomes based on the

into the Program Usage Data file, which contained one record

Program Usage Data is provided in Appendix B.

23 The median survey start date for 37-month survey respondents is 1,129 days after the date of random assignment.

A

3-Year Impacts of Housing and Services Interventions for Homeless Families

18

FAMILY OPTIONS STUDY

APPENDIX B.

CONSTRUCTION OF OUTCOMES

This appendix describes how the study team constructed is primarily taken from the Homeless Management Informa-outcome measures from the 37-month followup survey

tion System (HMIS) but also uses survey data. This measure

and administrative data. It supplements information

is the percentage of families who spent at least 1 night in

in Chapter 2. The section is organized by outcome domain:

emergency shelter in the 6 months before the 37-month

housing stability, family preservation, adult well- being, child

followup survey.

well-being, and self-sufficiency.

• Any stay in emergency shelter in months 21 to 32 after

random assignment. This binary variable is constructed

B.1. Measures of Housing Stability

using Program Usage Data. Families with program usage

records that indicate a stay in emergency shelter in the

Homelessness during followup period. The study team

period from 21 to 32 months after random assignment are

developed seven measures related to homelessness experienced

coded as 1. The time period is the latest 1-year period for

during the 37-month followup period.

which survey data are available for all families in the impact

• At least 1 night homeless or doubled up during past

analysis sample. HMIS records were the source of most (92

6 months. This binary variable is constructed from responses

percent) of the data for emergency shelter stays. The tracking

to Questions A9 and A11. It measures the percentage of

and followup surveys also provided information on stays in

study families who reported spending at least 1 night during

emergency shelter.

the 6 months before the 37-month followup survey either

• Number of days homeless or doubled up in the past

homeless (residing in a shelter or institution or staying in

6 months. This continuous variable is constructed from

a place not meant for human habitation, such as the street,

responses to Questions A10a1 to A10a3 and A12a1 to

car, abandoned building, or train station) or living with

A12a3. The outcome measures the total number of days

a friend or relative because they could not find or afford

spent homeless or doubled up in the 6 months before the

a place of their own. The survey item explicitly excluded

37-month followup survey. This outcome does not reflect

transitional housing.

stays in transitional housing.

• At least 1 night homeless during past 6 months. This

• Number of days homeless during past 6 months. This

binary variable is constructed from responses to Question A9.

continuous variable is constructed from responses to

It measures the percentage of families who reported having

Questions A10a1 to A10a3. The outcome measures the

spent at least 1 night in a shelter or place not meant for

average number of days spent in shelters or places not meant

human habitation in the 6 months before the 37-month

for human habitation in the 6 months before the 37-month

followup survey. This outcome does not include stays in

followup survey. It is measured from survey data. This

transitional housing.

outcome does not reflect stays in transitional housing.

• At least 1 night doubled up during past 6 months.

• Number of days doubled up during past 6 months. This

This binary variable is constructed from responses to

continuous variable is constructed from responses to Ques-

Question A11. It measures the percentage of families who

tions A12a1 to A12a3. The outcome measures the average

reported spending at least 1 night in the 6 months before the

number of days spent living with friends or relatives in the

37-month followup survey living with a friend or relative

6 months prior to the survey. It is measured from survey data.

because they could not find or afford a place of their own.

Confirmatory outcome. The study team constructed a single

• Any stay in emergency shelter in past 6 months. This

composite binary outcome for the 37-month followup point.

binary variable is constructed using Program Usage Data and

This outcome is defined as “at least one return to homelessness”

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

in the year before the 37-month survey measured from both

Questions A7 and A8 about receipt of housing assistance at

the followup survey and Program Usage Data.

the time of the 37-month followup survey. The outcome is

assigned a value of 1 for respondents who answered Ques-

• At least 1 night spent staying in a shelter or a place not

tion A4a to indicate they were living in a house or apartment

meant for human habitation or doubled up during the past

that they own or rent and who answered yes to Questions A7

6 months at the time of the survey (measured from survey

and A8 about whether they received housing assistance. This

data) OR any stay in emergency shelter in the 12 months

outcome measures the percentage of families who reported

prior to the date of the survey (measured from Program

living in their own house or apartment at the time of the survey

Usage Data).

and were receiving housing assistance to help pay the rent.

If either of the two binary outcomes were coded as 1, the com-

Number of places lived. The study team also measured an out-

posite confirmatory outcome was also coded as 1. The lengths

come related to housing instability during the followup period

of time in the reference periods for this outcome (6 months for

using the parent survey.

survey component, 12 months for Program Usage Data) are the

same as for the confirmatory outcome in Family Options Study:

• Number of places lived in the past 6 months. This con-

Short-Term Impacts of Housing and Services Interventions for

tinuous variable is constructed from responses to Question

Homeless Families.

A13. The outcome measures the number of places the family

lived in the 6 months before the survey. The variable is

Housing independence. The study team used data from the

top-coded at six places. A value of 7 means that the adult

adult survey to construct three outcomes pertaining to the

respondent reporting having lived in six or more places in

type of living arrangements at the time of the followup survey,

the 6 months before the survey.

number of places lived, and housing quality. All these measures

are from the followup survey.

Housing quality. The housing domain also includes two

outcomes measuring the quality of sample members’ housing at

• Living in own house or apartment at time of survey. This

the time of the 37-month followup survey.

binary variable is constructed from responses to Question A4a,

which asks if the respondent is currently living in a house

• Persons per room. This continuous variable is based on

or apartment that he or she owns or rents. The interviewer

responses to Question B7 (number of rooms not including

instructed the respondent not to include his or her parent’s

kitchens, bathrooms, and hallways) and Questions B1A_1

or guardian’s home or apartment. Survey respondents are

to B1A_14 and B3a and B3b (number of persons living

considered to have independent housing if they rented or

with the adult respondent). The outcome was constructed

owned their own housing at the time of the survey. (Housing

by dividing the number of people by the number of rooms.

owned or rented by a “boyfriend/girlfriend, fiancé or signif-

Housing situations with more than one person per room are

icant other” is not counted as living in the respondent’s own

considered crowded.

house or apartment). This outcome measures the percentage

• Housing quality is poor or fair. This binary variable is

of families who reported living in their own house or

based on responses to Question B8. Respondents were asked

apartment, either with or without housing assistance.

to rate the condition of their current house or apartment as

• Living in own house or apartment at time of survey

either excellent, good, fair, or poor. This outcome measures

with no housing assistance. This binary variable is based

the percentage of families self-reporting that the condition of

on responses to Question A4a about living situation and

their housing at the time of the survey was fair or poor.

Questions A7 and A8 about receipt of housing assistance at

the time of the 37-month followup survey. The outcome is

B.2. Measures of Family

assigned a value of 1 for respondents who answered Ques-

tion A4a to indicate they were living in a house or apartment

Preservation

that they own or rent and who answered no to Questions A7

The study team collected detailed information about the compo-

and A8 about whether they received housing assistance. This

sition of the study families and changes in family composition

outcome measures the percentage of families who reported

that occurred during the followup period. The study team collected

living in their own house or apartment at the time of the

names and ages of family members with the adult respondent

survey and were not receiving housing assistance.

in shelter at the time of enrollment and of family members who

• Living in own house or apartment at time of survey with

were not with the adult respondent at enrollment but whom

housing assistance. This binary variable is constructed

the adult respondent considered to be part of the family. Then,

from responses to Question A4a about living situation and

at the 37-month followup survey, the study team collected

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

information on the whereabouts of all family members reported

had rejoined the family at the time of the 37-month follow-

at baseline and about new family members who had joined

up survey. This outcome is measured only for respondent

the family since the previous survey. This section describes the

families in which a child was separated from the family at

outcomes constructed from these data.

the time of random assignment (339 families).

• Family has at least one child separated in past 6 months.

• Spouse or partner reunified, of those spouse or partner

This binary variable is constructed from items in Section D

absent at random assignment. This binary variable is

of the 37-month followup survey and from information

taken from Section D of the 37-month followup survey and

gathered from Section E of the baseline survey. This variable

Section E of the baseline survey. It measures the percentage

measures the percentage of families for whom a child who

of families in which a spouse or partner who was separated

had been with the family in shelter was separated from the

from the family at baseline had rejoined the family at the

family at any time in the 6 months before the 37-month

time of the 37-month followup survey. This outcome is

followup survey. This outcome includes both formal (that is,

measured only for respondent families in which a spouse or

with the involvement of a child welfare agency) and informal

partner was separated from the family at the time of random

separations from the family, and both ongoing and new

assignment (175 families).

separations in this period. The outcome is based on children

who are less than 18 years, 6 months at the time of followup

B.3. Measures of Adult Well-Being

in order to exclude children who had reached the legal age

of adulthood before the separation. The time period for this

The study team used the adult survey to construct the

outcome, 6 months before the survey, thus included at least

following outcomes measuring several aspects of well-being for

some time before the children reached adulthood.

the adult respondent: adult physical health, adult behavioral

• Family has at least one foster care placement in the past

health, adult trauma symptoms, adult substance abuse, and

6 months. This binary variable is constructed from items

experience of intimate partner violence.

in Section D of the 37-month followup survey. It measures

Adult physical health in past 30 days was poor or fair. This

the percentage of families who reported that a child was in

binary variable is constructed from responses to Question E1.

a formal foster care placement or adopted by another family

The outcome measures the percentage of families in which the

at any time in the 6 months prior to the 37-month survey.

adult respondent reported poor or fair health in the 30 days

It includes both new and ongoing foster care placements.

before the survey. Response options were excellent, very good,

This outcome excludes informal arrangements in which a

good, fair, or poor. The outcome value is 1 if respondents rated

child may have stayed with friends or family members but

health as fair or poor.

includes adoptions of children by another family.

Goal-oriented thinking. The adult survey collected six items

• Spouse/partner separated in past 6 months, of those with

of the State Hope Scale (Snyder et al., 1996) in Questions E2a

a spouse/partner present at random assignment. This

to E2f. Participants respond to each item using a 6-point scale.

binary variable is constructed from items in Section D of

1 = definitely false.

the 37-month followup survey and Section E of the baseline

survey. This outcome measures the percentage of respondent

2 = mostly false.

families in which a spouse or partner who had been present

3 = somewhat false.

at baseline (458 families) became separated in the 6 months

4 = somewhat true.

before the 37-month followup survey. The outcome includes

5 = mostly true.

both new and ongoing separations.

6 = definitely true.

The team also constructed the following two family reunifica-

tion outcomes that measure the return of family members who

The items are—

had been reported as separated from the family at baseline.

1. If I should find myself in a jam, I could think of many ways

to get out of it.

• Family has at least one child reunified, of those families

with at least one child absent at random assignment.

2. At the present time, I am energetically pursuing my goals.

This binary variable is constructed from Section D of the

3. There are lots of ways around any problem that I am facing

37-month followup survey and Section E of the baseline

now.

survey. It measures the percentage of families in which a

4. Right now I see myself as being pretty successful.

child who had been living apart from the family at baseline

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

5. I can think of many ways to reach my current goals.

Post-traumatic stress disorder (PTSD) symptoms. This bina-

6. At this time, I am meeting the goals that I have set for myself.

ry outcome is constructed from responses to Questions E4a to

E4q. It measures the presence of PTSD symptoms in adult re-

The study team created a score for the State Hope Scale if the

spondents in the 30 days before to the survey. These questions

respondent answered at least four of the six items. For each

are the Posttraumatic Stress Diagnostic Scale (PDS) assessment

respondent, the analysts averaged the responses given. This

that is designed to aid in the detection and diagnosis of PTSD.

process yields measures ranging from 1 to 6 with higher scores

The PDS assessment parallels DSM-IV® diagnostic criteria for a

indicating greater hope. The adult State Hope Scale measures

PTSD diagnosis and may be administered repeatedly over time

Snyder’s cognitive model of hope which defines hope as “a

to help monitor changes in symptoms.

positive motivational state that is based on an interactively

derived sense of successful (a) agency (goal-directed energy),

Respondents were asked to report on how much each of the

and (b) pathways (planning to meet goals)” (Snyder, Irving,

following items had bothered them in the 30 days prior to the

and Anderson, 1991: 287).

survey.

Psychological distress. This continuous variable is the Kessler 6

Subset 1.

(K6) Psychological Distress Scale (Kessler et al., 2003). It is

E4a. Repeated, disturbing memories, thoughts, or images of a

derived from six survey items (Questions E31 to E36). The

stressful experience?

respondents were asked how much of the time in the past 30

days they had felt each of six measures of distress—

E4b. Repeated, disturbing dreams of a stressful experience?

1. Nervous?

E4c. Suddenly acting or feeling as if stressful experiences were

2. Hopeless?

happening again (as if you were reliving it)?

3. Restless or fidgety?

E4d. Feeling very upset when something reminded you of a

4. So depressed that nothing could cheer you up?

stressful experience?

5. That everything was an effort?

E4e. Having physical reactions (for example, heart pounding,

6. Worthless?

trouble breathing, or sweating) when something reminded you

of a stressful experience?

Responses options were—

1 = all of the time.

Subset 2.

2 = some of the time.

E4f. Avoid thinking about or talking about the stressful experi-

ences or avoid having feelings related to it?

3 = a little of the time.

4 = none of the time.

E4g. Avoid activities or situations because they remind you of a

stressful experience?

The study team created a score for each respondent if the re-

spondent answered at least four of the items (imputing values for

E4h. Trouble remembering important parts of the stressful

missing items with the mean value of the nonmissing items).

experience?

Scores were reversed such that a response of all of the time

E4i. Loss of interest in things that you used to enjoy?

= 4, most of the time = 3, some of the time = 2, a little of the

time = 1, and none of the time = 0. The scores were summed,

E4j. Feeling distant or cut off from other people?

creating a continuous indicator of psychological distress. The

E4k. Feeling emotionally numb or being unable to have loving

score ranges from 0 to 24 with higher values indicating greater

feelings for those close to you?

psychological distress.

E4l. Feeling as if your future will somehow be cut short?

Exhibit 2-9 in Chapter 2 reports the percent of family heads

reporting symptoms of serious psychological distress. That

Subset 3.

measure was derived from the continuous distress scale using

E4m. Trouble falling or staying asleep?

a cutoff of 13 (scores of 13 and over were coded to indicate

serious psychological distress). This score was the optimal

E4n. Feeling irritable or having angry outbursts?

cutoff point for the general population sample in the Kessler et

E4o. Having difficulty concentrating?

al. (2003) validation study.

E4p. Being “super alert” or watchful on guard?

E4q. Feeling jumpy or easily startled?

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

Responses options were—

E8. During the past 6 months, have you failed to do what was

1 = not at all.

normally expected of you because of drinking?

2 = a little bit.

An affirmative answer to any of the items indicates an alcohol

3 = moderately.

problem.

4 = quite a bit.

The Rapid Alcohol Problems Screen is a five-item instrument,

5 = extremely.

derived from other screens, that is designed to maximize sensi-

tivity while maintaining good specificity. The RAPS-4, a further

The PTSD outcome was created if the respondent answered at

refinement of the five-item instrument, asks if an individual felt

least 12 of the 17 items. The first step in scoring the responses

guilt after their drinking (Remorse), could not remember things

was to assess if the respondent was symptomatic on each item.

said or did after drinking (Amnesia), failed to do what was

Responses of 3 = moderately, 4 = quite a bit, or 5 = extremely

normally expected after drinking (Perform), or had a morning

to any of the items indicate the respondent is symptomatic and

drink (Starter).

receive a value of 1. If a respondent answered 1 = not at all or

2 = a little bit to an item they were assessed as not symptomatic

Drug abuse. This outcome is constructed from responses to

and received a value of 0.

Questions E10a through E10h. The survey instrument included

six items regarding use of illegal drugs, all of which are part

The 17 items were then divided into subscales.

of the short version of the Drug Abuse Screening Test, or

• Subscale B: sum of 5 items in Subset 1: a–e.

DAST-10 (Skinner, 1982; Yudko, Lozhkina, and Fouts, 2007).

• Subscale C: sum of 7 items in Subset 2: f–l.

The following six items were asked of respondents in relation

to the 6 months before the survey date.1

• Subscale D: sum of 5 items in Subset 3: m–q.

E10a. Have you used more than one drug at a time?

To receive a value of 1 for the PTSD binary variable, the

respondent had to be symptomatic on one or more items in

E10b. Have you had “blackouts” or “flashbacks” as a result of

subscale B, three or more items in subscale C, and on two or

drug use?

more items in subscale D. This measure of PTSD was also col-

E10e. Have you ever not spent time with your family or missed

lected at baseline and serves as a covariate scored in the same

work because of drug use?

manner (see Appendix C).

Adult substance abuse. The study team measured three

E10f. Have you engaged in illegal activities in order to obtain

outcomes related to substance abuse in the 6 months before the

drugs?

survey. The first is a binary variable indicating alcohol depen-

E10g. Have you ever experienced withdrawal symptoms as a

dence, the second is a binary variable indicating drug abuse,

result of heavy drug intake?

and the third is a binary variable indicating alcohol dependence

or drug abuse.

E10h. Have you had medical problems as a result of drug use

(for example memory loss, hepatitis, convulsions, bleeding?)

Alcohol dependence. This outcome is constructed from re-

sponses to Questions E5 through E8 in the 37-month followup

An affirmative answer to any of these six items indicates a drug

survey, which asked the following four items in the Rapid

problem.

Alcohol Problems Screen (RAPS-4; Cherpitel, 2000).

Alcohol dependence or drug abuse. If the respondent is

E5. Do you sometimes take a drink in the morning when you

determined to have an alcohol problem or a drug problem,

first get up?

the outcome alcohol dependence or drug abuse is assigned a

value of 1. This outcome measures the percentage of families

E6. During the past 6 months, has a friend or family member

in which the adult respondent displayed evidence of alcohol

ever told you about things you said or did while you were

dependence or drug abuse in the 6 months before the survey.

drinking that you could not remember?

E7. During the past 6 months, have you had a feeling of guilt

Experienced intimate partner violence in the past 6 months

or remorse after drinking?

prior to survey. This binary variable is based on responses

to Question E11. E11 asks if, in the past 6 months, the adult

respondent has been physically abused or threatened with

1 Respondents were also asked about two additional items related to drug use that are not used in the DAST-10 and were not used to create the drug abuse outcome. E10c asked, “Have your friends or relatives known or suspected that you used drugs?” and E10d asked, “Have you ever lost friends because of drugs?”

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

violence by a person with whom she/he was romantically

• Well-child checkup in past 12 months. This binary

involved, such as a spouse, boy/girlfriend, or partner. The out-

outcome is based on responses to Question F19. Parents are

come measures the percentage of adult respondents reporting

asked whether outcome measures the percentage of focal

experience of intimate partner violence in the 6 months prior

children who received a physical examination or well-child

to the survey.

checkup in the year prior to the 37-month followup survey.

• Child has regular source of health care. This binary

B.4. Measures of Child Well-Being

outcome is based on responses to Question F18a. This

outcome measures the percentage of focal children who had

The study team used information reported by the parent about

a regular provider of health care at the time of the 37-month

the focal children to construct the following child well-being

followup survey, based on the parent’s report.

outcomes across all age groups.

• Sleep problems. This continuous variable is based on

Child Outcomes Across All Age Groups

responses to Questions F26i, F26j, and F26k. The parent is

asked to report on the frequency of two indicators of sleep

Child Education

problems—tiredness on waking and tiredness during the

• Number of schools attended since random assignment.

day. The allowable responses are—

This continuous variable is based on responses to Question F12a.

1 = almost always.

The number of schools is top-coded at four or more schools.

The values of the outcome are—

2 = most days.

1 = one school.

3 = sometimes.

2 = two schools.

4 = rarely.

3 = three schools.

5 = almost never.

4 = four or more schools.

The questions are worded as follows.

• Grade completion—not held back. This binary variable

Would you say that…

is based on responses to Question F12b From the parent

[CHILD] has difficulty waking up in the morning?

survey and defined for children who have been in school at

any time since random assignment and who are less than

[CHILD] has difficulty waking up on school days?

age 18 at the time of the 37-month survey. The outcome

[CHILD] is tired during the day?

measures the percentage of these children who have not

To create the outcome, the study team reverse-coded the

repeated a grade level and have not been prevented from

response options (for example, almost never = 1; almost

moving on to the next grade level since random assignment.

always = 5). The value of the outcome thus ranges from

• School grades. This continuous outcome was constructed

1 to 5 with higher values indicating greater frequency of

using responses to Question F12c. The parent was asked to

these sleep problems.

describe the child’s grades in the most recently completed

school term. The outcome uses a 4-point scale with the

Behavioral Strengths and Challenges

following values.

• Behavioral problems. This continuous variable is based on

1 = mostly Ds and Fs.

the Strengths and Difficulties Questionnaire (SDQ). The SDQ

2 = mostly Cs.

is a behavioral and personality assessment with a battery

of items about the behavioral strengths and difficulties of

3 = mostly Bs.

children. The total problem score measures emotional symp-

4 = mostly As.

toms, conduct problems, hyperactivity, and peer problems.

The outcome is measured using responses to Questions F21

Health

(for 3-year-olds), F22 (4- to 10-year-olds), and F23 (11- to

• Poor or fair health. This binary outcome is based on

17-year-olds) on the parent survey. Parents indicated whether

responses to Question F18, which asks the parent to assess

a series of statements were not true, somewhat true, or certainly

the child’s health at the time of the 37-month followup

true for the child during the 6 months before the survey.

survey. Allowable responses were excellent, very good, good,

The reported scores are standardized by age and gender, so

fair, or poor. The outcome has a value of 1 if responses were

that children can be compared with their peers in a national

poor or fair.

sample. Higher scores indicate greater behavior problems.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

• Prosocial behavior. This continuous variable is measured as

− For children ages 4 to 17: Good attention span; sees work

the nationally standardized prosocial score from the SDQ.

through to the end. For children age 3: Good attention

span; sees tasks through to the end.

The SDQ test was administered to parents during the parent

survey, asking about all focal children between ages 3 years

• Peer problems—

and 17 years, 11 months. Parents were asked the same ques-

− Generally liked by other youth/children.

tions for all focal children, although the wording of some

questions varied depending on the age group: 3-year-olds

− For youth ages 11 to 17: Would rather be alone than with

(Question F21a–y), 4- to 10-year-olds (Question F22a–y),

other youth. For children ages 3 to 10: Rather solitary;

and 11- to 17-year-olds (Question F23a–y).

prefers to play alone.

The SDQ contains 25 questions split into five sections: emotional

− Has at least one good friend.

symptoms, conduct problems, hyperactivity, peer problems,

− Picked on or bullied by other youth/children.

and prosocial. Each section asks a set of five descriptions

related to the aforementioned section headings. The respons-

− Gets along better with adults than with other children.

es to the descriptions are 1 = not true, 2 = somewhat true,

• Prosocial—

and 3 = certainly true. The items from each section follow.

− Considerate of other people’s feelings.

• Emotional symptoms—

− For children ages 11 to 17: Shares readily with other youth;

− Often unhappy, depressed, or tearful.

for example, shares books, games, and food. For children

− Often complains of headaches, stomach-aches, or sickness.

ages 3 to 10: Shares readily with other children; for example,

shares toys, treats, and pencils.

− Many worries or often seems worried.

− Helpful if someone is hurt, upset, or feeling ill.

− For children ages 11 to 17: Nervous in new situations;

easily loses confidence. For children ages 3 to 10: Nervous

− Kind to younger children.

or clingy in new situations; easily loses confidence.

− Often offers to help others (parents, teachers, other children).

− Many fears; easily scared.

The descriptions are more often worded with negative

• Conduct problems—

valence—”not true” = 0 points, “somewhat true” = 1 point,

and “certainly true” = 2 points—such that more negative be-

− For children ages 4 to 17: Often lies or cheats. For children

haviors or tendencies are given more points. Some questions

age 3: Often argumentative with adults.

are worded with positive valence, however, such as “Gener-

− Often loses temper.

ally liked by other children.” Questions of a positive valence

are scored so that “not true” is given 2 points, “some what

− Generally well behaved; usually does what adults request.

true” is given 1 point, and “certainly true” is given 0 points.

− For children ages 11 to 17: Often fights with other youth or

This scoring arrangement is such that higher scores for the

bullies them. For children ages 3 to 10: Often fights with

Total Difficulties score indicate more negative behavior.

other children or bullies them.

Each of the five sections is given an individual score ranging

− For children ages 4 to 17: Steals from home, school, or

from 0 to 10. Only the first four sections are included in the

elsewhere. For children age 3: Can be spiteful to others.

overall Total Difficulties score; the prosocial scale is exclud-

ed. The Total Difficulties score has a range from 0 to 40.

• Hyperactivity—

To compute a total score, at least two-thirds (three of five)

− Easily distracted; concentration wanders.

of the questions within each domain had to be answered. If

one or two items were missing within a domain, the average

− Restless, overactive; cannot stay still for long.

score of those items was multiplied by 5 to impute the

− Constantly fidgeting or squirming.

total score for that domain. If more than three items in any

domain were missing, both the domain score and the total

− For children ages 4 to 17: Thinks things out before acting.

problems score were counted as missing. The scores were

For children age 3: Can stop and think things out before acting.

also standardized by age and gender.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

7

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

Child Outcomes for Specific Age Groups

1 = not at all.

Ages 2 to 5 Years2

2 = not very much.

• Preschool or Head Start enrollment. This binary variable

3 = some.

measures whether a child within the specified age range

4 = pretty much.

(2 to 5 years) was enrolled in preschool or center-based

5 = very much.

childcare at the time of the 37-month followup survey and

is based on responses to Questions F10a and F11_2 on the

• Child care or preschool conduct problems. This binary

parent survey.3 The adult respondent was first asked about

variable is based on responses to Questions F14 and F16. The

the number of different childcare arrangements or schools

outcome measures whether or not the parent reports having

the child had been in for at least 10 hours a week at the time

been contacted by the child’s school or childcare provider

of the 37-month survey (F10a). If the child had at least one

regarding the child’s conduct problems or if the child was

arrangement, the respondent was then asked to describe the

suspended or expelled. The outcome uses the following values.

type of care from the following list.

0 = no calls to parent.

− Family-based care in someone’s home with other children.

1 = parent got reports of bad conduct or suspension/expulsion.

− Preschool- or center-based care.

−

Ages 2 Years to 5 Years, 6 Months

Childcare provided in my home.

• Met developmental milestones. This outcome is defined

If the response was preschool- or center-based care, then the

as scoring above the typical developmental cutoffs in all five

indicator for preschool or Head Start enrollment was coded

domains of the Ages and Stages Questionnaire (ASQ-3). This

as yes.

binary variable indicates if the child passed all five domains

in the ASQ-3 corresponding to his or her age. The study used

• Child care or preschool absences in past month. This

9 versions of the ASQ-3 for children ages 23 to 66 months.

variable is treated as continuous based on responses to Ques-

In the 20-month data collection, the ASQ-3 was collected for

tion F13 for children ages 1 year, 6 months to 17 years, 11

focal children less than 3 years, 6 months. In order to capture

months. Respondents were asked to report on the number

the same set of children at the later followup point, the outcome

of days the child missed school in the month prior to the

was collected for focal children up to age 5 years, 6 months.

survey (or the past month of school if the survey is conduct-

ed during the summer [F6_4]). The outcome is measured

The ASQ-3 is a developmental assessment for children of ages

using a scale of 0 to 3: 0 = no absences, 1 = 1 to 2 absences,

from 1 month to 5 years, 6 months. At 37 months, the study

2 = 3 to 5 absences, 3 = 6 or more absences.

team administered the test to the parents, asking about all focal

children up to age 5 years, 6 months. The test was typically

• Positive child care or school experiences. This variable

administered directly after the parent survey.

is based on responses to F17, in which parents assess the

child’s childcare or school experiences, using these ratings—

The study team administered 9 versions of the test across the

1 = mostly positive.

age groups.

0 = both positive and negative.

• The 24-month test for focal children ages 23 months through

-1 = mostly negative.

25 months, 15 days.

• Positive child care or preschool attitudes. This continuous

• The 27-month test for focal children ages 25 months and

variable is constructed from responses to Question F16 in

16 days through 28 months, 15 days.

which the parent assesses the child’s attitude toward school

• The 30-month test for focal children ages 28 months and

or childcare. The question is worded as follows.

16 days through 31 months, 15 days.

How much does your child like school/childcare? The

• The 33-month test for focal children ages 31 months and

response options were—

16 days through 34 months, 15 days.

2 This age group includes focal children who were ages 4 years or younger on the September 1 prior to the 37-month survey. These children had thus not reached 5 years of age (typical school age) in the school year corresponding to the 37-month survey.

3 Although this enrollment outcome is named differently than the school enrollment outcome for children ages 5 to 17 years, the two outcomes are defined identically. For this younger group, enrollment is overwhelmingly in preschool or center-based childcare. For the older group, enrollment is overwhelmingly in school.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

8

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

• The 36-month test for focal children ages 34 months and

A small number (19 of 697, or 2.7 percent) of cases had dis-

16 days through 38 months, 30 days.

crepancies between the age of the child in our survey and the

version of the ASQ-3 administered. In 15 of the 19 cases, the

• The 42-month test for focal children ages 39 months

structure of the ASQ-3 survey allowed for imputation of scores,

through 44 months, 30 days.

resulting in a final missing data rate of less than 1 percent (0.6

• The 48-month test for focal children ages 45 months

percent) because of age-version discrepancies (details about

through 50 months, 30 days.

imputation and missing data are available on request).

• The 54-month test for focal children ages 51 months

Ages 3 Years, 6 Months to 7 Years

through 56 months, 30 days.

• Verbal ability. This outcome is measured as the nationally

•

standardized score from the Woodcock-Johnson III (WJ III)

The 60-month test for focal children ages 57 months

Letter-Word Identification test.

through 66 months, 30 days.

• Math ability. This outcome is measured as the nationally

The test is structured the same for each version of the test, although

standardized score from the WJ III Applied Problems test.

the questions differ. Each test has six sections: (1) com mun-

ication, (2) gross motor, (3) fine motor, (4) problemsolving,

The study team administered two tests from the WJ III tests of

(5) personal-social, and (6) overall. The first five sections have

achievement to eligible sample children ages 3 to 7 years—the

six questions that can be answered “yes,” “sometimes,” or

Letter-Word Identification test, with 76 possible questions,

“not yet.” The last section—Overall—has between 8 and 10

and the Applied Problems test, with 63 possible questions. The

open-ended questions that are not included in the final score.

interviewers began tests at Question1 regardless of age. Inter-

viewers did not calculate final raw scores in the field because it

Each “yes” answer receives 10 points, “sometimes” answer

is subject to error. The analysis team independently calculated

receives 5 points, and “not yet” answer receives 0 points. The

raw scores.

scores for each section range from 0 to 60. A raw score was

calculated separately for each of the five sections. A section is

For both the Letter-Word Identification and Applied Problems

scored when at least four of the six questions in the section

tests, the study team calculated raw scores based on a series of

are answered. When all six questions are answered, the scores

rules. First, children were allowed to refuse the test either at the

from those six available answers are summed together. When a

beginning or during the test. Refusals were coded as missing

section has four or five answers, the missing scores are assigned

test scores. The raw score was a sum of all the correct answers,

a value derived from the average of the available scores, and all

starting at Question 1, until the child answered six consecutive

six scores are summed.

questions incorrectly. Each question was weighted the same,

with a value of “1” indicating a correct response and a value of

The ASQ-3 has national norms for scores for each domain by

“0” indicating an incorrect response.

each age version and a raw score that can be used as a diagnos-

tic cutoff point for the domain set at two standard deviations

Exhibit B-1 shows the final distribution of the number of

below the mean. Scores greater than the cutoff are considering

children who completed the WJ III tests.

“passing,” whereas scores at or below the cutoff may indicate

the potential presence of a developmental delay in that domain

Exhibit B-1. Children’s Completion Rates at 37 Months

and can be used for making a referral for a more comprehen-

for WJ III Letter-Word Identification and Applied Problems

sive assessment. The team calculated z-scores for each domain

Tests

by subtracting the domain raw score from the national domain

WJ III Test Distribution

N

Percent

mean (for the appropriate age version) and dividing by the

Total children completed WJ III

850

100.0

national domain standard deviation. For the outcome measure,

Both tests (letter-word identification and

815

95.9

raw scores in each domain were compared with the cutoff

applied problems) completed

scores and scores above the cutoff were counted as passing. The

Only letter-word identification test completed

17

2.0

Only applied problems test completed

18

2.1

outcome then was whether children passed all valid domains.

WJ III = Woodcock-Johnson III.

If children were missing one of the five domains, they were still

Source: Family Options Study Child Assessment Data

included in the outcome and were assessed on whether they

passed all four nonmissing domains.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

9

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

The study team entered raw scores into the WJ III Compuscore

Children who answered 5 or fewer of the 10 questions incor-

software to calculate z-scores that are age and gender adjusted.4

rectly moved to the second set of 10 questions. For the second

Raw scores were entered into “Woodcock-Johnson III Test of

part, children were instructed to touch their knees when told

Academic Achievement—Form A” for “Standard Battery” tests 1

to “touch your shoulders” and to touch their shoulders when

and 10, where test 1 is the Letter-Word Identification test and

told to “touch your knees.” This principle of doing the opposite

test 10 is the Applied Problems test.

of what is told was the same for this part of the test—the only

change was the body part touched. The children were taken

The study team exported the key information into an Excel file

through a second demonstration in which they practiced

and entered raw scores into the WJ III Compuscore software

touching their knees when told to touch their shoulders and

that converts raw scores into the z-scores used in report

touching their shoulders when instructed to touch their knees.

analyses. Data from the Excel file were copied and pasted rather

After practice and only one reminder, the second set of 10

than entered manually into Compuscore to reduce data entry

questions began.

error. Within the Compuscore software, the following infor-

mation was entered: first name, gender, study identification

In the second set of 10 questions, children received all four

number and date of birth, and date of testing. On exporting

instructions—touching the head when instructed to touch the

data from the Compuscore software, the “Norm Basis” of age

toes and vice versa and touching the shoulders when instructed

and the “Standard Set” of scores were selected for inclusion in

to touch the knees and vice versa, adding to the complexity

the export, resulting in a comma-delimited file that includes,

of the test. The scoring was the same, with correct responses

among the standard set of scores, the z-score of each test for

receiving 2 points, self-corrected responses receiving 1 point,

each child.

and incorrect responses receiving 0 points.

• Executive functioning (self-regulation). This outcome is

Missing values were imputed if two-thirds of the overall

measured with the score on the Head Toes Knees Shoulders

questions children were administered were nonmissing. The

(HTKS) assessment.

scores assigned to these missing values were the average from

the answered items multiplied by the total number of trials

HTKS is a development assessment testing inhibitory control,

each child was eligible for, so that, if a child were administered

attention, and working memory. The study team administered

10 trials, the imputation would be the average score of the

this test directly to focal children ages 3 years, 6 months to

answered items times 10. Children could receive a total score

7 years, 11 months. All children were given the same test,

of between 0 and 40 points from the 20 questions.

regardless of age.

The test consists of 20 questions divided into two parts. The test

Ages 5 to 17 Years5

began with a demonstration of the exercise in which the chil-

In this age group, the last four outcomes about school-aged

dren were instructed to touch their toes when they were told to

children (school absences, school experiences, school attitudes,

“touch your head” and to touch their head when told to “touch

and school conduct problems) were collected from only the

your toes,” in effect doing the opposite of what they were told.

first 38 percent of parents surveyed due to an error in data

After some practice and repeated reminders to make sure that

collection. The parallel outcomes for younger, preschool-aged

the children understood the instructions, the assessment began.

children were collected from all parents. Because of the dis-

The first 10 questions instructed the children to “touch your

crepancy in data collection between the two age groups, these

head” or “touch your toes.” The responses to “touch your head”

outcomes are analyzed separately by age group.

would be for the child to touch his or her toes (the correct

• School enrollment. This binary variable is based on re-

response), to motion toward touching his or her head and then

sponses to Questions F6, F8, and F9. The adult respondent

correct him/herself and touch his or her toes (a self-corrected

was asked about enrollment of children ages 5 to 17 years

response), or to touch his or her head (the incorrect response).

in school (ages 5 to 17). If the respondent answered yes or

Each correct response received 2 points, each self-corrected

volunteered information that the child is homeschooled or

response received 1 point, and each incorrect response received

on summer/school vacation and the response to the child’s

0 points. If a child had 6 or more incorrect responses in the first

highest grade or year of school completed was not that they

10 questions, the test was discontinued.

were currently in any form of childcare or preschool (F8),

4 Reference the WJ III technical manual for details about the z-score (McGrew, Shrank, and Woodcock, 2007).

5 This age group includes focal children who were ages 5 to 17 years on the September 1st prior to the 37-month parent survey and no older than 17 years at the time of the survey. These children were thus typical school age in the school year corresponding to the 37-month survey.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

10

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

then enrollment was indicated as yes. Also, if the respondent

• Anxiety. Question A1 on the child survey is used to create

indicated that the child’s highest grade or year of school that

an indicator of anxiety. Question A1 is the A-Trait scale

he or she ever completed was the 12th grade and the child

from the State-Trait Anxiety Inventory for Children, or

received a high school diploma, then school enrollment was

STAIC (Spielberger et al., 1973). Scores range from 20

also indicated as yes.

to 60, with higher scores indicating greater anxiety. The

scale is proprietary, so only a partial list is shown here.

• School absences in past month. This variable is treated as

Respondents reported on the frequency with which they

continuous based on responses to Question F13 for children

felt several items using these response options—

ages 5 to 17 years. Respondents were asked to report on

the number of days the child missed school in the month

1 = hardly ever.

prior to the survey (or the past month of school if the survey

2 = sometimes.

is conducted during the summer [F6_4]). The outcome is

3 = often.

measured using a scale of 0 to 3: 0 = no absences, 1 = 1 to 2

absences, 2 = 3 to 5 absences, 3 = 6 or more absences.

Examples of the items are—

−

•

I worry about making mistakes.

Positive school experiences. This variable is based on

responses to F17, in which parents assess the child’s school

− I have trouble deciding what to do.

experiences, using these ratings—

− I worry about things that may happen.

1 = mostly positive.

• Fears. This outcome is based on responses to Questions

0 = both positive and negative.

B1 to B33 of the child survey. Respondents reported on the

-1 = mostly negative.

extent to which they had fears using these response options.

•

1 = not at all.

Positive school attitudes. This continuous variable is

constructed from responses to Question F16 in which

2 = some.

the parent assesses the child’s attitude toward school. The

3 = a lot.

question is worded as follows.

Questions B1 to B33 asked about the following fears:

How much does your child like school/childcare? The

spiders, getting sick, being robbed, having no friends, dogs,

response options were—

what will happen in the future, having no place to live,

1 = not at all.

something bad happening to people in my family, snakes,

getting bad grades, people fighting, being teased, what

2 = not very much.

other people think of me, being hit by a car or truck, drug

3 = some.

dealers, being alone, flunking school, gangs, being lost,

4 = pretty much.

rats, that other children/tweens will not want to play/spend

5 = very much.

time with me, police, having no place to sleep, dying,

nightmares, being hungry, having no food to eat, being sent

• School conduct problems. This binary variable is based on

to the principal, guns, fire, losing my favorite stuff, I worry

responses to Questions F14 and F16. The outcome measures

about my parents, I worry about my brothers and sisters,

whether or not the parent reports having been contacted by

I worry about myself. This outcome is measured using

the child’s school regarding the child’s conduct problems or

the Fears Scale (Ramirez, Masten, and Samsa, 1991). The

if the child was suspended or expelled. The outcome uses

response scores were summed, yielding total scores ranging

the following values.

from 33 to 99, with higher scores indicating greater fear.

0 = no calls to parent.

• Substance use. This outcome has values of 0 to 2 and is

1 = parent got reports of bad conduct or suspension/

based on responses to Questions D6 to D23 on the child

expulsion.

survey. The outcome is measured with 23 items from the

Centers for Disease Control and Prevention 2011 Youth

Ages 8 to 17 Years

Risk Behavior Survey. This outcome measures whether

The study team collected information from children and youth

the child had used tobacco, alcohol, or marijuana in the

ages 8 to 17 years on the child survey from which several out-

past 30 days or had ever used other substances (cocaine,

come measures were constructed. The outcomes are described

inhalants, steroids—ages 8 to 17—or ecstasy, meth, heroin,

in this section.

controlled prescription drugs, or injected drugs—ages 13

to 17 only).

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

11

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

• Goal-oriented thinking. This outcome is measured using

child had had any problems that involved the police contact-

responses to Questions G1 to G6 on the child survey. These

ing the parent in the 6 months before the adult survey.

items are a modified version of the Children’s Hope Scale

(Snyder et al., 1997) which measures positive, goal-oriented

B.5. Measures of Self-Sufficiency

thinking. Scores range from 6 to 30 with higher scores

indicating greater hope. Respondents indicated how

The impact analysis examines effects of the four interventions

frequently they felt six items using these response options—

on several outcomes related to self-sufficiency of sample members.

1 = none of the time.

These outcomes pertain to employment status (adult respondent),

income sources (family), participation in education and training

2 = a little of the time.

(adult respondent), food security (family), and economic hard-

3 = a lot of the time.

ship (family).

4 = most of the time.

Employment status. Adult respondents reported on work activity

5 = all of the time.

in the week before the 37-month followup survey. If they had

completed the 20-month survey, they were asked if they had

The six items were—

worked since the month of their 20-month survey response. If

1. You think you are doing pretty well.

they had not completed the 20-month survey, they were asked

2. You can think of many ways to get the things in life that

whether they had worked at any time since random assignment.

are most important to you.

Those who had worked since the 20-month survey/random

3. You are doing just as well as other kids your age.

assignment answered questions about the number of jobs held

since the 20-month survey/random assignment and the number

4. When you have a problem, you can come up with lots of

of months worked since the 20-month survey/random assign-

ways to solve it.

ment. Sample members who reported having worked for pay in

5. You think the things you have done in the past will help

the week before the 37-month followup survey were asked to

you in the future.

provide details about the number of hours worked per week and

6. Even when others want to quit, you know you can find

earnings at the main job. The study team used this information

ways to solve the problem.

to construct six outcomes.

• School effort in past month. This outcome is constructed

• Work for pay in the week before the survey. This binary

from responses to Questions E4 and E5. Respondents were

variable is based on responses to Question C1. This outcome

asked two questions about their school effort in the month

measures the percentage of survey respondents who reported

before the child survey. The outcome measure ranges from

working for pay in the week prior to the 37-month followup

1 to 4, with higher scores indicating greater effort at school

survey.

and on homework.

• Any work for pay since 20-month survey. This binary

Response options were—

variable is based on responses to Question C2. This outcome

measures the percentage of survey respondents who reported

1 = could have done a lot better.

working for pay at any time since the date of their 20-month

2 = could have done a little better.

survey. It is defined only for families who responded to both

3 = did about as well as you could.

20-month and 37-month followup surveys.

4 = did very well; could not have done better.

• Months worked for pay since 20-month survey (includes

partial months). This binary variable is based on responses

The questions are worded—

to Question C2. This outcome is a count of the months

In the last month, how hard have you worked on your

worked since random assignment, including partial months.

homework?

It is defined only for families who responded to both

20-month and 37-month followup surveys.

In the last month, how hard have you tried to work during

the school day?

• Any work for pay since random assignment. This binary

•

variable is based on responses to Question C2 in the 37-month

Arrests or police involvement in past 6 months. This

survey and (for 20-month respondents) Question C2 in the

binary outcome is constructed from responses from the

20-month survey. This outcome measures the percentage of

parent to Questions F24 and F25 on the parent survey.

survey respondents who reported working for pay at any time

Parents were asked whether the child had been arrested in

since random assignment.

the 6 months before the adult survey and whether the focal

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

12

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

• Months worked for pay since random assignment (includes

• Total family income. Adult respondents were asked in

partial months). This continuous variable is based on responses

Questions C13 through C19 to estimate total annual income

to Question C4 in the 37-month survey and (for 20-month

for the family from all sources for the most recently complet-

respondents) Question C4 in the 20-month survey. This

ed calendar year preceding the 37-month followup survey

out come is a count of the months worked since random

(2013 for all respondents). The study team used responses

assignment, including partial months. Respondents who reported

to these questions to construct a continuous outcome variable

that they had not worked since random assignment were

measuring total annual family income for the family.

assumed to have worked 0 months since random assignment.

Education and training. The study team used the adult survey

• Hours of work per week at current main job. This contin-

to construct five outcomes pertaining to participation in edu-

uous variable is based on responses to Question C5.6 For

cation and training activities during the followup period. Adult

adult respondents who had more than one job in the week

respondents indicated whether they had participated in any

prior to the 37-month survey, the main job is defined as the

education or training activities since random assignment, and,

job at which she or he usually worked the most number of

if so, how many weeks they spent in such programs. For up to

hours per week. For adult respondents who said they did

six programs reported, sample members reported on the type

not work in the week before the survey, the number of hours

of program, using the following response options.

worked was assumed to be 0.

1 = regular high school, directed to high school (HS) diploma.

Income sources and amounts. Question C12 on the 37-month

followup survey asked whether the respondent or anyone in

2 = preparation for a general educational development (GED)

the respondent’s family received income from various sources

exam.

or various types of government assistance in the past month.

3 = 2-year college directed toward a degree.

The study team constructed outcomes that measure the per-

centage of families who reported receiving income from each of

4 = 4-year college directed toward a degree.

the following sources in the month prior to the survey.

5 = graduate courses.

• Earnings.

6 = college courses not directed toward a degree.

• Temporary Assistance for Needy Families (TANF).

• Social Security Disability Insurance (SSDI).

7 = vocational education outside a college (business or

technical schools, employer or union-provided training, or

• Supplemental Security Income (SSI).

military training in vocational but not military skills).

• Supplemental Nutrition Assistance Program (SNAP).

8 = nonvocational adult education not directed toward a degree

• Special Supplemental Nutrition Program for Women,

(basic education, literacy training, English as a second

Infants, and Children (WIC).

language).

The study team also constructed two other outcomes related to

9 = job search assistance, job finding, orientation to the world

income.

of work.

• Annualized earnings from the main job. This continuous

The study team used this information to construct the follow-

variable is based on responses to Questions C6 through C11

ing education and training outcomes.

about wages paid at the main job. The outcome measures

the annualized value of current earnings from the main job

• Participated in any school or training lasting 2 weeks

reported at the time of the 37-month survey. This value

or more since random assignment. This binary variable is

usually represents either the product of the reported hourly

constructed from responses to Question C24. This outcome

wage and usual hours per week multiplied by 52 weeks or

measures the percentage of families in whom the adult

the reported usual weekly earnings multiplied by 52 weeks.

respondent reported having participated in any type of school

By construction, the measure ignores any seasonality in

or training lasting 2 or more weeks since random assignment.

earnings. For adult respondents who said they did not work

• Number of weeks in school or training programs since

in the week before the survey, the annualized earnings are

random assignment. This continuous variable is based on

assumed to be 0.

responses to Question C27.

6 If the respondent reported having more than one job, the interviewers instructed the respondent to provide the number of hours worked at the main job. The main job was defined as the job at which the respondent usually worked the most number of hours per week.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

13

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

• Participated in 2 weeks or more of school since random

Families study (Mills et al., 2006). Five of these six items are

assignment. This binary variable is based on responses to

included in the U.S. Department of Agriculture (USDA) “short

Question C26. School or academic training is defined as

form” measure of food security.7 Item C32 is included in the

regular high school directed toward a high school diploma,

18-item U.S. Household Food Security Survey Module, but not

preparation for a GED exam, 2-year college, 4-year college,

in the “short-form” measure. The excluded item from the “short-

or graduate courses.

form” measure is an item that would be asked after C29: “In the

last 30 days, how many days did this happen?” (__ days).

• Participated in 2 weeks or more of basic education since

random assignment. This binary variable is constructed

Items C30 and C31 were asked only of family heads if the

from responses to Question C26. Basic education is defined

response to item C29 was “Yes.” If a family head responded

as nonvocational adult education such as basic education,

“No” to item C29, responses of “No” were imputed for items

literacy training, or English as a second language) not

C30 and C31 for the purposes of creating food security items

directed toward a degree.

(and so were not considered missing).

• Participated in 2 weeks or more of vocational education

Household is food insecure. The first outcome is a binary

or training since random assignment. This binary variable

variable that equals 1 if a household was “food insecure” ac-

is constructed from responses to Question C26. Vocational

cording to criteria used by the USDA. Survey respondents were

education or training is defined as vocational education

asked a series of questions used by USDA and the U.S. Census

outside a college such as business or technical schools,

Bureau to measure food security. Two or more affirmative

employer- or union-provided training, or military training in

answers to these questions meant that a household was consid-

vocational skills (not military skills).

ered “insecure” at the time of the followup survey. (Responses

to C28a and C28b of either “often true” or “sometimes true”

Food security. The study team collected information about food

were considered affirmative answers.)

security on the adult survey in Questions C28 through C32.

Food insecurity scale. This outcome measures the food

C28a. The first statement is “We couldn’t afford to eat balanced

insecurity level of each family based on responses to the USDA

meals.” Was that often, sometimes, or never true for you

food security questions included on the followup survey. The

in the last 30 days?

outcome was defined as the total number of affirmative answers

C28b. The second statement is: “The food that I bought just

to the six items. The food insecurity scale ranges from 0 to 6,

didn’t last, and I didn’t have money to get any more.”

with higher values indicating greater food insecurity.

Was that often, sometimes, or never true for you in the

Economic stress. The study team also measured the economic

last 30 days?

hardship reported by each family at the time of the 37-month

C29. In the past 30 days, did you or other adults in your

followup survey on a measure derived from Pearlin and

household ever cut the size of your meals or skip meals

Schooler (1978). Questions 33a through 33d asked the adult

because there wasn’t enough money for food? (Yes/No)

respondents to report on the frequency with which the family

experienced four items related to economic hardship in the

C30. In the last 30 days, did you ever eat less than you felt

6 months before the survey. The response options were (1 =

you should because there wasn’t enough money to buy

never; 2 = once in a while; 3 = fairly often; and 4 = very often).

food? (Yes/No)

The question was worded as follows.

C31. In the last 30 days, were you ever hungry but didn’t eat

How often does it happen that you do not have enough money

because you couldn’t afford enough food? (Yes/No)

to afford—

C32. In the last 30 days, did you or other adults in your

• The kind of medical care your family should have?

household ever not eat for a whole day because there

wasn’t enough money for food? (Yes/No)

• The kind of clothing your family should have?

These six items are the same six items that were asked in the

• The leisure activities your family wants?

followup survey of the Effects of Housing Vouchers on Welfare

• Your rent?

7 See Nord, Andrews, and Carlsen (2005). Our assessment of food insecurity is based on two USDA “short form” metrics, which are scores assigned to households based on answers to six survey questions.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

14

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

Question C34 asked how the family’s finances usually work out

date, under the assumption that more recent data would have

at the end of the month, with these possibly response codes—

the most recently updated information.9 After removing these

1 = some money left over.

duplicate records, the study team aggregated records in each

extract to the person, state, and quarter level. We then merged

2 = just enough money to make ends meet.

the data in the extracts together to create a single file at the

3 = not enough money to make ends meet.

person, state, and quarter level. When quarters overlapped, we

For both questions, higher values indicate higher economic stress.

kept data in the most recent extract (again, under the assump-

The economic stress outcome is calculated for cases in which

tion that more recent data would have more recently updated

four of the five items (Question 33a through 33d and Question 34)

information). Finally, we summed across state to produce a file

are nonmissing. For Questions 33a through 33d, the responses

at the person-quarter level.

were recoded into a scale ranging from less economically

At this point, we implemented various data cleaning rules. We

stressed to more economically stressed where 1 = -1, 2 = -0.33,

observed a handful of records with quarterly earnings greater

3 = 0.33, 4 = 1. For Question 24, responses were recoded as

than $25,000. Based on the study sample of homeless families,

1 = -1, 2 = 0, 3 = 1. The nonmissing recoded responses were then

the study team decided that such observations likely do not

averaged. The economic stress scale ranges from -1 to 1, with

represent actual wages, but instead are the result of data entry

higher values indicating higher economic stress. The outcome,

errors. As a result, quarterly earnings greater than $25,000

expressed as an economic stress scale, measures the extent of

were recoded to missing values and excluded from analysis.

hardship using the responses about the frequency with which

On the other side of the wage scale, the study team observed a

the family said they experienced an inability to afford medical

number of very low quarterly earnings. The team reasoned that

care the family needed, clothing the family needed, leisure

earnings below some minimum threshold likely do not reflect

activities the family wanted, or rent. The economic stress scale

meaningful employment, but rather a one-time payment or

also takes into account the adult respondent’s assessment of the

other data anomaly. We selected a minimum threshold of $58

family’s monthly finances; that is, whether they usually have

(equal to the earnings a worker would receive by working a

some money left over at the end of the month, barely enough

single 8-hour day at the federal minimum wage of $7.25/hour),

to make ends meet, or not enough to make ends meet.

and recoded earnings below this threshold to $0.

Any individuals without a record in the aggregate person-quarter

B.6. Measures of Earnings and

file was deemed to have zero earnings for that quarter. We

Employment From NDNH Data

adjusted wages to constant 2015Q3 dollars according to the

Consumer Price Index for All Urban Consumers, or CPI-U,

The study team defined various earnings and employment

produced by the U.S. Bureau of Labor Statistics. Finally, we

outcomes using data from the National Directory of New Hires

considered an individual to be “employed” in a given quarter if

(NDNH). This section provides details of the steps required

they have positive earnings in that quarter (after all the adjust-

to clean and prepare the data for analysis and the definition of

ments to earnings described in this section).

outcomes generated from the NDNH data.

Defining Outcomes

Data Cleaning and Processing

We defined outcomes using reference periods based on time

Prior to creating the final outcomes, the study team undertook

since random assignment (rather than on calendar time). Be-

a variety of cleaning and other data preparation steps. OSCE

cause participants in the Family Options Study were randomly

provided HUD with four extracts (pulled from the NDNH

assigned between 2010Q3 and 2012Q1, different families have

database June 2015, October 2015, January 2016, and March

different intervals relative to random assignment of collected

2016) of de-identified quarterly wage records. The extracts

data. For the earliest enrolled families, data are available in

contain indicators for person, state, quarter, and firm, as well as

quarters 11 to 20 after random assignment (using the conven-

the date when OCSE processed the record.8 We found several

tion that the calendar quarter in which random assignment

instances of multiple records for a given person, at the same

took place is quarter “0”). For the latest enrolled families, data

firm, in a single state and quarter in the same extract. In these

are available in quarters 5 to 14 after random assignment.

cases, we used only the record with the most recent processed

8 Firms were identified with a randomly generated numeric code. This code allowed the study team to understand when different records contained information from the same employer.

9 Some records were missing a firm code. Because we could not be certain whether these records had been updated by a more recent record (through match on person, firm, state, and quarter), these records were always kept.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

15

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

The study team decided to define outcomes for the sample for

when the child returns to the parent’s custody, is placed with

the entire period 11 to 14 quarters after random assignment

a permanent caregiver through legal guardianship or adoption,

(rather than defining outcomes for each quarter). This interval

reaches the age of majority, or some other permanent living

provides for complete data for the entire sample, and contains

arrangement. Defined as a continuous period a child is under

exactly four quarters of observations to smooth out seasonal

the care of the child welfare agency, one spell can include

effects. Three outcomes were defined.

multiple out-of-home placements. Additionally, a child who

subsequently reenters out-of-home care after exiting out-of-

1. Earnings in quarters 11 to 14 after RA (2015Q3$). This

home care will have multiple spells.

outcome is the sum of quarterly earnings during the 11th

to 14th quarters after the quarter of random assignment.

The definition of an out-of-home spell in the Family Options

Amounts are in 2015, Quarter 3 dollars.

Study differs somewhat from the definition commonly found

in the literature. Although entrance to the spell is the same, in

2. Any employment in quarters 11 to 14 after RA. This

the study definition, spells end only when a child is reunified

outcome indicates whether the individual was employed

with his or her family or when the child reaches his or her 18th

(that is, had positive earnings) in any of the four quarters in

birthday. The spell does not end when the child leaves out-

the period.

of-home care because of adoption or guardianship. Using this

3. Number of quarters employed in quarters 11 to 14 after

spell definition and consistent followup periods within site, we

RA. This outcome measures the number of quarters in the

defined two outcomes for the family head based on the child

period in which the individual was employed (this outcome

welfare agency data.

could have a value of 0 to 4).

1. Had a formal child separation that began after random

Impacts were estimated for six comparisons: SUB versus UC,

assignment. This outcome indicates whether the head has

CBRR versus UC, PBTH versus UC, SUB versus CBRR, SUB

any child separation in the administrative data that begins

versus PBTH, and CBRR versus PBTH. Due to limitations

after random assignment during the site-specific followup

designed to protect the confidentiality of NDNH data, the study

period. It does not include separations that began prior to

team used only indicators for the site randomization ratio as a

random assignment.

covariate for estimating impacts.

2. Total days during followup separated from at least one

child. This outcome measures the total number of days

B.7. Measures of Formal Child

during the site-specific followup period when the family

Separations From Child Welfare

head was separated from at least one child, as measured in

Agency Records

the administrative data. This measure includes all formal

separations started either before or after random assignment.

The study collected child welfare agency records for families

in five sites. The coverage periods for these data go through

B.8. Program Use Outcomes

November or December of 2014. To determine the length

of the followup period for each site, the study team used the

The study team used the Program Usage Data to create the out-

minimum followup duration in each site, which was the date

comes reported in the report’s program use exhibits. The study

of the last random assignment until the end of the coverage

team created each type of program use outcome for each of seven

period. A followup end date was then calculated for each family

program types: (1) permanent housing subsidies offered to the

head by adding the length of the followup period to their date

SUB group, (2) rapid re-housing rental assistance, (3) transitional

of random assignment. For example, the followup period for

housing, (4) permanent supportive housing (PSH), (5) public

a family randomly assigned on January 1, 2011 in Baltimore

housing, (6) project-based vouchers/Section 8 projects, and

would end on December 6, 2013—1070 days (or 35 months)

(7) emergency shelter.

after random assignment.10

• Ever used a particular program type. These series of

The notion of an out-of-home spell is a key construct in the

binary variables were coded as 1 if any monthly binary

child welfare literature used to measure the length of time a

indicator from the calendar month of random assignment

child is separated from his or her parent. It begins with the

to the calendar month of the 37-month followup survey

day the child is removed from his or her parent and ends

response indicated use of the program type.

10 The followup periods are Alameda County=1,075 days, Baltimore=1,071 days, Kansas City=1,069 days, Minneapolis=1,046 days, and Phoenix=1,123 days.

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

16

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

• Used a particular program type in the survey month.

In the Program Usage Data prepared for the Short-Term Impacts

These series of binary variables were coded as 1 if the

report, emergency shelter stays were processed using the

monthly binary indicator from the calendar month of the

same assumptions as transitional housing and PSH. In the

37-month followup survey response indicated use of the

new Program Usage Data prepared for this report, durations

program type.

in emergency shelter are based on entry and exit dates in the

source data for emergency shelter (92 percent of which was

• Number of months of use of a particular program type.

from HMIS records), rather than on adjusted counts of monthly

These continuous variables were defined using assumptions

dummy variables for emergency shelter use. Durations in days

about how families use the various homeless and housing

are converted into durations in months by multiplying day

programs. Because the monthly indicator variables in the

durations by (12/365). Therefore, the measures of numbers of

Program Usage Data are coded as 1 if a particular program

months of emergency shelter use in this report are prepared

was used at least 1 night of a particular calendar month, simply

in a different manner than the duration measures for all other

counting the number indicator variables equal to 1 would

program types. As emergency shelter is the program type most

systematically inflate measures of program use. The study

likely to have stays of less than a month, it is the program

team assumed—

type where the method of adjusted counts of monthly dummy

− Entry to all program types could happen at any time

variables is most likely to be biased upward (when a family has

during the month.

short stays in shelter in consecutive months). Given the high

monthly costs of emergency shelter, basing emergency shelter

− Exits from transitional housing and PSH could happen at

durations on actual entry and exit dates is particularly import-

any time during the month.

ant to guard against upward bias in program cost estimates.

− Exits from SUB, CBRR, public housing, and project-based

Appendix E shows impact estimates for additional program use

vouchers/Section 8 projects always happened at the end

outcomes. The outcomes that measure any use of a particular

of the month, because assistance is provided in monthly

program type (or types) during months 0 to 32 or months 7 to

increments.

32 are coded as 1 if any monthly binary indicator during the

These assumptions were developed and confirmed with

relevant time period indicated use of the program type (or types).

practitioners in the field. In addition, the study team assumed

The outcomes that measure number of months of emergency

all stays in the followup survey month (for all program types)

shelter or transitional housing during months 0 to 32, number

extended to the end of the month, because the end of the ob-

of months of emergency shelter during months 0 to 32, and

servation “window” was an artifact of data collection. A single

number of months of transitional housing during months 0 to

“stay” of a program type was identified in the data by month

32 are created in a nearly identical manner to the number of

indicators before and after a stay with no use of that particular

months outcomes described previously. The only difference is

program type. Using these assumptions as a basis for correcting

how transitional housing stays that include the 32nd month

counts meant—

after random assignment are counted. If the 33rd month

indicator showed use of the transitional housing, then it was as-

For stays longer than 1 month—

sumed that the transitional housing stay continued through the

• Subtracting 1 month from counts of calendar months for

end of the 32nd month. If the 33rd month indicator showed no

emergency shelter, transitional housing, and PSH stays.

use of transitional housing, then it was assumed that the stay

ended at some point during the month. For transitional hous-

• Subtracting 1/2 month for stays in SUB, CBRR, public

ing stays of more than 1 month that included the 32nd month,

housing, and project-based vouchers/Section 8 projects.

either a full month in the 32nd month was counted (if the stay

continued to the 33rd month) or 1/2 month in the 32nd month

For stays that lasted a single calendar month—

was counted (if the stay did not continue to the 33rd month).

• Stays for transitional housing, and PSH were shortened to

For single-calendar-month transitional housing stays in the

1/4 month (1/4 month is the expected length assuming that

32nd month, either 1/2 month in the 32nd month was counted

entry and exit are equally likely at any point in the month).

(if the stay continued to the 33rd month) or 1/4 month in the

32nd month was counted (if the stay did not continue to the

• Stays for SUB, CBRR, public housing, and project-based

33rd month). The durations in emergency shelter stays are

vouchers/Section 8 projects were shortened to 1/2 month

measured with entry and exit dates in the source data.

(1/2 month is the expected length assuming that entry is

equally likely at any point in the month and that exit occurs

Appendix I shows impacts on the length of the baseline stay

at the end of the month).

in emergency shelter (for families with a baseline stay in the

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

17

FAMILY OPTIONS STUDY

Appendix B. Construction of Outcomes

Program Usage Data). These lengths of stay begin at random

Impacts report, when the length of baseline stay in emergency

assignment and are based on exit dates in the source data, rath-

shelter was based on adjusted counts of monthly dummy

er than on adjusted counts of monthly dummy variables. This

variables.

outcome construction differs from that used in the Short-Term

B

3-Year Impacts of Housing and Services Interventions for Homeless Families

18

FAMILY OPTIONS STUDY

APPENDIX C.

ANALYSIS METHODS

This appendix provides details about the impact impact assignment to a group that offers a less-costly estimation used in the report, including covariates

intervention (CBRR)?

used in the impact models, imputation of missing

• What is the impact of having priority access to a housing

data, family/adult weights, child weights, and the multiple

subsidy with no time limit (SUB) compared with the

comparisons adjustment for confirmatory hypothesis tests.

impact of having priority access to interventions that offer a

time-limited housing subsidy (PBTH + CBRR)?

C.1. Methodology

The explanation of the estimation model begins with some

This report presents separate impact estimates for each of the

terminology that describes how random assignment was

6 pairwise comparisons of a single assignment group to another

implemented in this study. Enrollment and random assignment

assignment group, plus 4 additional comparisons of pooled

was a multistep process. The PBTH, CBRR, and (in some

assignment groups to a single assignment group (see Exhibit 1-1

sites) SUB interventions had multiple service providers in

and Chapters 3 through 6). The four assignment groups are

each site. Before random assignment, the number of slots

(1) SUB, in which families have priority access to a permanent

currently available at all providers for each of the interventions

housing subsidy; (2) CBRR, in which families have priority

was assessed. An intervention was deemed available if at least

access to community-based rapid re-housing; (3) PBTH, in

one slot at one provider of that intervention in the site was

which families have priority access to project-based transitional

currently available. After an intervention was determined to be

housing; or (4) UC, in which families do not have priority

available, the interviewer asked the family a series of questions

access to any particular program. All 10 comparisons have been

to assess provider-specific eligibility for the available inter-

analyzed separately using the same basic estimation model.

ventions and programs. A family was considered eligible for a

particular intervention if the household head’s responses to the

eligibility questions showed that the family met the eligibility

requirements for at least one provider of that intervention that

currently had an available slot. For example, some programs

required that families have a source of income that would allow

for them to pay rent on their own within a designated period

of time. The study team thus asked families if they wanted to

be considered for programs with such an income requirement.

Other programs required families to pay a monthly program

Pooled Comparisons

fee, and the screening question asked if families wanted to be

• What is impact of having priority access to any kind of

considered for programs with this type of requirement.

housing subsidy for homeless families (SUB + CBRR +

Other programs required participants to demonstrate sobriety,

PBTH) compared with the impact of usual care (UC)?

pass criminal background checks, or agree to participate in case

• What is the impact of having priority access to a housing

management or other services. The study team asked screening

subsidy with heavy services on homeless families (PBTH)

questions for these questions that ascertained families’ willing-

compared with the impact of having priority access to a

ness to be considered for programs with these requirements.

housing subsidy with light or no services (SUB + CBRR)?

To undergo random assignment, a family needed to be eligible

• What is the impact of having priority access to interventions

for at least one available intervention in addition to UC.1 Based

that are more costly (PBTH + SUB) compared with the

on this approach to random assignment, each family has a

randomization set.

1 Altogether, 183 of the screened families were not eligible for any available interventions besides UC. These families were not enrolled in the study.

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

The set of interventions to which it was possible for a family to

C = a vector of background characteristics2 of family i,

i

be assigned was determined by considering both the availability

of the intervention and the assessed eligibility of the family. In

I = indicator variable for “site-RA regime”3 k for family i,

k, i

the study, each family has one of seven possible randomization

e = residual for family i (assumed mean-zero and i.i.d. [inde-

sets. These sets are {PBTH, SUB, CBRR, UC}, {PBTH, SUB, UC},

i

pendently and identically distributed]),

{PBTH, CBRR, UC}, {SUB, CBRR, UC}, {PBTH, UC}, {SUB,

UC}, and {CBRR, UC}.

a = a constant term, and

q, r

The randomization set of each family determines the pairwise

b = other regression coefficients.

q, r

comparisons in which the family is included. A family is included

The estimate of the impact parameter d is the intention-

in the pairwise comparisons of its assigned intervention with

q, r

to-treat, or ITT, estimate. For the pairwise comparisons, it is

the other interventions in its randomization set. For example,

an estimate of the average effect of being offered intervention q

families assigned to the PBTH intervention with randomization

rather than intervention r. The average effect is taken over all

set {PBTH, SUB, UC} are included in these two pairwise com-

families in the q,r comparison, regardless of whether families

parisons: PBTH versus UC; and SUB versus PBTH.

actually participated in the intervention to which they were

assigned.

Impact Estimation Model for Family and

Adult Outcomes

This model assumes that the true impact of intervention q

relative to intervention r is homogeneous across sites. The

For each pairwise comparison, the study team estimated impacts

impact parameter is thus implicitly a weighted average of the

for the sample of families who (1) had both interventions in

point estimates of site-level impacts, with each site-level impact

their randomization set and (2) were randomly assigned to one

weighted by the number of families in the site.

of the two interventions. The team used multivariate regression

to increase the precision of our impact estimates and to adjust

A slight modification of this model is used to estimate impacts

for any chance imbalances between assignment groups on

in the pooled comparisons. In that modification, additional

background characteristics (Orr, 1999).

site-RA regime covariates are included, and q represents being

offered one of two or three interventions rather than a single

Consider two interventions q and r (for example, PBTH versus

intervention.

SUB), where the second option ( r) is treated as the base case.

Then, the impact on an outcome Y (for example, at least 1 night

Standard Errors

homeless or doubled up during past 6 months, working for

The model described previously was estimated using weighted

pay in week before survey, or adult psychological distress) of

least squares, or WLS, and heteroskedasticity-consistent

intervention q relative to intervention r is estimated through

standard errors, also known as robust standard errors (that

Equation 1 for those families who had both options q and r as

is, Huber-Eicker-White robust standard errors; see Greene,

possible assignments, and were assigned to one of them. The

2003; Huber, 1967; and White 1980, 1984). Heteroskedastic

estimation equation was—

residuals would arise if some types of families have higher

(1)

,

variability in their outcomes (even conditional on covariates)

than other families or if the different interventions themselves

where

influence this variability. Furthermore, this study uses the

U

linear probability model for binary outcomes, rather than a

= outcome Y for family i,

i

logit or probit model, because of the ease of interpretation of

T = indicator variable that equals 1 if family i was assigned to

least squares parameter estimates. The linear probability model,

q, i

intervention q,

however, induces heteroskedasticity (Angrist and Pischke,

d

2008). To address this potential heteroskedasticity, robust

= average impact of being assigned to intervention q

q, r

standard errors were estimated and used in tests of statistical

relative to being assigned to intervention r,

significance. These standard errors are appropriate for making

2 These background characteristics are listed in Appendix C.

3 Of the 12 sites, 10 had a single random assignment regime during the 15-month study enrollment period. The remaining 2 sites changed random assignment probabilities a single time each, creating 14 site-RA regime groups. The equation includes 13 indicator variables and omits 1. These indicator variables are included so that the impact estimate is based on within-site comparisons.

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

inferences about intervention effects for the sites in this study.

Impact Estimation Model for Moderator

The standard errors do not take into account variability in

Analysis

site-level effects, however, and so are not appropriate for

The moderator analysis presented in Chapter 8 presents evi-

generalizing results to other sites.

dence on whether the study interventions are more effective for

Adult Survey Nonresponse Weights

families with different levels of psychosocial needs or housing

The adult survey achieved a 78-percent response rate at the

barriers. The estimation model for the moderator analysis is—

37-month followup. Nonresponse raises two concerns. First,

(2)

,

nonresponse to a followup survey used to measure outcomes

presents a challenge to the internal validity of the study if the

where all terms appearing in Equation 1 have the same defini-

intervention groups (that is, PBTH, SUB, CBRR, and UC) have

tion,

different patterns of nonresponse.

M = potential moderator index variable (either psychosocial

i

Second, followup survey nonresponse can threaten the

challenges or housing barriers) for family i,

generalizability of results to the entire enrolled sample if survey

p = change in impact of being assigned to intervention q

nonrespondents differ from respondents, even if they do so

q, r

relative to being assigned to intervention r associated with a

symmetrically across randomization arms. To address both of

one-unit change in M index, and

these issues, the analysis team prepared a set of weights based

on family characteristics measured in the baseline survey that

g = other regression coefficient.

q, r

attempt to adjust for adult survey nonresponse for each pair-

The potential moderator index variable, M, is entered in the

wise comparison.4 The weights were used in estimating impacts

model both alone and interacted with treatment, T.

on all family and adult outcomes.

The test of statistical significance of the p coefficient serves

q, r

Impact Estimation Model for Child

as the test for whether impacts differ significantly according to

Well-Being Outcomes

the M index. Standard errors and weights for family, adult, and

child outcomes are the same as in the main impact estimation.

The estimation model for impacts on child well-being outcomes

differs from the model described previously in two respects.

First, the standard errors are modified to accommodate the fact

C.2. Covariates

that some child well-being impact regressions include two chil-

dren from the same family. To allow for correlation between

Covariates in the impact models improved the precision of

impacts on children in the same family, the model estimates

the estimates. Because individuals were randomly assigned to

the robust standard errors clustered within family. Second, to

control and treatment groups, the addition of these covariates

address the process by which individual child observations

does not affect the expected value of the estimate. All covariates

came to be included in impact regressions, the weighting

had to be characteristics that were known (or determined)

strategy includes more steps. The child weights are the product

before randomization. In selecting covariates, the study team

of three components.

considered (1) the importance of the variable in predicting the

outcomes of interest, (2) the extent of variation on the variable

1. The adult survey nonresponse weight.

for the sample, and (3) the completeness of the data.

2. The inverse probability of being selected as a focal child.

A full set of covariates measured in the baseline survey was

included in the impact models for housing stability, adult

3. A child nonresponse (to child assessment or child survey)

well-being, and self-sufficiency outcomes. Because of smaller

weight (conditional on the parent being an adult survey

sample sizes, more-limited sets of covariates were included in

respondent).

the impact models for family preservation and child outcomes.

The aim of the analysis is to represent equally all children in

The superscript “a” indicates those covariates included in the

all study families. Therefore, the focal children from families

impact model for family preservation outcomes. The superscript

with more children receive more weight in the analysis of child

“b” indicates those covariates included in the impact model for

well-being than the focal children from families with fewer

child outcomes.

children.

4 The construction of weights to address survey nonresponse is discussed in Little (1986).

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

Continuous Variables

Immunodeficiency Virus) positive; has AIDS (Acquired

•

Immune Deficiency Syndrome); uses drugs intravenously;

Age of family head at baseline (linear), age squared (quadratic).

has other medical condition).

• Number of children with family in shelter.

• Severe psychological distress at baseline.

• Annualized current earnings.

• Post-traumatic stress disorder (PTSD) symptoms at baseline.

• Family income (linear categories: ≥ $0 to < $5,000; $5,000

• A child family member has a disability or an adult family

to < $10,000; $10,000 to < $15,000; $15,000 to < $20,000;

member has a disability that limits or prevents work for pay.

$20,000 to < $25,000; ≥ $25,000; and income categories

squared).

• Family head has a disability that limits or prevents working

•

for pay.

Total years stayed with family or friends because of econom-

ic necessity in past 5 years as an adult.

• Substance abuse problems (drug or alcohol).

• Total years previously homeless in your life before entering

• Highest level of education (categories: less than a high

the current shelter.

school diploma; high school diploma; more than a high

school diploma or general educational development).

Binary Variables

• Working for pay at baseline.

• Race/ethnicity (categories: White; Black or African-American;

Asian, Native Hawaiian, or other Pacific Islander; Hispanic

• Ever worked for pay.

or Latino; other).

• Unemployment (categories: no work in the past 6 months;

• Gender.

no work in the past 24 months).

• Marital status (categories: divorced; married; single/never

• Receipt of various types of public assistance at baseline

married; widowed).

(categories: any health insurance—Medicaid, state health

insurance, State Children’s Health Insurance Program;

• Children of a certain age group (categories: family has a

Supplemental Nutrition Assistance Program, or SNAP; Sup-

child younger than age 1; family has a child between age

plemental Security Income (SSI) or Social Security Disability

1 and 5 years; family has a child between age 13 and 17

Income (SSDI); Temporary Assistance for Needy Families,

years).

or TANF; unemployment insurance; Special Supplemental

• Children not with family in shelter at baseline (categories:

Nutrition Program for Women, Infants, and Children).

any child; two or more children).

• Family income is under $5,000.

• Number of children with family in shelter (categories: one

• Owned or rented own house or apartment before entering

child; two children; three children; four or more children).

shelter.

• Have a spouse or partner that is with the family in shelter at

• Number of months since family had a regular place to stay

baseline.

and months squared.

• Have a spouse or partner that is not with the family in

• Previously stayed with family or friends because of economic

shelter at baseline.

necessity.

• Pregnant at baseline.

• Previously experienced homelessness.

• Any health problems (has self-reported poor health; has

• Past evictions, lease violations, or problems with a landlord.

diabetes; has anemia; has high blood pressure; has heart

disease; had a stroke; has hepatitis/liver problems; has

• Ever convicted of a felony.

arthritis, rheumatism, joint problems; has chest infection,

• Ever been in foster care as a child (foster home, group home,

cold, cough, bronchitis; has pneumonia; has tuberculosis;

or any other kind of institution).

has cancer; has problems walking, a lost limb, or other

mobility impairment; has gonorrhea, syphilis, herpes, chla-

• Ever homeless as a child.

mydia, other sexually transmitted diseases; is HIV (Human

• Ever experienced intimate partner violence in adulthood.

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

• Ineligible families (categories: 1 to 7, indicating the family

C.3. Missing Data and Imputations

was not eligible for an available treatment group).

Although respondents were asked to complete all questions

• Site location × random assignment regime interaction terms

from the baseline survey, some data in the Family Options

(categories: Alameda County; Atlanta; Baltimore; Boston;

Study remained missing. Overall, most covariates used in

Connecticut; Denver; Honolulu; Kansas City; Louisville-1;

the imputation models had no missing data. Only eight of

Louisville-2; Minneapolis; Phoenix; Salt Lake City-1; Salt

the covariates had more than 1 percent missing data and

Lake City-2).

no covariates had more than 5 percent missing. To account

for missing data on covariates, the study team used a single

Binary Variables for Child Outcomes Only

stochastic imputation using SAS’s PROC MI to impute missing

• Focal child age.

covariate values.5 This method assigns values to missing

variables using a regression model that predicts the value of the

• Focal child gender.

missing variable based on other characteristics of the sample

member and the responses of other study participants who are

Additional Covariates for Pooled

similar. The characteristics used in the imputation include all

Comparisons and Selected Outcomes

covariates used in the impact model.

In addition to including these sets of covariates, the impact

models for the pooled comparisons (SUB + CBRR + PBTH ver-

C.4. Family/Adult Weights

sus UC, SUB + CBRR versus PBTH, SUB + PBTH versus CBRR,

and CBRR + PBTH versus SUB) included interaction terms

The study achieved a 78.2-percent response rate for the

between site/random assignment regime and randomization

37-month followup survey. Nonresponse raises two concerns.

set (to correctly control for differing random assignment ratios

First, nonresponse to a followup survey used to measure out-

across sites and assignment groups).

comes presents a challenge to the internal validity of the study

if the intervention groups (that is, SUB, CBRR, PBTH, and

Finally, a few outcomes included one or two additional

UC) have different patterns of nonresponse. Second, followup

covariates to control for closely related baseline variables (when

survey nonresponse can threaten the generalizability of results

these baseline variables were not already included in the main

to the entire enrolled sample if survey nonrespondents differ

covariate set).

from respondents, even if they do so symmetrically across ran-

• Outcome: anyone in family had earnings in past month;

domization arms. Appendix D provides analysis of nonresponse

extra covariate: anyone in family had earnings at baseline.

to the 37-month followup survey.

• Outcomes: anyone in family received SSI in past month,

To address both of these issues, the study team prepared 10

anyone in family received SSDI in past month; extra

sets of weights that adjusted for adult survey nonresponse to

covariate: anyone in family received SSI at baseline.

the 37-month survey—1 set for each pairwise and pooled com-

parison.6 The weights were used in the impact regressions for

• Outcome: adult health in past 30 days was poor or fair;

the outcomes in this report that are defined at the family level

extra covariates: adult health in past 30 days was poor, adult

and at the adult respondent level. These weights were con-

health in past 30 days was fair.

structed by, (1) for each intervention group within a pairwise

• Outcomes: alcohol dependence or drug abuse, alcohol

comparison (or each side of a pooled comparison), separately

dependence, drug abuse; extra covariates: drug abuse at

regressing a dummy variable for survey response on the same

baseline, behavioral health problem at baseline.

baseline characteristics included in the impact model and using

the results to generate a propensity to respond for each family;7

• Outcomes: goal-oriented thinking, psychological distress,

(2) for each intervention group within a pairwise comparison

PTSD symptoms in past 30 days; extra covariate: behavioral

(or each side of a pooled comparison), dividing each group into

health problem at baseline.

5 Single stochastic imputation adds a random error term to every imputed value so that the data do not have artificially low variability. This varying component is randomly drawn from a distribution with the same variance as the observed values.

6 The construction of weights to address survey nonresponse is discussed in Little (1986).

7 The purpose of the nonresponse regressions was purely predictive, rather than inferential, which implied that the number of covariates in the model was not of concern (as it was in the impact regressions). Thus, rather than using single stochastic imputation to address missing covariate values for the nonresponse regressions, all missing values were imputed as the value “0”. Then, in addition to the impact model baseline covariates, the regression models also included dummy variables that indicated when values for covariates were missing.

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

quintiles based on its modeled propensity; (3) within each in-

ChildSelectionWeight = the inverse probability of being selected

ij

tervention group-quintile, the total number of sample families

as a focal child for child j in family i. (The focal child selection

in the quintile divided by the number of respondent families

process is described in Appendix A, Section A.6.) 8

in the quintile calculated the weights for respondents. This last

The weights for other types of outcomes were calculated as—

step raises the representation of respondent families to the level

of the full sample in the weighted data, thereby restoring the

CW[data source] = CWPR × ChildNonResponseWeight[data

ij

ij

composition of the analysis data to that of the full sample on

source] ,

ij

the factors used to estimate propensities to respond.

where—

C.5. Child Weights

CW[data source] = the child weight for [data source] (either

ij

ASQ-3, WJ III, HTKS, or child survey) for child j in family i.

The study team prepared 50 sets of weights to be used for

estimating impacts on child outcomes in the 37-month data—

ChildNonResponseWeight[data source] = the child nonresponse

ij

5 sets for each of the 10 pairwise and pooled comparisons.

weight for [data source] for child j in family i.

The 5 sets of weights correspond to the 5 types of data used to

The child nonresponse weights were calculated in a three-step

construct child outcomes.

process: (1) for each intervention group within a pairwise

1. Parent-report survey data (from the 37-month adult survey).

comparison (or each side of a pooled comparison), separately

regressing a dummy variable for unit response to the question-

2. Ages and Stages Questionnaire (ASQ-3) data.

naire, assessment, or survey on a limited set of predictors9 and

using the results to generate a propensity to respond for each

3. Woodcock-Johnson III (WJ III) assessment data.

child to the particular instrument; (2) for each intervention

4. Head Toes Knees Shoulders (HTKS) assessment data.

group within a pairwise comparison (or each side of a pooled

comparison), dividing the group into quintiles based on its mod-

5. Child survey data.

eled propensity; (3) within each intervention group-quintile,

The weights for the parent-reported outcomes were calculated

calculating the nonresponse weight for the respondents the

as—

weighted total number of focal children in the quintile divided

by the weighted number of respondent children in the quintile,

CWPR = FamilyNonResponseWeight × ChildSelectionWeight ,

ij

i

ij

where the weights were the child selection weights (inverse

where—

probability of focal child selection). The construction of the

child weights from family nonresponse weights, focal child

CWPR = the child weight for parent-reported outcomes for

ij

selection weights, and child nonresponse weights implies that,

child j in family i.

for all child outcomes, the respondent samples are weighted

FamilyNonResponseWeight = the family/adult nonresponse

to represent all the appropriately aged children in all study

i

weight for family i (described in Section C.3).

families.10

8 Section A.6 notes that after two focal children were selected for a family, the focal child screening ceased. Therefore, collection of information for screening criteria other than date of birth was not performed for every child in the study families. For “nonscreened” children, the study team used other information collected in the survey about whether each child was currently living with the family to determine ex-post eligibility for selection (to calculate selection probabilities for selected children). It was assumed that children currently living with the family would be eligible for focal child selection (if age was in targeted range), and it was assumed that children not currently living with the family would be ineligible (regardless of age). The assumption of ineligibility for unscreened children not currently living with the family was based on the fact that most screened children who were not currently living with the family did not meet the extra criteria necessary for eligibility: for 89 percent of these children, the parent either did not regularly spend time with the child or was not at least somewhat familiar with the child’s activities.

9 The relatively small sample sizes for each collection instrument necessitated a smaller set of predictor variables than that used to create family/adult nonresponse weights.

The predictors included: child’s age, child’s gender, parent respondent’s age, parent respondent’s gender, parent’s race/ethnicity (categories: White; Black or African-American; Asian, Native Hawaiian or other Pacific Islander; Hispanic or Latino; other), children not with family in shelter at baseline (categories: any child; two or more children), children of a certain age group (categories: family has a child younger than 1 year, a child ages 1 to 5 years, a child ages 13 to 17 years), parent’s substance abuse problems (drug or alcohol), parent ever convicted of a felony, family income category, family income under $5,000, number of children with the family at baseline, whether the adult respondent has a spouse or partner at baseline (either in shelter or not in shelter), parent had previously experienced homelessness, parent working for pay at baseline, and site location × random assignment regime interaction terms.

10 An implicit assumption in this weighting method is that, within an adult survey response propensity quintile, the distribution of numbers and ages of children in the families who did not respond at all to the 37-month adult survey is the same as that of the families who did respond to the 37-month adult survey.

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

C.6. Partial Paired t-Test for

C.7. Multiple Comparisons

Differences in Means Between

Adjustment for Confirmatory

Followup Waves

Hypothesis Tests

Chapter 2 presents statistical tests for a few outcomes for

Statement of the Problem

differences between the 20-month mean UC group value and

the 37-month mean UC group value. Because the respondent

Simply stated, the multiple comparisons problem is that, as the

samples at the two followup waves are not identical, the appro-

number of hypothesis tests conducted grows, the likelihood

priate statistical test is the partial paired t-test, where “partial”

of finding a statistically significant impact somewhere among

refers to the partially overlapping samples for the two means.11

the tested outcomes simply by chance increases far above the

The variance of the difference between the two sample means

desired risk level for producing false positive results. This

is—

multiple comparisons problem is particularly salient for the

Family Options Study because the number of hypothesis tests

Let x 1 denote the estimated mean from the first sample of size

n

performed is extremely large (a total of 840 tests).

1 . Let x 2 denote the proportion from the second sample of

size n

Because the study design is based on four intervention groups,

2 . We are interested in testing the difference between the

two sample means. We can write the estimated variance of the

the study examines impacts in six pairwise comparisons and

difference between the two sample mean as

four pooled comparisons. For each of these comparisons,

the study looks at five outcome domains (housing stability,

.

self-sufficiency, adult well-being, child well-being, and family

Under simple random sampling, the variance of the difference

preservation), with each domain containing several outcome

can be written as

variables.

The multiple arms, multiple domains, and multiple outcomes

cumulatively generate an extremely large number of hypothesis

.

tests in the main impact analysis (10 comparisons × 84 out-

v( x

comes in the 5 outcome domains = 840 tests).

1 ) is the estimated variance of the first mean based on a

sample of n 1 units, v( x 2 ) is the estimated variance of the

Given this large number of tests, the probability of finding

second proportion based on n 2 units, and m is the number of

an impact, even in the case of no true impacts, is quite large,

families who are in the analysis samples at both followup waves

well above the nominal 10-percent level. In particular, the

(that is, the “overlap”). The correlation (r x ) is estimated

1x2

probability of finding at least one significant impact at the .10

based on the overlap.

level in k independent tests when all true impacts are 0 is given

The square root of the variance gives the standard error of the

by Equation 3.

difference in the two means, which can be used in a statistical

(3) Prob(min p ≤ .10 | all true impacts = 0) = 1– 0.90 k.

test recognizing that we have overlapping samples and they are

not independent.

Thus, if 10 independent tests are performed, then the probability

of finding at least one significant impact at the .10 level—often

.

taken as the litmus test for a “successful” intervention—when

Test Statistic is—

all true impacts are equal to 0 is 1–0.9010=0.65; that is, about

two-thirds of the time one would conclude an unsuccessful

intervention is successful. When 20 independent tests are per-

.

formed, the probability is 0.88; that is, nearly 9 times out of 10.

In fact, with hundreds of tests, it is nearly certain to spuriously

The p-value of the observed value of t is calculated from the

detect a “successful” intervention, even if the intervention was

t-distribution with n–2 degrees of freedom.

not truly “successful” for any outcome.12

This probability of finding at least one significant impact (or

more generally, rejecting at least one null hypothesis) when all

11 This test is described in Kish (1965).

12 Although the study team does not expect the hundreds of hypothesis tests performed in this report to be independent, the likelihood of at least one spurious finding of statistical significance will still be extremely high.

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

7

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

true impacts equal 0 (or more generally, when all null hypoth-

• The six pairwise policy comparisons and one pooled

eses are true) in a “family” of k tests is called the familywise

comparison (PBTH + SUB + CBRR versus UC).

error rate (FWER). In general, the FWER decreases as the k test

statistics used become more correlated (that is, the outcome

• A single composite outcome indicating a stay in emergency

measures tested become more closely related), leading to

shelter or a place not meant for human habitation or an

somewhat less risk of false positive conclusions than indicated

experience of doubling up. This outcome was constructed

in the previous numerical estimates. Many multiple comparison

from two binary outcomes within the housing stability

adjustment procedures have been devised to keep the FWER

domain.

at or below the desired level (such as 0.05 or 0.10), some of

1. At least 1 night spent in emergency shelter or a place not

which take account of correlation among outcomes.

meant for human habitation or doubled up during the

past 6 months at the time of the followup survey (from

Study Response to the Problem

the adult survey).

The study team took two steps to address the multiple compar-

2. Any stay in emergency shelter in the past 12 months at

isons problem.

the time of the followup survey (from Program Usage

Adjust the standard of evidence used to declare a subset of

Data, largely based on Homeless Management Informa-

individual impact estimates statistically significant. The study

tion System, or HMIS, records).

team divided the hypothesis tests into a small set of 7 “confirma-

The six pairwise comparisons were included to assess the

tory” tests and a much larger set of 833 “exploratory” tests. The

relative effectiveness of the interventions in contributing to

team then used a multiple comparisons procedure to adjust the

housing stability (thereby addressing the study’s first research

results of the 7 confirmatory tests to maintain the integrity of the

question stated in Section 1.4). The study team also included

statistical inferences made at the confirmatory level.

the pooled comparison of PBTH + SUB + CBRR versus UC

1. Prespecify impacts to present in the executive summary.

because it provided evidence on whether a housing subsidy of

The study team prespecified the impacts on 18 key out-

any type improved housing stability. Using two sources of data

comes in the 6 pairwise comparisons (for 108 total impact

to construct this outcome enabled the study team to measure

estimates) to present in the executive summary before seeing

housing stability as robustly as possible and made use of all

the results. This step was taken to prevent the selective

available data on return to homelessness.

presentation of statistically significant results in the executive

summary.

Implementing the Multiple Comparisons

Procedure

The first step hinges on the definition and implications of

confirmatory hypothesis tests. Following Schochet (2009), the team

The p-values on the seven impact coefficients were adjusted to

defined confirmatory hypothesis tests as those tests that “assess how

account for the presence of seven confirmatory tests. The team

strongly the study’s prespecified central hypotheses are support-

chose the Westfall-Young resampling method as the procedure

ed by the data,” (Schochet, 2009: 549). Statistically significant

to control the FWER at a .10 level for the seven tests.13 This

findings from confirmatory hypothesis tests are considered

procedure was chosen for the additional statistical power (rela-

definitive evidence of a nonzero intervention impact, effectively

tive to Bonferroni-type methods) it was expected to provide in

ending debate on whether the intervention achieved an impact

tests of a binary outcome variable.

in the study sites. All other hypothesis test results are deemed

The Westfall-Young resampling method involves reassigning

exploratory. For these tests, statistically significant impacts consti-

each study family to the interventions in its randomization

tute suggestive evidence of possible intervention effects.

set (using the original assignment probabilities in effect for

Before beginning analysis, HUD determined that the housing

the family at random assignment) many times to form many

stability domain is the most important outcome domain for the

sample replicates. For each replicate, the seven impacts on the

study. Therefore, the study team designated seven hypothesis

confirmatory outcome were recalculated, as follows.

tests related to housing stability as confirmatory. These

In notation, let—

hypothesis tests were conducted for—

A, B, C, D, E, F, G = seven impact estimates on the confirmatory outcome.

13 Westfall-Young methods are described in Westfall, Tobias, and Wolfinger (2011).

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

8

FAMILY OPTIONS STUDY

Appendix C. Analysis Methods

p-values from t-tests

=

of impact estimates. These values are the “raw,” unadjusted

raw p-values in order from smallest to largest.

p-values for each impact estimate.

Then, some large number R (the study used 20,000) permu-

The impact estimates were then placed in the order of their raw

tation replicates were formed. Within each replicate sample,

p-values.

study families were reassigned to the interventions in their

randomization sets using the original probabilities. For each

IMPACT1, IMPACT2, IMPACT3, IMPACT4, IMPACT5, IMPACT6, replicate, the seven impacts were estimated, producing seven

IMPACT7 = the impact estimates in order of their raw p-values.

p-values.

IMPACT1 is the impact estimate with the smallest raw p-value

and IMPACT7 is the impact estimate with the largest raw p-value.

Next, the adjusted p-values were calculated as follows—

where

is the p-value for an impact estimate in a

particular replicate.

Exhibit C-1 shows the unadjusted and adjusted p-values for the

study’s seven confirmatory hypothesis tests.

Exhibit C-1. Confirmatory Hypothesis Tests in 37-Month Analysis

ITT Impact on “at Least 1 Night Homelessa or Doubled

Pairwise or Pooled

p-Value

Up (past 6 months) or in Shelter (past 12 months)” (%)

p-Value

Comparison

(unadjusted)

(adjusted for multiple

comparisons)

Impact

(SE)

SUB vs. UC

– 21.1

(3.0)

< 0.0001

< 0.0001

CBRR vs. UC

1.9

(3.6)

0.6057

0.9416

PBTH vs. UC

0.3

(4.7)

0.9480

0.9480

SUB vs. CBRR

– 20.4

(3.6)

< 0.0001

< 0.0001

SUB vs. PBTH

– 24.4

(4.6)

< 0.0001

< 0.0001

CBRR vs. PBTH

– 3.1

(6.2)

0.6127

0.9416

SUB + CBRR + PBTH vs. UC

– 8.6

(2.6)

0.0008

0.0041

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SE = standard error.

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

Notes: Impact estimates are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definition.

Sources: Family Options Study 37-month followup survey; Program Usage Data

C

3-Year Impacts of Housing and Services Interventions for Homeless Families

9

FAMILY OPTIONS STUDY

APPENDIX D.

ANALYSIS OF 37-MONTH SURVEY

NONRESPONSE

D.1. Introduction

Respondents Versus Nonrespondents

The impact estimates in this report are based on outcome meas-

Survey nonresponse may also be related to participant

ures derived largely from the 37-month adult followup survey.

characteristics such that families who respond to the survey

This appendix analyzes the extent to which survey nonresponse

are not comparable with families who do not. If this difference

influenced these estimates. Not all study families completed the

was the case, and if the study findings differ on the same

followup survey, which successfully gathered information for

characteristics that relate to survey respondents, then the study

1,784 of the 2,282 families who enrolled in the study. This ap-

findings may not be applicable to the entire sample including

pendix addresses whether, in light of this nonresponse, impact

nonrespondents. To assess the extent to which findings are

estimates are (1) internally valid in the sense that the families

likely applicable to nonrespondents in addition to respondents,

in the sides of each impact comparison remain comparable and

we address the two following questions.

(2) likely valid for the entire study sample after weighting to

1. Do respondents and nonrespondents have systematic

account for nonresponse.1

differences in observable baseline characteristics?

2. How were the main results of this report affected by the use

Balance in Impact Comparison Groups

of nonresponse analysis weights?

After Nonresponse

The Family Options Study randomly assigned families to study

Overview of Findings

interventions so that differences in outcomes among families

In general, the analysis presented in this appendix (1) indicates

who received different interventions would be attributable

that the impact results in the Family Options Study remain

to assignment to the intervention. The Interim Report: Family

internally valid after survey nonresponse and (2) provides

Options Study presented evidence confirming that random

nondefinitive evidence that the impact results may be applica-

assignment successfully produced equivalent samples when

ble to the entire study sample. We find that response rates do

comparing the treatment groups within each of the six pairwise

vary based on the treatment to which families were assigned.

impact comparisons in the study. This equivalence testing was

Response rates were slightly lower for families assigned to

conducted on all families participating in the study, however.

usual care (UC) compared with those for the three active

It is possible that whether a family responded to the followup

interventions. Response rates for the UC group ranged from

survey was influenced by the treatment to which they were

73.1 to 76.4 percent, depending on the impact comparison.

assigned in ways that could disrupt this balance. This possi-

The permanent housing subsidy (SUB) group had the highest

bility, in turn, is indicative of whether families in each side of

response rate: 83.6 percent (for all SUB families. These differ-

the impact comparisons are comparable—sometimes referred

ences indicate the importance of our second analysis assessing

to as the study’s “internal validity.” We assess the extent to

internal validity—a comparison of baseline characteristics

which nonresponse affected internal validity by addressing the

for each side of each impact comparison. Here we find that,

following two questions.

although nonresponse patterns somewhat degraded the base-

1. What were the response rates for the Family Options Study

line equivalence samples as reported in the Interim Report for

37-month followup survey, and how did they vary between

comparisons involving the priority access to community-based

assignment groups in pairwise comparison samples?

rapid re-housing (CBRR) group, omnibus test results including

all our comparison characteristics suggest no systematic

2. Did the analysis sample remain balanced for each impact

differences between sides of the impact comparisons, with the

comparison after nonresponse?

exception of the CBRR-versus-UC comparison.

1 This appendix parallels Appendix D from Family Options Study: Short-Term Impacts of Housing and Services Interventions for Homeless Families. The tabulations here are based on response to the 37-month followup survey whereas the appendix to the Short-Term Impacts report was based on response to the 20-month followup survey.

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Turning to our analysis relevant to the applicability of study

Equivalence at Baseline of Analysis Sample

findings to the entire baseline sample, we find some evidence

by Impact Comparison

that baseline characteristics do predict survey response,

Did the analysis sample remain balanced for each impact

which suggests that respondents and nonrespondents may be

comparison after nonresponse? The second step in the analysis

systematically different. This finding in part motivates the use

of the comparability of both sides of each impact comparison is

of survey nonresponse weights, as described in Appendix C,

a comparison of baseline characteristics. If the balance in observ-

Section C.3. In this appendix, we present estimates calculated

able characteristics between groups at baseline remained after

without the nonresponse weights for the study’s headline

nonresponse, survey nonresponse was not related to observable

outcomes. Substantive differences between impact estimates

characteristics and therefore was unlikely to be related to unob-

calculated with and without the nonresponse weights would

servable characteristics. In that case, impact estimates remained a

indicate that impacts for nonrespondents (which cannot be

valid comparison of the effect of receiving different interventions

estimated) may differ from those estimated in the study for

on the particular outcome for the survey respondent population.

respondents. The estimates did not vary substantially from the

weighted estimates. Although not definitive, this finding serves

Exhibit D-2 lists the baseline characteristics that are compared

as evidence that the impact results may be applicable to the

within each impact comparison. These characteristics were

entire study sample.

the same baseline characteristics used to demonstrate baseline

equivalence in the Interim Report, and were chosen because they

D.2. Balance in Impact Comparison

were either major demographic characteristics or they were

baseline measures in the study’s five outcome domains.

Groups After Nonresponse

This section reports results from statistical tests performed to

This section presents two analyses that address the threat to

determine both if groups being compared differed on the each

the internal validity of the study’s impact findings of survey

of the baseline characteristics described previously and if the

nonresponse. To assess the extent to which the groups in each

combined set of characteristics suggested the groups differed

impact comparison remain comparable after nonresponse,

(an omnibus F-test). As a review of the baseline equivalence

this section first reports and compares response rates for each

findings of the full baseline sample, the Interim Report reported

treatment group of each impact comparison. Next, the section

presents an analysis of the balance on baseline characteristics

Exhibit D-1. Survey Nonresponse Incidence by Impact

for each impact comparison within the analysis sample of

Comparison—37-Month Adult Survey

37-month survey respondents.

Baseline

Adult

Response

Families

Surveys

Completed

Rate (%)

Chi sq

Survey Nonresponse

Pairwise comparisons

2,282

1,784

78.2

SUB versus

599

501

83.6

***

What were response rates for the Family Options Study

UC

540

395

73.1

37-month followup survey, and how did the rates vary between

CBRR versus

569

434

76.3

pairwise comparison samples? Exhibit D-1 reports the number

UC

575

434

75.5

of respondents to the followup survey by impact comparison.

PBTH versus

368

293

79.6

UC

339

259

76.4

Results based on raw response rates for each impact compar-

SUB versus

435

362

83.2

***

ison suggest that, relative to assignment to UC, assignment to

CBRR

382

290

75.9

the CBRR, project-based transitional housing (PBTH), or SUB

SUB versus

256

215

84.0

group increased the propensity to respond to the followup sur-

PBTH

240

201

83.8

CBRR versus

232

180

77.6

vey. Participants assigned to SUB were most likely to respond

PBTH

239

184

77.0

to the followup survey, with an overall response rate of 83.6

Pooled comparisons

percent. For each impact comparison, the study team tests for

SUB + CBRR + PBTH

1,536

1,228

79.9

***

a statistically significant difference between the two assignment

versus UC

746

556

74.5

SUB + PBTH

674

546

81.0

groups in the response rates. The team found a statistically

versus CBRR

494

382

77.3

significant difference in the response rates of the two groups in

SUB + CBRR

488

395

80.9

two of the six pairwise comparisons (SUB versus UC and SUB

versus PBTH

363

290

79.9

versus CBRR) and in two of the four pooled comparisons. In

CBRR + PBTH

622

491

78.9

**

these cases, it is particularly relevant to test for differences in

versus SUB

551

463

84.03

baseline characteristics across the sides of the comparisons.

CBRR = priority access to community-based rapid re-housing. PBTH = priority access

to project-based transitional housing. SUB = priority access to permanent housing

subsidy. UC = usual care.

Note: Significantly different response rates are indicated for p-value *** < .01 and ** < .05.

Sources: Family Options Baseline Survey; 37-month adult survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-2. Characteristics Examined in Baseline Equiv-

sample of study participants. Specifically, analyzing the baseline

alency Testing

characteristics of 37-month respondents detects additional char-

Baseline Characteristic at the Time of Random Assignment

acteristics with significant differences for a number of pairwise

Age of household head

Previously homeless (before

comparisons. However, the omnibus F-test statistic continues

Gender

current spell)

to be significant in the CBRR-versus-UC comparison only and

Marital status

Previously lived in doubled up housing

Race/ethnicity

Number of barriers in finding

observed characteristics differences remain small in magnitude.

Educational attainment

housinga

Number of adults in family

Household head has a child

For the sample of survey respondents, differences across

Number of children in family

under 18 living elsewhere

groups within the pairwise impact comparison are observed for

Worked for pay in past week

Number of major life challenges

Previously convicted of a felony

facedb

educational attainment in the SUB-versus-UC, CBRR-versus-UC,

Family annual income

CBRR-versus-PBTH, and two of the grouped comparisons. Age

a Barriers to finding housing were reported by adult respondent as “big problems” in

differed across samples in the SUB-versus-CBRR and SUB +

finding housing. The maximum number of barriers was 19. The 19 possible barriers

were (1) not having enough income to pay rent, (2) inability to pay a security deposit

PBTH-versus-CBRR comparisons. An omnibus F-test continued

or first/last month’s rent, (3) lack of transportation to look for housing, (4) poor credit to indicate the responding samples differed on observable base-history, (5) racial discrimination, (6) not being currently employed, (7) no rent history at all, (8) recently moved to community and no local rent history, (9) no reference from past line characteristics for the CBRR-versus-UC comparison, with

landlords, (10) a past eviction, (11) problems with past landlords, (12) past lease vio-

lations, (13) having problems with police, (14) having a criminal record or background,

statistically significant differences for educational attainment,

(15) having a felony drug record, (16) having three or more children in the household,

number of children, and income. Three baseline characteristics

(17) having teenagers in the household, (18) having someone in the household under

21 years old, and (19) having someone in the household who has a disability.

are also individually statistically different across groups for the

b The seven major life challenges measured were (1) psychological distress, (2) post-

CBRR-versus-PBTH comparison, although the omnibus F-test

traumatic stress disorder, (3) felony conviction, (4) experience of domestic violence,

(5) childhood separation (foster care, group home, or institutionalization), (6) self-reported was not statistically significant. Each of the other pairwise and

medical condition, and (7) substance abuse.

pooled comparisons also had either one or two characteristics

Source: Family Options baseline survey—reproduced from Exhibit 4-9 of Interim Report: The Family Options Study

that had a statistically significant difference across groups.

statistically significant differences in the SUB-versus-UC and

Taken together, these comparisons indicate that, although non-

CBRR-versus-UC comparisons in educational attainment and in

response patterns somewhat degraded the baseline equivalence

the PBTH-versus-CBRR group in age of household head. Only

samples as reported in the Interim Report for comparisons involv-

the CBRR-versus-UC comparison had a p-value of the omnibus

ing the CBRR intervention, omnibus tests results including all

F-test that indicated statistically significant differences in the

our comparison characteristics suggests no systematic differences

two groups at the .05 level. This result suggested an “unlucky”

between sides of the impact comparisons, with the exception of

division of families into the CBRR and UC interventions.

the CBRR-versus-UC comparison. Recall that this comparison

Differences in means and percentages for individual variables,

was found to have an “unlucky” draw with statistically signif-

however, were not substantively large.

icant but relatively small-in-magnitude difference in baseline

characteristics for the entire study sample. Exhibits D-4 through

As reported in Exhibit D-3, slightly different results emerge

D-13 report the summary statistics for baseline characteristics

for baseline equivalency testing for the sample of survey

for each side of each comparison—which, together, is the infor-

respondents as compared with the results for the full baseline

mation summarized in Exhibit D-3.

Exhibit D-3. Summary of Equivalence Testing in Impact Comparisons, 37-Month Adult Survey Number of

Pairwise Impact Comparison

Characteristics With

Characteristic(s) With

p-Value of Omnibus

Significant Differences

Significant Difference

F-test

(out of 15; a = 0.10)

SUB versus UC

1

Educational attainment

0.852

CBRR versus UC

3

Educational attainment, number of children, income

0.002

PBTH versus UC

1

Worked for pay

0.283

SUB versus CBRR

2

Average age, child living elsewhere

0.570

SUB versus PBTH

1

Child living elsewhere

0.478

CBRR versus PBTH

3

Educational attainment, number of adults, ever doubled up

0.295

SUB + CBRR + PBTH versus UC

2

Educational attainment

0.235

SUB + PBTH versus CBRR

2

Average age, ever doubled up

0.304

SUB + CBRR versus PBTH

1

Educational attainment

0.135

CBRR + PBTH versus SUB

2

Race/ethnicity, child living elsewhere

0.689

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-4. Equivalence at Baseline of Analysis Sample for SUB Versus UC Impact Comparison, 37-Month Adult Survey Characteristic

SUB

UC

Difference

Significance Level

Stars

Number of families

501

395

Age of household head at RA (percent)

Less than 21 years old

0.08

0.10

– 0.02

0.79

21–24 years

0.22

0.21

0.01

25–29 years

0.23

0.24

– 0.01

30–34 years

0.19

0.16

0.03

35–44 years

0.20

0.21

– 0.01

45 years and older

0.08

0.09

0.00

Mean age (years)

30.47

30.79

– 0.32

0.67

Gender (percent)

Female

0.93

0.92

0.01

0.56

Male

0.07

0.08

– 0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.74

0.70

0.04

0.17

Married or marriage-like situation

0.26

0.30

– 0.04

Race/ethnicity (percent)

Black/African American, not Hispanic

0.37

0.38

– 0.02

0.84

White, not Hispanic

0.21

0.22

– 0.01

Hispanic

0.24

0.23

0.01

Other

0.19

0.17

0.02

Educational attainment (percent)

Less than high school diploma

0.34

0.41

– 0.07

0.08

*

High school diploma/GED

0.39

0.33

0.06

More than high school diploma

0.26

0.25

0.01

Number of adults in family (percent)

1 adult

0.73

0.69

0.04

2 or more adults

0.27

0.31

– 0.04

0.20

Number of children in family (percent)

1 child

0.44

0.43

0.00

0.93

2 children

0.32

0.33

– 0.01

3 children

0.15

0.15

– 0.01

4 children or more

0.09

0.08

0.02

Missing data

0.01

0.01

0.00

Worked for pay past week (percent)

0.13

0.14

– 0.02

0.30

Ever convicted of a felony (percent)

0.12

0.11

0.00

0.82

Family annual income (percent)

Less than $5,000

0.32

0.34

– 0.02

0.96

$5,000–9,999

0.32

0.29

0.03

$10,000–14,999

0.16

0.17

– 0.01

$15,000–19,999

0.07

0.08

0.00

$20,000–24,999

0.04

0.04

0.00

$25,000 or more

0.05

0.05

0.00

Missing data

0.03

0.02

0.01

Ever been homeless before (percent)

0.62

0.66

– 0.04

0.23

Ever been doubled up before (percent)

0.84

0.87

– 0.03

0.26

Major barrier to finding housinga

0.45

0.45

0.00

0.92

Child under 18 living elsewhere (percent)

0.24

0.23

0.01

0.63

Number of major life challengesb

1.60

1.60

0.00

0.75

F-test on all characteristics except site

F value =

0.745

F-test p-value =

0.852

SUB = priority access to permanent housing subsidy. UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-5. Equivalence at Baseline of Analysis Sample for CBRR Versus UC Impact Comparison, 37-Month Adult Survey Characteristic

CBRR

UC

Difference

Significance Level

Stars

Number of families

434

434

Age of household head at RA (percent)

Less than 21 years old

0.09

0.09

0.00

0.13

21–24 years

0.18

0.21

– 0.02

25–29 years

0.24

0.22

0.01

30–34 years

0.18

0.15

0.04

35–44 years

0.24

0.23

0.01

45 years and older

0.06

0.11

– 0.05

Mean age (years)

30.62

31.25

– 0.62

0.77

Gender (percent)

Female

0.92

0.93

– 0.01

0.51

Male

0.08

0.07

0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.75

0.75

0.00

0.87

Married or marriage-like situation

0.25

0.25

0.00

Race/ethnicity (percent)

Black/African American, not Hispanic

0.50

0.43

0.06

0.24

White, not Hispanic

0.18

0.20

– 0.02

Hispanic

0.18

0.22

– 0.04

Other

0.15

0.15

0.00

Educational attainment (percent)

Less than high school diploma

0.30

0.38

– 0.08

0.01

**

High school diploma/GED

0.41

0.32

0.08

More than high school diploma

0.29

0.29

0.00

Number of adults in family (percent)

1 adult

0.72

0.73

– 0.01

2 or more adults

0.28

0.27

0.01

0.76

Number of children in family (percent)

1 child

0.40

0.43

– 0.03

0.05

*

2 children

0.31

0.32

– 0.01

3 children

0.14

0.16

– 0.02

4 children or more

0.15

0.08

0.06

Missing data

0.01

0.01

0.00

Worked for pay past week (percent)

0.18

0.21

– 0.02

0.44

Ever convicted of a felony (percent)

0.10

0.11

– 0.01

0.65

Family annual income (percent)

Less than $5,000

0.29

0.33

– 0.04

0.09

*

$5,000–9,999

0.32

0.25

0.07

$10,000–14,999

0.19

0.17

0.02

$15,000–19,999

0.09

0.09

0.00

$20,000–24,999

0.03

0.06

– 0.03

$25,000 or more

0.06

0.07

– 0.01

Missing data

0.02

0.03

– 0.01

Ever been homeless before (percent)

0.62

0.63

– 0.01

0.78

Ever been doubled up before (percent)

0.88

0.86

0.02

0.36

Major barrier to finding housinga

0.42

0.47

– 0.05

0.15

Child under 18 living elsewhere (percent)

0.21

0.22

– 0.01

0.74

Number of major life challengesb

1.46

1.66

– 0.19

0.44

F-test on all characteristics except site

F value =

1.890

F-test p-value =

0.002

CBRR = priority access to community-based rapid re-housing. UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-6. Equivalence at Baseline of Analysis Sample for PBTH Versus UC Impact Comparison, 37-Month Adult Survey Characteristic

PBTH

UC

Difference

Significance Level

Stars

Number of families

293

259

Age of household head at RA (percent)

Less than 21 years old

0.10

0.06

0.04

0.38

21–24 years

0.15

0.16

– 0.01

25–29 years

0.26

0.22

0.05

30–34 years

0.19

0.21

– 0.02

35–44 years

0.21

0.25

– 0.04

45 years and older

0.09

0.10

– 0.01

Mean age (years)

30.95

32.31

– 1.36

0.58

Gender (percent)

Female

0.90

0.93

– 0.03

0.27

Male

0.10

0.07

0.03

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.68

0.68

0.00

0.94

Married or marriage-like situation

0.32

0.32

0.00

Race/ethnicity (percent)

Black/African American, not Hispanic

0.41

0.42

– 0.01

0.96

White, not Hispanic

0.19

0.18

0.01

Hispanic

0.15

0.14

0.01

Other

0.25

0.26

– 0.01

Educational attainment (percent)

Less than high school diploma

0.37

0.41

– 0.04

0.59

High school diploma/GED

0.36

0.33

0.03

More than high school diploma

0.27

0.26

0.01

Number of adults in family (percent)

1 adult

0.68

0.67

0.01

2 or more adults

0.32

0.33

– 0.01

0.85

Number of children in family (percent)

1 child

0.40

0.42

– 0.02

0.48

2 children

0.29

0.27

0.02

3 children

0.19

0.20

– 0.01

4 children or more

0.12

0.10

0.03

Missing data

0.00

0.01

– 0.01

Worked for pay past week (percent)

0.17

0.23

– 0.06

0.07

*

Ever convicted of a felony (percent)

0.10

0.15

– 0.04

0.11

Family annual income (percent)

Less than $5,000

0.29

0.30

– 0.01

0.61

$5,000–9,999

0.26

0.23

0.03

$10,000–14,999

0.18

0.17

0.01

$15,000–19,999

0.13

0.10

0.03

$20,000–24,999

0.05

0.07

– 0.02

$25,000 or more

0.05

0.07

– 0.01

Missing data

0.03

0.06

– 0.02

Ever been homeless before (percent)

0.61

0.65

– 0.04

0.36

Ever been doubled up before (percent)

0.83

0.84

– 0.02

0.62

Major barrier to finding housinga

0.44

0.45

– 0.01

0.79

Child under 18 living elsewhere (percent)

0.23

0.24

– 0.01

0.70

Number of major life challengesb

1.69

1.64

0.04

0.89

F-test on all characteristics except site

F value =

1.062

F-test p-value =

0.380

PBTH = priority access to project-based transitional housing. UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-7. Equivalence at Baseline of Analysis Sample for SUB Versus CBRR Impact Comparison, 37-Month Adult Survey Characteristic

SUB

CBRR

Difference

Significance Level

Stars

Number of families

362

290

Age of household head at RA (percent)

Less than 21 years old

0.08

0.12

– 0.03

0.46

21–24 years

0.21

0.19

0.02

25–29 years

0.23

0.23

0.00

30–34 years

0.19

0.19

0.01

35–44 years

0.19

0.22

– 0.02

45 years and older

0.09

0.06

0.03

Mean age (years)

30.56

30.13

0.43

0.07

*

Gender (percent)

Female

0.94

0.92

0.02

0.27

Male

0.06

0.08

– 0.02

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.75

0.76

0.00

0.98

Married or marriage-like situation

0.25

0.24

0.00

Race/ethnicity (percent)

Black/African American, not Hispanic

0.39

0.44

– 0.06

0.32

White, not Hispanic

0.22

0.23

– 0.01

Hispanic

0.24

0.20

0.04

Other

0.16

0.13

0.03

Educational attainment (percent)

Less than high school diploma

0.36

0.33

0.03

0.77

High school diploma/GED

0.40

0.42

– 0.02

More than high school diploma

0.25

0.25

0.00

Number of adults in family (percent)

1 adult

0.73

0.71

0.02

2 or more adults

0.27

0.29

– 0.02

0.54

Number of children in family (percent)

1 child

0.44

0.42

0.03

0.51

2 children

0.32

0.32

0.00

3 children

0.13

0.12

0.02

4 children or more

0.10

0.13

– 0.04

Missing data

0.00

0.01

0.00

Worked for pay past week (percent)

0.14

0.16

– 0.02

0.39

Ever convicted of a felony (percent)

0.11

0.11

0.01

0.79

Family annual income (percent)

Less than $5,000

0.33

0.30

0.03

0.61

$5,000–9,999

0.32

0.33

– 0.02

$10,000–14,999

0.17

0.19

– 0.02

$15,000–19,999

0.06

0.08

– 0.02

$20,000–24,999

0.05

0.02

0.02

$25,000 or more

0.06

0.05

0.00

Missing data

0.01

0.01

0.00

Ever been homeless before (percent)

0.63

0.64

– 0.01

0.89

Ever been doubled up before (percent)

0.84

0.88

– 0.04

0.14

Major barrier to finding housinga

0.46

0.41

0.05

0.22

Child under 18 living elsewhere (percent)

0.25

0.19

0.06

0.09

*

Number of major life challengesb

1.61

1.49

0.12

0.65

F-test on all characteristics except site

F value =

0.938

F-test p-value =

0.570

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

GED =general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

7

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-8. Equivalence at Baseline of Analysis Sample for SUB Versus PBTH Impact Comparison, 37-Month Adult Survey Characteristic

SUB

PBTH

Difference

Significance Level

Stars

Number of families

215

201

Age of household head at RA (percent)

Less than 21 years old

0.07

0.09

– 0.02

0.46

21–24 years

0.24

0.17

0.07

25–29 years

0.21

0.26

– 0.05

30–34 years

0.19

0.20

– 0.01

35–44 years

0.19

0.19

0.00

45 years and older

0.10

0.08

0.02

Mean age (years)

30.82

30.58

0.24

0.73

Gender (percent)

Female

0.93

0.90

0.03

0.28

Male

0.07

0.10

– 0.03

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.69

0.68

0.01

0.80

Married or marriage-like situation

0.31

0.32

– 0.01

Race/ethnicity (percent)

Black/African American, not Hispanic

0.35

0.37

– 0.02

0.31

White, not Hispanic

0.20

0.24

– 0.04

Hispanic

0.22

0.15

0.07

Other

0.23

0.24

– 0.01

Educational attainment (percent)

Less than high school diploma

0.29

0.31

– 0.02

0.86

High school diploma/GED

0.43

0.40

0.02

More than high school diploma

0.28

0.28

0.00

Number of adults in family (percent)

1 adult

0.67

0.68

– 0.01

2 or more adults

0.33

0.32

0.01

0.88

Number of children in family (percent)

1 child

0.42

0.40

0.02

0.35

2 children

0.34

0.31

0.03

3 children

0.17

0.19

– 0.02

4 children or more

0.07

0.10

– 0.04

Missing data

0.01

0.00

0.01

Worked for pay past week (percent)

0.18

0.13

0.05

0.15

Ever convicted of a felony (percent)

0.14

0.11

0.03

0.36

Family annual income (percent)

Less than $5,000

0.29

0.30

– 0.01

0.56

$5,000–9,999

0.31

0.28

0.03

$10,000–14,999

0.17

0.18

– 0.01

$15,000–19,999

0.07

0.12

– 0.05

$20,000–24,999

0.06

0.05

0.01

$25,000 or more

0.05

0.04

0.01

Missing data

0.05

0.03

0.02

Ever been homeless before (percent)

0.64

0.60

0.04

0.35

Ever been doubled up before (percent)

0.84

0.82

0.02

0.66

Major barrier to finding housinga

0.45

0.45

0.00

0.98

Child under 18 living elsewhere (percent)

0.28

0.20

0.08

0.07

*

Number of major life challengesb

1.59

1.74

– 0.15

0.70

F-test on all characteristics except site

F value =

0.995

F-test p-value =

0.478

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

8

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-9. Equivalence at Baseline of Analysis Sample for CBRR Versus PBTH Impact Comparison, 37-Month Adult Survey Characteristic

CBRR

PBTH

Difference

Significance Level

Stars

Number of families

180

184

Age of household head at RA (percent)

Less than 21 years old

0.07

0.09

– 0.03

0.60

21–24 years

0.16

0.13

0.03

25–29 years

0.28

0.25

0.03

30–34 years

0.22

0.21

0.00

35–44 years

0.22

0.22

0.00

45 years and older

0.06

0.10

– 0.04

Mean age (years)

30.73

31.51

– 0.77

0.33

Gender (percent)

Female

0.91

0.91

– 0.01

0.80

Male

0.09

0.09

0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.66

0.73

– 0.08

0.11

Married or marriage-like situation

0.34

0.27

0.08

Race/ethnicity (percent)

Black/African American, not Hispanic

0.47

0.43

0.03

0.86

White, not Hispanic

0.16

0.17

– 0.01

Hispanic

0.12

0.14

– 0.02

Other

0.26

0.26

0.01

Educational attainment (percent)

Less than high school diploma

0.29

0.41

– 0.11

0.05

**

High school diploma/GED

0.39

0.29

0.10

More than high school diploma

0.32

0.30

0.01

Number of adults in family (percent)

1 adult

0.63

0.73

– 0.09

2 or more adults

0.37

0.27

0.09

0.05

*

Number of children in family (percent)

1 child

0.40

0.41

– 0.01

0.78

2 children

0.29

0.27

0.02

3 children

0.14

0.17

– 0.03

4 children or more

0.16

0.15

0.01

Missing data

0.01

0.00

0.01

Worked for pay past week (percent)

0.23

0.21

0.02

0.62

Ever convicted of a felony (percent)

0.11

0.10

0.01

0.81

Family annual income (percent)

Less than $5,000

0.27

0.28

– 0.01

0.93

$5,000–9,999

0.29

0.26

0.03

$10,000–14,999

0.20

0.18

0.02

$15,000–19,999

0.11

0.13

– 0.02

$20,000–24,999

0.04

0.05

– 0.02

$25,000 or more

0.07

0.06

0.01

Missing data

0.03

0.04

– 0.01

Ever been homeless before (percent)

0.63

0.61

0.01

0.79

Ever been doubled up before (percent)

0.91

0.83

0.07

0.04

**

Major barrier to finding housinga

0.45

0.43

0.02

0.69

Child under 18 living elsewhere (percent)

0.28

0.22

0.06

0.18

Number of major life challengesb

1.33

1.68

– 0.35

0.25

F-test on all characteristics except site

F value =

1.130

F-test p-value =

0.295

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

9

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-10. Equivalence at Baseline of Analysis Sample for SUB + CBRR + PBTH Versus UC Impact Comparison, 37-Month Adult Survey

Characteristic

SUB, CBRR, PBTH

UC

Difference

Significance Level

Stars

Number of families

1,228

556

Age of household head at RA (percent)

Less than 21 years old

0.09

0.09

0.00

0.52

21–24 years

0.19

0.20

– 0.01

25–29 years

0.24

0.23

0.01

30–34 years

0.19

0.16

0.03

35–44 years

0.22

0.23

– 0.01

45 years and older

0.08

0.10

– 0.02

Mean age (years)

30.64

31.22

– 0.59

0.90

Gender (percent)

Female

0.92

0.93

– 0.01

0.50

Male

0.08

0.07

0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.73

0.73

0.00

0.96

Married or marriage-like situation

0.27

0.27

0.00

Race/ethnicity (percent)

Black/African American, not Hispanic

0.42

0.42

0.00

0.73

White, not Hispanic

0.19

0.19

0.00

Hispanic

0.20

0.21

– 0.02

Other

0.19

0.17

0.02

Educational attainment (percent)

Less than high school diploma

0.34

0.39

– 0.06

0.02

**

High school diploma/GED

0.39

0.33

0.06

More than high school diploma

0.27

0.28

0.00

Number of adults in family (percent)

1 adult

0.71

0.71

0.00

2 or more adults

0.29

0.29

0.00

0.98

Number of children in family (percent)

1 child

0.41

0.44

– 0.03

0.26

2 children

0.31

0.31

– 0.01

3 children

0.15

0.15

0.00

4 children or more

0.12

0.08

0.03

Missing data

0.00

0.01

0.00

Worked for pay past week (percent)

0.16

0.19

– 0.03

0.15

Ever convicted of a felony (percent)

0.11

0.11

– 0.01

0.64

Family annual income (percent)

Less than $5,000

0.30

0.33

– 0.03

0.46

$5,000–9,999

0.30

0.26

0.04

$10,000–14,999

0.18

0.17

0.00

$15,000–19,999

0.09

0.08

0.01

$20,000–24,999

0.04

0.06

– 0.02

$25,000 or more

0.06

0.06

0.00

Missing data

0.03

0.03

– 0.01

Ever been homeless before (percent)

0.62

0.64

– 0.02

0.39

Ever been doubled up before (percent)

0.85

0.85

0.00

0.92

Major barrier to finding housinga

0.44

0.46

– 0.02

0.33

Child under 18 living elsewhere (percent)

0.23

0.22

0.00

0.88

Number of major life challengesb

1.57

1.66

– 0.09

0.72

F-test on all characteristics except site

F value =

1.169

F-test p-value =

0.235

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

10

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-11. Equivalence at Baseline of Analysis Sample for SUB + PBTH Versus CBRR Impact Comparison, 37-Month Adult Survey

Characteristic

SUB, PBTH

CBRR

Difference

Significance Level

Stars

Number of families

546

382

Age of household head at RA (percent)

Less than 21 years old

0.09

0.10

– 0.01

0.49

21–24 years

0.18

0.18

0.00

25–29 years

0.24

0.24

0.00

30–34 years

0.20

0.19

0.01

35–44 years

0.20

0.23

– 0.03

45 years and older

0.09

0.06

0.03

Mean age (years)

30.88

30.44

0.44

0.05

**

Gender (percent)

Female

0.93

0.92

0.01

0.50

Male

0.07

0.08

– 0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.75

0.74

0.01

0.75

Married or marriage-like situation

0.25

0.26

– 0.01

Race/ethnicity (percent)

Black/African American, not Hispanic

0.40

0.47

– 0.06

0.26

White, not Hispanic

0.20

0.19

0.01

Hispanic

0.21

0.18

0.02

Other

0.19

0.16

0.03

Educational attainment (percent)

Less than high school diploma

0.37

0.31

0.06

0.11

High school diploma/GED

0.36

0.41

– 0.05

More than high school diploma

0.27

0.28

– 0.02

Number of adults in family (percent)

1 adult

0.73

0.70

0.04

2 or more adults

0.27

0.30

– 0.04

0.22

Number of children in family (percent)

1 child

0.43

0.40

0.04

0.38

2 children

0.31

0.32

– 0.01

3 children

0.15

0.13

0.01

4 children or more

0.11

0.15

– 0.04

Missing data

0.00

0.01

0.00

Worked for pay past week (percent)

0.16

0.18

– 0.01

0.62

Ever convicted of a felony (percent)

0.11

0.11

0.00

1.00

Family annual income (percent)

Less than $5,000

0.32

0.29

0.02

0.88

$5,000–9,999

0.30

0.32

– 0.02

$10,000–14,999

0.18

0.19

– 0.01

$15,000–19,999

0.08

0.09

0.00

$20,000–24,999

0.05

0.03

0.02

$25,000 or more

0.06

0.06

0.00

Missing data

0.02

0.02

0.00

Ever been homeless before (percent)

0.63

0.63

0.00

0.95

Ever been doubled up before (percent)

0.84

0.88

– 0.04

0.06

*

Major barrier to finding housinga

0.45

0.42

0.03

0.42

Child under 18 living elsewhere (percent)

0.24

0.22

0.02

0.47

Number of major life challengesb

1.63

1.45

0.18

0.40

F-test on all characteristics except site

F value =

1.114

F-test p-value =

0.304

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

11

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-12. Equivalence at Baseline of Analysis Sample for SUB + CBRR Versus PBTH Impact Comparison, 37-Month Adult Survey

Characteristic

SUB, CBRR

PBTH

Difference

Significance Level

Stars

Number of families

395

290

Age of household head at RA (percent)

Less than 21 years old

0.07

0.10

– 0.03

0.51

21–24 years

0.20

0.15

0.05

25–29 years

0.25

0.26

– 0.01

30–34 years

0.20

0.19

0.01

35–44 years

0.21

0.21

– 0.01

45 years and older

0.08

0.09

– 0.01

Mean age (years)

30.78

30.98

– 0.20

0.55

Gender (percent)

Female

0.92

0.90

0.01

0.56

Male

0.08

0.10

– 0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.68

0.69

– 0.01

0.78

Married or marriage-like situation

0.32

0.31

0.01

Race/ethnicity (percent)

Black/African American, not Hispanic

0.40

0.41

0.00

0.90

White, not Hispanic

0.18

0.19

– 0.01

Hispanic

0.17

0.15

0.02

Other

0.25

0.25

0.00

Educational attainment (percent)

Less than high school diploma

0.29

0.37

– 0.08

0.09

*

High school diploma/GED

0.41

0.36

0.05

More than high school diploma

0.30

0.27

0.03

Number of adults in family (percent)

1 adult

0.66

0.68

– 0.02

2 or more adults

0.34

0.32

0.02

0.58

Number of children in family (percent)

1 child

0.41

0.40

0.01

0.41

2 children

0.32

0.29

0.03

3 children

0.16

0.19

– 0.03

4 children or more

0.11

0.12

– 0.01

Missing data

0.01

0.00

0.01

Worked for pay past week (percent)

0.21

0.17

0.03

0.28

Ever convicted of a felony (percent)

0.13

0.10

0.02

0.35

Family annual income (percent)

Less than $5,000

0.28

0.29

– 0.01

0.62

$5,000–9,999

0.30

0.26

0.04

$10,000–14,999

0.18

0.18

0.00

$15,000–19,999

0.08

0.13

– 0.04

$20,000–24,999

0.05

0.05

0.00

$25,000 or more

0.06

0.05

0.01

Missing data

0.04

0.03

0.00

Ever been homeless before (percent)

0.64

0.61

0.02

0.56

Ever been doubled up before (percent)

0.87

0.82

0.04

0.11

Major barrier to finding housinga

0.45

0.44

0.01

0.79

Child under 18 living elsewhere (percent)

0.28

0.23

0.05

0.12

Number of major life challengesb

1.47

1.69

– 0.22

0.38

F-test on all characteristics except site

F value =

1.293

F-test p-value =

0.135

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

12

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-13. Equivalence at Baseline of Analysis Sample for CBRR + PBTH Versus SUB Impact Comparison, 37-Month Adult Survey

Characteristic

CBRR, PBTH

SUB

Difference

Significance Level

Stars

Number of families

491

463

Age of household head at RA (percent)

Less than 21 years old

0.11

0.08

0.03

0.36

21–24 years

0.18

0.22

– 0.05

25–29 years

0.24

0.23

0.02

30–34 years

0.19

0.19

0.00

35–44 years

0.21

0.20

0.01

45 years and older

0.07

0.08

– 0.01

Mean age (years)

30.32

30.39

– 0.07

0.22

Gender (percent)

Female

0.91

0.93

– 0.02

0.16

Male

0.09

0.07

0.02

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.73

0.74

– 0.01

0.69

Married or marriage-like situation

0.27

0.26

0.01

Race/ethnicity (percent)

Black/African American, not Hispanic

0.41

0.37

0.05

0.08

*

White, not Hispanic

0.23

0.21

0.02

Hispanic

0.18

0.24

– 0.06

Other

0.18

0.18

– 0.01

Educational attainment (percent)

Less than high school diploma

0.32

0.35

– 0.02

0.77

High school diploma/GED

0.41

0.40

0.02

More than high school diploma

0.26

0.26

0.01

Number of adults in family (percent)

1 adult

0.70

0.72

– 0.02

2 or more adults

0.30

0.28

0.02

0.43

Number of children in family (percent)

1 child

0.41

0.44

– 0.03

0.71

2 children

0.32

0.32

0.00

3 children

0.15

0.14

0.00

4 children or more

0.12

0.10

0.03

Missing data

0.00

0.01

0.00

Worked for pay past week (percent)

0.15

0.14

0.01

0.64

Ever convicted of a felony (percent)

0.11

0.12

– 0.01

0.52

Family annual income (percent)

Less than $5,000

0.30

0.32

– 0.02

0.57

$5,000–9,999

0.31

0.31

0.00

$10,000–14,999

0.19

0.17

0.01

$15,000–19,999

0.10

0.07

0.03

$20,000–24,999

0.04

0.05

– 0.01

$25,000 or more

0.05

0.06

– 0.01

Missing data

0.02

0.03

– 0.01

Ever been homeless before (percent)

0.62

0.63

– 0.01

0.76

Ever been doubled up before (percent)

0.86

0.84

0.01

0.57

Major barrier to finding housinga

0.43

0.46

– 0.03

0.38

Child under 18 living elsewhere (percent)

0.20

0.25

– 0.05

0.05

**

Number of major life challengesb

1.59

1.61

– 0.02

0.66

F-test on all characteristics except site

F value =

0.854

F-test p-value =

0.700

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

13

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

D.3. Respondents Versus

families assigned to CBRR that responded to the survey from

Nonrespondents

those that did not, with a correspondingly large test statistic for

the omnibus F-test. For families assigned to UC, two baseline

This section provides evidence regarding whether the impacts

characteristics had statistically significant average differences,

estimated on the sample of 37-month survey respondents are

with an 0.08 p-value of the joint omnibus F-test.

applicable to the entire study population, including nonrespon-

Exhibit D-15 summarizes the results of tests from exhibits D-20

dents. We present results of two analyses. The first assesses

to D-29 comparing 37-month respondents and nonrespon-

whether respondents and nonrespondents have systematic

dents. Among the pairwise comparison samples, comparisons

differences in observable baseline characteristics. The second

containing CBRR had two or three baseline characteristic

compares unweighted impact estimates with the weighted

averages that had differences for response as opposed to non-

impact estimates presented in the body of the report.

response that were statistically significant, whereas the other

three pairwise comparison samples had one differing charac-

Do Respondents Differ From Nonrespon-

teristic. Meanwhile, omnibus F-tests indicated that baseline

dents on Baseline Characteristics?

characteristics were generally predictive of nonresponse across

Do respondents and nonrespondents to the 37-month adult sur-

the comparison samples, with all but two meeting a .10 p-value

vey have systematic differences in observable baseline character-

threshold. Taken together, these tests suggest that respondents

istics? Exhibit D-14 summarizes the results of tests from exhibits

seemed to systematically differ from nonrespondents.

D-16 to D-19 comparing baseline characteristics of 37-month

The magnitude of these statistically significant differences is

respondents and nonrespondents for each intervention. For SUB

reported in Exhibits D-16 through D-29. When differences

and PBTH the joint F-test on all characteristics in a regression

were statistically significant relative to nonrespondents, partic-

indicated that the set of baseline characteristics were not jointly

ipants who responded to the followup survey were more often

significant in predicting nonresponse. Consistent with the results

more educated, less likely to have a child living elsewhere, and

discussed previously, six baseline characteristics differed for

more often had been doubled up before.

Exhibit D-14. Summary of Equivalence Testing of 37-Month Respondents Versus Nonrespondents, by Assigned Intervention

RA

Number of Baseline Characteristics With

Characteristic(s) With

p-Value of

Result

Significant Differences Between

Respondents and Nonrespondents (out of 15; a = 0.10)

Significant Difference

Omnibus F-test

SUB

0

None

0.76

CBRR

6

Marital status, race/ethnicity, educational attainment,

0.00

felony conviction, doubled up, major challenges

PBTH

3

Age, gender, prior homeless, child elsewhere

0.43

UC

2

Children, ever homeless

0.08

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

RA = random assignment.

Sources: Family Options Baseline Survey; 37-month followup survey

Exhibit D-15. Summary of Baseline Equivalence Testing of 37-Month Respondents Versus Nonrespondents, by Impact Comparison

Pairwise Impact Comparison sample

Number of characteristics

(response and nonresponse

with significant differences

Characteristic(s) With

p-Value of

compared for both treatment groups)

(out of 15; a = 0.10)

Significant Difference

Omnibus F-test

SUB versus UC

1

Educational attainment

0.47

CBRR versus UC

3

Educational attainment, number of children, annual income

0.04

PBTH versus UC

1

Worked for pay

0.19

SUB versus CBRR

2

Age, child elsewhere

0.00

SUB versus PBTH

1

Child elsewhere

0.31

CBRR versus PBTH

3

Educational attainment, number of adults, doubled up

0.03

SUB + CBRR + PBTH versus UC

1

Educational attainment

0.00

SUB + PBTH versus CBRR

2

Age, doubled up

0.00

SUB + CBRR versus PBTH

1

Educational attainment

0.01

PBTH + CBRR versus SUB

2

Race/ethnicity, child elsewhere

0.00

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

14

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-16. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

501

98

84%

Age of household head at RA (percent)

Less than 21 years old

0.08

0.08

0.00

0.79

21–24 years

0.22

0.17

0.05

25–29 years

0.23

0.26

– 0.03

30–34 years

0.19

0.23

– 0.05

35–44 years

0.20

0.18

0.02

45 years and older

0.08

0.07

0.01

Mean age (years)

30.47

30.39

0.08

0.90

Gender (percent)

Female

0.93

0.90

0.04

0.20

Male

0.07

0.10

– 0.04

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.74

0.70

0.04

0.45

Married or marriage-like situation

0.26

0.30

– 0.04

Race/ethnicity (percent)

Black/African American, not Hispanic

0.37

0.29

0.08

0.14

White, not Hispanic

0.21

0.30

– 0.09

Hispanic

0.24

0.20

0.04

Other

0.19

0.21

– 0.03

Educational attainment (percent)

Less than high school diploma

0.34

0.42

– 0.08

0.36

High school diploma/GED

0.39

0.35

0.05

More than high school diploma

0.26

0.23

0.03

Number of adults in family (percent)

1 adult

0.73

0.68

0.05

2 or more adults

0.27

0.32

– 0.05

0.34

Number of children in family (percent)

1 child

0.44

0.50

– 0.06

0.61

2 children

0.32

0.26

0.06

3 children

0.15

0.16

– 0.02

4 children or more

0.09

0.08

0.01

Missing data

0.01

0.00

0.01

Worked for pay past week (percent)

0.14

0.10

0.04

0.31

Ever convicted of a felony (percent)

0.11

0.12

– 0.01

0.81

Family annual income (percent)

Less than $5,000

0.32

0.34

– 0.02

0.25

$5,000–9,999

0.32

0.29

0.03

$10,000–14,999

0.16

0.14

0.02

$15,000–19,999

0.07

0.04

0.03

$20,000–24,999

0.04

0.10

– 0.06

$25,000 or more

0.05

0.07

– 0.02

Missing data

0.03

0.02

0.01

Ever been homeless before (percent)

0.62

0.62

– 0.01

0.92

Ever been doubled up before (percent)

0.84

0.84

0.01

0.85

Major barrier to finding housinga

0.45

0.44

0.01

0.88

Child under 18 living elsewhere (percent)

0.24

0.32

– 0.07

0.12

Number of major life challengesb

1.60

1.85

– 0.25

0.30

F-test on all characteristics except site

F value =

0.812

F-test p-value = 0.761

SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

15

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-17. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for CBRR, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

434

135

76%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.07

0.02

0.83

21–24 years

0.18

0.21

– 0.03

25–29 years

0.24

0.21

0.02

30–34 years

0.18

0.21

– 0.03

35–44 years

0.24

0.24

0.00

45 years and older

0.06

0.04

0.02

Mean age (years)

30.62

30.36

0.26

0.81

Gender (percent)

Female

0.92

0.88

0.04

0.15

Male

0.08

0.12

– 0.04

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.75

0.68

0.07

0.10

*

Married or marriage-like situation

0.25

0.32

– 0.07

Race/ethnicity (percent)

Black/African American, not Hispanic

0.50

0.37

0.13

0.05

**

White, not Hispanic

0.18

0.23

– 0.05

Hispanic

0.18

0.19

– 0.01

Other

0.15

0.21

– 0.07

Educational attainment (percent)

Less than high school diploma

0.30

0.42

– 0.12

0.03

**

High school diploma/GED

0.41

0.33

0.07

More than high school diploma

0.29

0.24

0.05

Number of adults in family (percent)

1 adult

0.72

0.67

0.05

2 or more adults

0.28

0.33

– 0.05

0.25

Number of children in family (percent)

1 child

0.40

0.50

– 0.10

0.12

2 children

0.31

0.30

0.01

3 children

0.14

0.09

0.05

4 children or more

0.15

0.10

0.05

Missing data

0.01

0.01

– 0.01

Worked for pay past week (percent)

0.18

0.20

– 0.02

0.68

Ever convicted of a felony (percent)

0.10

0.16

– 0.06

0.04

**

Family annual income (percent)

Less than $5,000

0.29

0.31

– 0.02

0.40

$5,000–9,999

0.32

0.31

0.01

$10,000–14,999

0.19

0.13

0.06

$15,000–19,999

0.09

0.08

0.01

$20,000–24,999

0.03

0.07

– 0.04

$25,000 or more

0.06

0.07

– 0.01

Missing data

0.02

0.01

0.00

Ever been homeless before (percent)

0.62

0.64

– 0.01

0.79

Ever been doubled up before (percent)

0.88

0.79

0.09

0.01

**

Major barrier to finding housinga

0.42

0.35

0.08

0.12

Child under 18 living elsewhere (percent)

0.21

0.28

– 0.07

0.11

Number of major life challengesb

1.46

1.85

– 0.39

0.00

***

F-test on all characteristics except site

F value =

1.915

F-test p-value = 0.002

CBRR = priority access to community-based rapid re-housing.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

16

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-18. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for PBTH, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

293

75

80%

Age of household head at RA (percent)

Less than 21 years old

0.10

0.05

0.04

0.19

21–24 years

0.15

0.17

– 0.02

25–29 years

0.26

0.23

0.04

30–34 years

0.19

0.28

– 0.09

35–44 years

0.21

0.24

– 0.03

45 years and older

0.09

0.03

0.07

Mean age (years)

30.95

30.75

0.20

0.75

Gender (percent)

Female

0.90

0.83

0.08

0.06

*

Male

0.10

0.17

– 0.08

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.68

0.63

0.06

0.36

Married or marriage-like situation

0.32

0.37

– 0.06

Race/ethnicity (percent)

Black/African American, not Hispanic

0.41

0.35

0.06

0.62

White, not Hispanic

0.19

0.25

– 0.06

Hispanic

0.15

0.16

– 0.01

Other

0.25

0.24

0.01

Educational attainment (percent)

Less than high school diploma

0.37

0.33

0.04

0.74

High school diploma/GED

0.36

0.36

0.00

More than high school diploma

0.27

0.31

– 0.04

Number of adults in family (percent)

1 adult

0.68

0.61

0.06

2 or more adults

0.32

0.39

– 0.06

0.31

Number of children in family (percent)

1 child

0.40

0.40

0.00

1.00

2 children

0.29

0.28

0.01

3 children

0.19

0.20

– 0.01

4 children or more

0.12

0.12

0.00

Missing data

0.00

0.00

0.00

Worked for pay past week (percent)

0.17

0.28

– 0.11

0.03

**

Ever convicted of a felony (percent)

0.10

0.16

– 0.06

0.16

Family annual income (percent)

Less than $5,000

0.29

0.27

0.02

0.83

$5,000–9,999

0.26

0.24

0.02

$10,000–14,999

0.18

0.19

0.00

$15,000–19,999

0.13

0.12

0.01

$20,000–24,999

0.05

0.08

– 0.03

$25,000 or more

0.05

0.04

0.01

Missing data

0.03

0.07

– 0.03

Ever been homeless before (percent)

0.61

0.55

0.07

0.29

Ever been doubled up before (percent)

0.83

0.81

0.01

0.80

Major barrier to finding housinga

0.44

0.41

0.03

0.68

Child under 18 living elsewhere (percent)

0.23

0.32

– 0.09

0.09

*

Number of major life challengesb

1.69

1.45

0.24

0.42

F-test on all characteristics except site

F value =

1.029

F-test p-value =

0.428

PBTH = priority access to project-based transitional housing.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

17

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-19. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for UC, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

556

190

75%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.04

0.04

0.28

21–24 years

0.20

0.19

0.00

25–29 years

0.23

0.28

– 0.05

30–34 years

0.16

0.17

– 0.01

35–44 years

0.23

0.24

– 0.01

45 years and older

0.10

0.07

0.02

Mean age (years)

31.22

31.38

– 0.15

0.22

Gender (percent)

Female

0.93

0.92

0.02

0.46

Male

0.07

0.08

– 0.02

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.73

0.67

0.06

0.13

Married or marriage-like situation

0.27

0.33

– 0.06

Race/ethnicity (percent)

Black/African American, not Hispanic

0.42

0.40

0.02

0.77

White, not Hispanic

0.19

0.23

– 0.03

Hispanic

0.21

0.22

– 0.01

Other

0.17

0.15

0.01

Educational attainment (percent)

Less than high school diploma

0.39

0.42

– 0.02

0.84

High school diploma/GED

0.33

0.33

0.00

More than high school diploma

0.28

0.26

0.02

Number of adults in family (percent)

1 adult

0.71

0.67

0.05

2 or more adults

0.29

0.33

– 0.05

0.23

Number of children in family (percent)

1 child

0.44

0.41

0.03

0.02

**

2 children

0.31

0.26

0.06

3 children

0.15

0.16

– 0.01

4 children or more

0.08

0.17

– 0.08

Missing data

0.01

0.01

0.00

Worked for pay past week (percent)

0.19

0.14

0.05

0.13

Ever convicted of a felony (percent)

0.11

0.09

0.02

0.36

Family annual income (percent)

Less than $5,000

0.33

0.33

0.00

0.18

$5,000–9,999

0.26

0.30

– 0.04

$10,000–14,999

0.17

0.14

0.04

$15,000–19,999

0.08

0.11

– 0.02

$20,000–24,999

0.06

0.06

– 0.01

$25,000 or more

0.06

0.06

0.00

Missing data

0.03

0.00

0.03

Ever been homeless before (percent)

0.64

0.56

0.08

0.06

*

Ever been doubled up before (percent)

0.85

0.84

0.02

0.56

Major barrier to finding housinga

0.46

0.42

0.04

0.32

Child under 18 living elsewhere (percent)

0.22

0.25

– 0.03

0.43

Number of major life challengesb

1.66

1.52

0.14

0.66

F-test on all characteristics except site

F value =

1.378

F-test p-value =

0.085

UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

18

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-20. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB Versus UC

Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

896

243

79%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.06

0.03

0.79

21–24 years

0.22

0.20

0.01

25–29 years

0.23

0.27

– 0.04

30–34 years

0.17

0.20

– 0.03

35–44 years

0.21

0.20

0.01

45 years and older

0.09

0.07

0.02

Mean age (years)

30.61

30.51

0.11

0.67

Gender (percent)

Female

0.93

0.92

0.01

0.56

Male

0.07

0.08

– 0.01

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.72

0.69

0.03

0.17

Married or marriage-like situation

0.28

0.31

– 0.03

Race/ethnicity (percent)

Black/African American, not Hispanic

0.37

0.33

0.04

0.84

White, not Hispanic

0.21

0.27

– 0.06

Hispanic

0.24

0.22

0.02

Other

0.18

0.18

0.00

Educational attainment (percent)

Less than high school diploma

0.37

0.42

– 0.05

0.08

*

High school diploma/GED

0.37

0.35

0.02

More than high school diploma

0.26

0.23

0.03

Number of adults in family (percent)

1 adult

0.71

0.67

0.04

2 or more adults

0.29

0.33

– 0.04

0.20

Number of children in family (percent)

1 child

0.43

0.44

– 0.01

0.93

2 children

0.32

0.27

0.05

3 children

0.15

0.15

0.00

4 children or more

0.09

0.13

– 0.04

Missing data

0.01

0.00

0.01

Worked for pay past week (percent)

0.15

0.12

0.04

0.30

Ever convicted of a felony (percent)

0.11

0.11

0.00

0.82

Family annual income (percent)

Less than $5,000

0.33

0.35

– 0.02

0.96

$5,000–9,999

0.31

0.31

0.00

$10,000–14,999

0.17

0.15

0.02

$15,000–19,999

0.07

0.07

0.00

$20,000–24,999

0.04

0.07

– 0.02

$25,000 or more

0.05

0.05

0.00

Missing data

0.03

0.01

0.02

Ever been homeless before (percent)

0.63

0.59

0.05

0.23

Ever been doubled up before (percent)

0.86

0.85

0.01

0.26

Major barrier to finding housinga

0.45

0.42

0.02

0.92

Child under 18 living elsewhere (percent)

0.24

0.27

– 0.04

0.63

Number of major life challengesb

1.60

1.68

– 0.08

0.75

F-test on all characteristics except site

F value =

0.996

F-test p-value =

0.474

SUB = priority access to permanent housing subsidy. UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

19

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-21. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for CBRR Versus UC

Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

868

276

76%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.06

0.03

0.13

21–24 years

0.19

0.20

– 0.01

25–29 years

0.23

0.25

– 0.02

30–34 years

0.16

0.19

– 0.03

35–44 years

0.23

0.24

– 0.01

45 years and older

0.09

0.06

0.03

Mean age (years)

30.94

30.72

0.22

0.77

Gender (percent)

Female

0.93

0.91

0.02

0.51

Male

0.07

0.09

– 0.02

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.75

0.68

0.07

0.87

Married or marriage-like situation

0.25

0.32

– 0.07

Race/ethnicity (percent)

Black/African American, not Hispanic

0.46

0.40

0.06

0.24

White, not Hispanic

0.19

0.22

– 0.03

Hispanic

0.20

0.19

0.01

Other

0.15

0.19

– 0.04

Educational attainment (percent)

Less than high school diploma

0.34

0.42

– 0.07

0.01

**

High school diploma/GED

0.37

0.33

0.04

More than high school diploma

0.29

0.25

0.04

Number of adults in family (percent)

1 adult

0.72

0.68

0.05

2 or more adults

0.28

0.32

– 0.05

0.76

Number of children in family (percent)

1 child

0.42

0.47

– 0.05

0.05

*

2 children

0.31

0.26

0.05

3 children

0.15

0.12

0.03

4 children or more

0.11

0.14

– 0.02

Missing data

0.01

0.01

0.00

Worked for pay past week (percent)

0.19

0.18

0.02

0.44

Ever convicted of a felony (percent)

0.10

0.12

– 0.02

0.65

Family annual income (percent)

Less than $5,000

0.31

0.31

0.00

0.09

*

$5,000–9,999

0.28

0.30

– 0.01

$10,000–14,999

0.18

0.13

0.05

$15,000–19,999

0.09

0.11

– 0.02

$20,000–24,999

0.05

0.07

– 0.02

$25,000 or more

0.07

0.08

– 0.01

Missing data

0.02

0.01

0.02

Ever been homeless before (percent)

0.63

0.60

0.03

0.78

Ever been doubled up before (percent)

0.87

0.82

0.05

0.36

Major barrier to finding housinga

0.45

0.39

0.05

0.15

Child under 18 living elsewhere (percent)

0.22

0.28

– 0.06

0.74

Number of major life challengesb

1.56

1.66

– 0.10

0.44

F-test on all characteristics except site

F value =

1.498

F-test p-value =

0.038

CBRR = priority access to community-based rapid re-housing. UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

20

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-22. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for PBTH Versus UC

Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

552

155

78%

Age of household head at RA (percent)

Less than 21 years old

0.08

0.05

0.03

0.38

21–24 years

0.16

0.20

– 0.04

25–29 years

0.24

0.25

0.00

30–34 years

0.20

0.21

– 0.01

35–44 years

0.23

0.25

– 0.02

45 years and older

0.10

0.05

0.05

Mean age (years)

31.59

31.06

0.52

0.58

Gender (percent)

Female

0.92

0.86

0.06

0.27

Male

0.08

0.14

– 0.06

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.68

0.62

0.06

0.94

Married or marriage-like situation

0.32

0.38

– 0.06

Race/ethnicity (percent)

Black/African American, not Hispanic

0.41

0.34

0.08

0.96

White, not Hispanic

0.18

0.25

– 0.06

Hispanic

0.15

0.19

– 0.04

Other

0.25

0.23

0.02

Educational attainment (percent)

Less than high school diploma

0.39

0.41

– 0.02

0.59

High school diploma/GED

0.35

0.32

0.03

More than high school diploma

0.26

0.27

– 0.01

Number of adults in family (percent)

1 adult

0.67

0.59

0.08

2 or more adults

0.33

0.41

– 0.08

0.85

Number of children in family (percent)

1 child

0.41

0.39

0.02

0.48

2 children

0.28

0.30

– 0.01

3 children

0.19

0.15

0.04

4 children or more

0.11

0.15

– 0.04

Missing data

0.00

0.01

0.00

Worked for pay past week (percent)

0.20

0.21

– 0.01

0.07

*

Ever convicted of a felony (percent)

0.12

0.14

– 0.02

0.11

Family annual income (percent)

Less than $5,000

0.30

0.30

0.00

0.61

$5,000–9,999

0.24

0.28

– 0.03

$10,000–14,999

0.18

0.15

0.03

$15,000–19,999

0.11

0.10

0.01

$20,000–24,999

0.06

0.08

– 0.02

$25,000 or more

0.06

0.06

0.00

Missing data

0.05

0.03

0.01

Ever been homeless before (percent)

0.63

0.53

0.10

0.36

Ever been doubled up before (percent)

0.83

0.83

0.00

0.62

Major barrier to finding housinga

0.45

0.46

– 0.02

0.79

Child under 18 living elsewhere (percent)

0.23

0.28

– 0.05

0.70

Number of major life challengesb

1.67

1.48

0.19

0.89

F-test on all characteristics except site

F value =

1.233

F-test p-value =

0.188

PBTH = priority access to project-based transitional housing. UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

21

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-23. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB Versus CBRR Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

652

165

80%

Age of household head at RA (percent)

Less than 21 years old

0.10

0.08

0.01

0.46

21–24 years

0.20

0.17

0.03

25–29 years

0.23

0.24

– 0.01

30–34 years

0.19

0.24

– 0.05

35–44 years

0.20

0.20

0.00

45 years and older

0.08

0.06

0.02

Mean age (years)

30.37

30.42

– 0.05

0.07

*

Gender (percent)

Female

0.93

0.90

0.03

0.27

Male

0.07

0.10

– 0.03

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.75

0.68

0.08

0.98

Married or marriage-like situation

0.25

0.32

– 0.08

Race/ethnicity (percent)

Black/African American, not Hispanic

0.41

0.28

0.13

0.32

White, not Hispanic

0.22

0.29

– 0.07

Hispanic

0.22

0.19

0.03

Other

0.14

0.24

– 0.09

Educational attainment (percent)

Less than high school diploma

0.35

0.39

– 0.05

0.77

High school diploma/GED

0.40

0.36

0.05

More than high school diploma

0.25

0.25

0.00

Number of adults in family (percent)

1 adult

0.73

0.68

0.04

2 or more adults

0.27

0.32

– 0.04

0.54

Number of children in family (percent)

1 child

0.43

0.54

– 0.11

0.51

2 children

0.32

0.26

0.06

3 children

0.13

0.11

0.02

4 children or more

0.11

0.08

0.03

Missing data

0.00

0.01

0.00

Worked for pay past week (percent)

0.15

0.12

0.03

0.39

Ever convicted of a felony (percent)

0.11

0.15

– 0.04

0.79

Family annual income (percent)

Less than $5,000

0.32

0.35

– 0.02

0.61

$5,000–9,999

0.33

0.26

0.06

$10,000–14,999

0.18

0.13

0.05

$15,000–19,999

0.07

0.07

0.00

$20,000–24,999

0.04

0.10

– 0.06

$25,000 or more

0.05

0.08

– 0.03

Missing data

0.01

0.01

0.00

Ever been homeless before (percent)

0.63

0.62

0.01

0.89

Ever been doubled up before (percent)

0.86

0.80

0.06

0.14

Major barrier to finding housinga

0.44

0.39

0.05

0.22

Child under 18 living elsewhere (percent)

0.22

0.33

– 0.10

0.09

*

Number of major life challengesb

1.55

1.92

– 0.36

0.65

F-test on all characteristics except site

F value =

1.920

F-test p-value =

0.002

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

22

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-24. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB Versus PBTH Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

416

80

84%

Age of household head at RA (percent)

Less than 21 years old

0.08

0.04

0.04

0.46

21–24 years

0.20

0.24

– 0.03

25–29 years

0.24

0.26

– 0.02

30–34 years

0.19

0.24

– 0.05

35–44 years

0.19

0.18

0.01

45 years and older

0.09

0.05

0.04

Mean age (years)

30.70

29.98

0.73

0.73

Gender (percent)

Female

0.91

0.88

0.04

0.28

Male

0.09

0.13

– 0.04

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.69

0.70

– 0.01

0.80

Married or marriage-like situation

0.31

0.30

0.01

Race/ethnicity (percent)

Black/African American, not Hispanic

0.36

0.21

0.15

0.31

White, not Hispanic

0.22

0.34

– 0.12

Hispanic

0.19

0.18

0.01

Other

0.24

0.28

– 0.04

Educational attainment (percent)

Less than high school diploma

0.30

0.38

– 0.07

0.86

High school diploma/GED

0.42

0.35

0.07

More than high school diploma

0.28

0.28

0.01

Number of adults in family (percent)

1 adult

0.68

0.63

0.05

2 or more adults

0.32

0.38

– 0.05

0.88

Number of children in family (percent)

1 child

0.41

0.44

– 0.03

0.35

2 children

0.32

0.25

0.07

3 children

0.18

0.21

– 0.03

4 children or more

0.08

0.10

– 0.02

Missing data

0.00

0.00

0.00

Worked for pay past week (percent)

0.16

0.23

– 0.07

0.15

Ever convicted of a felony (percent)

0.13

0.14

– 0.01

0.36

Family annual income (percent)

Less than $5,000

0.30

0.34

– 0.04

0.56

$5,000–9,999

0.30

0.29

0.01

$10,000–14,999

0.18

0.14

0.04

$15,000–19,999

0.09

0.08

0.02

$20,000–24,999

0.06

0.08

– 0.02

$25,000 or more

0.05

0.04

0.01

Missing data

0.04

0.05

– 0.01

Ever been homeless before (percent)

0.62

0.56

0.06

0.35

Ever been doubled up before (percent)

0.83

0.85

– 0.02

0.66

Major barrier to finding housinga

0.45

0.48

– 0.03

0.98

Child under 18 living elsewhere (percent)

0.24

0.25

– 0.01

0.07

*

Number of major life challengesb

1.66

1.59

0.07

0.70

F-test on all characteristics except site

F value =

1.118

F-test p-value =

0.306

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

23

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-25. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for CBRR Versus PBTH Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

364

107

77%

Age of household head at RA (percent)

Less than 21 years old

0.08

0.03

0.05

0.60

21–24 years

0.14

0.18

– 0.03

25–29 years

0.27

0.22

0.04

30–34 years

0.21

0.24

– 0.03

35–44 years

0.22

0.28

– 0.06

45 years and older

0.08

0.05

0.03

Mean age (years)

31.12

31.36

– 0.24

0.33

Gender (percent)

Female

0.91

0.84

0.07

0.80

Male

0.09

0.16

– 0.07

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.70

0.62

0.08

0.11

Married or marriage-like situation

0.30

0.38

– 0.08

Race/ethnicity (percent)

Black/African American, not Hispanic

0.45

0.39

0.06

0.86

White, not Hispanic

0.16

0.24

– 0.08

Hispanic

0.13

0.16

– 0.03

Other

0.26

0.21

0.05

Educational attainment (percent)

Less than high school diploma

0.35

0.36

– 0.01

0.05

**

High school diploma/GED

0.34

0.36

– 0.03

More than high school diploma

0.31

0.27

0.04

Number of adults in family (percent)

1 adult

0.68

0.59

0.09

2 or more adults

0.32

0.41

– 0.09

0.05

*

Number of children in family (percent)

1 child

0.40

0.41

– 0.01

0.78

2 children

0.28

0.35

– 0.07

3 children

0.16

0.16

0.00

4 children or more

0.15

0.08

0.07

Missing data

0.00

0.00

0.00

Worked for pay past week (percent)

0.22

0.28

– 0.06

0.62

Ever convicted of a felony (percent)

0.11

0.14

– 0.03

0.81

Family annual income (percent)

Less than $5,000

0.27

0.22

0.05

0.93

$5,000–9,999

0.27

0.28

– 0.01

$10,000–14,999

0.19

0.18

0.01

$15,000–19,999

0.12

0.12

0.00

$20,000–24,999

0.05

0.10

– 0.06

$25,000 or more

0.07

0.06

0.01

Missing data

0.03

0.04

0.00

Ever been homeless before (percent)

0.62

0.54

0.08

0.79

Ever been doubled up before (percent)

0.87

0.79

0.08

0.04

**

Major barrier to finding housinga

0.44

0.37

0.07

0.69

Child under 18 living elsewhere (percent)

0.25

0.31

– 0.06

0.18

Number of major life challengesb

1.51

1.47

0.04

0.25

F-test on all characteristics except site

F value =

1.556

F-test p-value =

0.033

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

24

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-26. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB + CBRR +

PBTH Versus UC Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

1,784

498

78%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.06

0.03

0.52

21–24 years

0.19

0.19

0.00

25–29 years

0.24

0.25

– 0.01

30–34 years

0.18

0.21

– 0.03

35–44 years

0.22

0.23

– 0.01

45 years and older

0.08

0.06

0.03

Mean age (years)

30.82

30.81

0.01

0.90

Gender (percent)

Female

0.93

0.89

0.04

0.50

Male

0.07

0.11

– 0.04

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.73

0.67

0.06

0.96

Married or marriage-like situation

0.27

0.33

– 0.06

Race/ethnicity (percent)

Black/African American, not Hispanic

0.42

0.36

0.06

0.73

White, not Hispanic

0.19

0.24

– 0.05

Hispanic

0.20

0.20

0.00

Other

0.18

0.19

– 0.01

Educational attainment (percent)

Less than high school diploma

0.35

0.41

– 0.05

0.02

**

High school diploma/GED

0.37

0.34

0.03

More than high school diploma

0.27

0.26

0.02

Number of adults in family (percent)

1 adult

0.71

0.66

0.05

2 or more adults

0.29

0.34

– 0.05

0.98

Number of children in family (percent)

1 child

0.42

0.45

– 0.03

0.26

2 children

0.31

0.27

0.04

3 children

0.15

0.15

0.01

4 children or more

0.11

0.12

– 0.02

Missing data

0.01

0.01

0.00

Worked for pay past week (percent)

0.17

0.17

0.00

0.15

Ever convicted of a felony (percent)

0.11

0.13

– 0.02

0.64

Family annual income (percent)

Less than $5,000

0.31

0.32

– 0.01

0.46

$5,000–9,999

0.29

0.29

0.00

$10,000–14,999

0.18

0.14

0.03

$15,000–19,999

0.09

0.09

0.00

$20,000–24,999

0.05

0.08

– 0.03

$25,000 or more

0.06

0.06

– 0.01

Missing data

0.03

0.02

0.01

Ever been homeless before (percent)

0.63

0.59

0.03

0.39

Ever been doubled up before (percent)

0.85

0.82

0.03

0.92

Major barrier to finding housinga

0.45

0.40

0.04

0.33

Child under 18 living elsewhere (percent)

0.23

0.28

– 0.06

0.88

Number of major life challengesb

1.60

1.66

– 0.06

0.72

F-test on all characteristics except site

F value =

2.146

F-test p-value =

0.000

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

25

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-27. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB + PBTH

Versus CBRR Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

928

240

79%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.07

0.02

0.49

21–24 years

0.18

0.18

0.01

25–29 years

0.24

0.23

0.00

30–34 years

0.20

0.24

– 0.04

35–44 years

0.21

0.22

– 0.01

45 years and older

0.08

0.06

0.02

Mean age (years)

30.70

30.70

0.00

0.05

**

Gender (percent)

Female

0.93

0.87

0.05

0.50

Male

0.07

0.13

– 0.05

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.74

0.65

0.09

0.75

Married or marriage-like situation

0.26

0.35

– 0.09

Race/ethnicity (percent)

Black/African American, not Hispanic

0.43

0.33

0.10

0.26

White, not Hispanic

0.20

0.29

– 0.09

Hispanic

0.20

0.17

0.03

Other

0.18

0.22

– 0.04

Educational attainment (percent)

Less than high school diploma

0.35

0.38

– 0.03

0.11

High school diploma/GED

0.38

0.37

0.01

More than high school diploma

0.27

0.26

0.02

Number of adults in family (percent)

1 adult

0.72

0.65

0.06

2 or more adults

0.28

0.35

– 0.06

0.22

Number of children in family (percent)

1 child

0.42

0.49

– 0.07

0.38

2 children

0.31

0.30

0.01

3 children

0.14

0.13

0.02

4 children or more

0.13

0.09

0.04

Missing data

0.00

0.00

0.00

Worked for pay past week (percent)

0.17

0.17

0.00

0.62

Ever convicted of a felony (percent)

0.11

0.15

– 0.04

1.00

Family annual income (percent)

Less than $5,000

0.30

0.30

0.00

0.88

$5,000–9,999

0.31

0.26

0.05

$10,000–14,999

0.18

0.16

0.02

$15,000–19,999

0.09

0.09

0.00

$20,000–24,999

0.04

0.09

– 0.05

$25,000 or more

0.06

0.07

– 0.01

Missing data

0.02

0.03

– 0.01

Ever been homeless before (percent)

0.63

0.60

0.03

0.95

Ever been doubled up before (percent)

0.86

0.81

0.04

0.06

*

Major barrier to finding housinga

0.44

0.40

0.04

0.42

Child under 18 living elsewhere (percent)

0.23

0.32

– 0.08

0.47

Number of major life challengesb

1.56

1.74

– 0.18

0.40

F-test on all characteristics except site

F value =

1.989

F-test p-value =

0.001

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Source: Family Options Baseline Survey; 18-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

26

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-28. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for SUB + CBRR

Versus PBTH Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

685

166

80%

Age of household head at RA (percent)

Less than 21 years old

0.08

0.03

0.05

0.51

21–24 years

0.18

0.20

– 0.02

25–29 years

0.25

0.23

0.02

30–34 years

0.20

0.24

– 0.05

35–44 years

0.21

0.25

– 0.05

45 years and older

0.09

0.05

0.04

Mean age (years)

30.87

31.02

– 0.15

0.55

Gender (percent)

Female

0.91

0.86

0.05

0.56

Male

0.09

0.14

– 0.05

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.68

0.65

0.03

0.78

Married or marriage-like situation

0.32

0.35

– 0.03

Race/ethnicity (percent)

Black/African American, not Hispanic

0.40

0.31

0.09

0.90

White, not Hispanic

0.19

0.27

– 0.08

Hispanic

0.16

0.17

– 0.01

Other

0.25

0.25

0.00

Educational attainment (percent)

Less than high school diploma

0.33

0.37

– 0.05

0.09

*

High school diploma/GED

0.39

0.36

0.03

More than high school diploma

0.28

0.27

0.01

Number of adults in family (percent)

1 adult

0.66

0.61

0.06

2 or more adults

0.34

0.39

– 0.06

0.58

Number of children in family (percent)

1 child

0.41

0.42

– 0.01

0.41

2 children

0.30

0.30

0.00

3 children

0.17

0.18

– 0.01

4 children or more

0.11

0.10

0.02

Missing data

0.00

0.00

0.00

Worked for pay past week (percent)

0.19

0.25

– 0.06

0.28

Ever convicted of a felony (percent)

0.12

0.13

– 0.02

0.35

Family annual income (percent)

Less than $5,000

0.28

0.29

0.00

0.62

$5,000–9,999

0.28

0.28

0.00

$10,000–14,999

0.18

0.14

0.04

$15,000–19,999

0.10

0.09

0.01

$20,000–24,999

0.05

0.10

– 0.05

$25,000 or more

0.06

0.05

0.00

Missing data

0.04

0.04

– 0.01

Ever been homeless before (percent)

0.63

0.54

0.08

0.56

Ever been doubled up before (percent)

0.85

0.81

0.04

0.11

Major barrier to finding housinga

0.44

0.42

0.02

0.79

Child under 18 living elsewhere (percent)

0.26

0.28

– 0.02

0.12

Number of major life challengesb

1.56

1.51

0.06

0.38

F-test on all characteristics except site

F value =

1.676

F-test p-value =

0.013

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Source: Family Options Baseline Survey; 18-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

27

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Exhibit D-29. Equivalence at Baseline of Analysis Sample for Respondents Versus Nonrespondents for PBTH + CBRR

Versus SUB Impact Comparison, 37-Month Adult Survey

Characteristic

Respondents

Nonrespondents

Difference

Significance Level

Stars

Number of families

954

219

81%

Age of household head at RA (percent)

Less than 21 years old

0.09

0.07

0.02

0.36

21–24 years

0.20

0.18

0.02

25–29 years

0.24

0.25

– 0.01

30–34 years

0.19

0.24

– 0.05

35–44 years

0.20

0.20

0.00

45 years and older

0.08

0.05

0.02

Mean age (years)

30.35

30.34

0.01

0.22

Gender (percent)

Female

0.92

0.88

0.04

0.16

Male

0.08

0.12

– 0.04

Marital status (percent)

Single (never married/widowed/separated/divorced)

0.73

0.69

0.04

0.69

Married or marriage-like situation

0.27

0.31

– 0.04

Race/ethnicity (percent)

Black/African American, not Hispanic

0.39

0.26

0.13

0.08

*

White, not Hispanic

0.22

0.30

– 0.08

Hispanic

0.21

0.19

0.02

Other

0.18

0.25

– 0.07

Educational attainment (percent)

Less than high school diploma

0.33

0.38

– 0.05

0.77

High school diploma/GED

0.40

0.35

0.05

More than high school diploma

0.26

0.26

0.00

Number of adults in family (percent)

1 adult

0.71

0.67

0.04

2 or more adults

0.29

0.33

– 0.04

0.43

Number of children in family (percent)

1 child

0.42

0.50

– 0.08

0.71

2 children

0.32

0.26

0.06

3 children

0.14

0.14

0.01

4 children or more

0.11

0.10

0.01

Missing data

0.01

0.00

0.00

Worked for pay past week (percent)

0.14

0.16

– 0.01

0.64

Ever convicted of a felony (percent)

0.11

0.15

– 0.04

0.52

Family annual income (percent)

Less than $5,000

0.31

0.33

– 0.02

0.57

$5,000–9,999

0.31

0.27

0.04

$10,000–14,999

0.18

0.15

0.03

$15,000–19,999

0.08

0.08

0.01

$20,000–24,999

0.04

0.09

– 0.05

$25,000 or more

0.05

0.06

– 0.01

Missing data

0.02

0.03

0.00

Ever been homeless before (percent)

0.63

0.63

0.00

0.76

Ever been doubled up before (percent)

0.85

0.81

0.04

0.57

Major barrier to finding housinga

0.44

0.41

0.03

0.38

Child under 18 living elsewhere (percent)

0.22

0.31

– 0.08

0.05

**

Number of major life challengesb

1.60

1.85

– 0.25

0.66

F-test on all characteristics except site

F value =

2.024

F-test p-value =

0.001

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

GED = general educational development. RA = random assignment.

* .10 level. ** .05 level. *** .01 level.

a Barriers to finding housing were reported by family heads as “big problems” in finding housing. The maximum number of barriers is 19.

b The seven major life challenges measured are: psychological distress, post-traumatic stress disorder, felony conviction, experience of domestic violence, childhood separation (foster care, group home, or institutionalization), medical condition, and substance abuse.

Notes: Each row reports the proportion of each group with the characteristic. Chi-square tests are used to test for significant differences in the proportions between groups for all characteristics except mean age, for which a t-test is used. The F-test is of the joint significance of all listed characteristics in a regression predicting assignment group. The regression also includes site indicators (which are not included in the joint test).

Sources: Family Options Baseline Survey; 37-month followup survey

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

28

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

Unweighted Impact Estimates

vary with observable participant characteristics. The impact

models also controlled for observable baseline characteristics.

How were the main results of this report affected by the use of

Together, these measures adjust the impact estimates to reflect

nonresponse analysis weights? As discussed in Appendix C, the

potential differences between the groups in each comparison

study team used nonresponse weights to produce all estimates in

that could have been induced by nonresponse. In this section,

this report. This methodology is motivated in part by the finding

we look for evidence of such variation by comparing the study’s

in the previous section that for most impact comparisons,

headline impact estimates to estimates that do not use survey

respondents and nonrespondents differed systematically on a

nonresponse weights. The results are also presented for com-

number of baseline characteristics. Survey nonresponse weights

pleteness for readers interested in the unadjusted estimates.

adjust impact estimates such that the analysis sample reflects the

observable characteristics of the baseline sample. This adjust-

Exhibit D-30 presents these unadjusted estimates, which are

ment represents a “correction,” however, only insofar as impacts

comparable to Exhibit ES-7 in the executive summary. Changes

Exhibit D-30. Executive Summary Impact Estimates, Estimated Without Nonresponse Weights

Mean

ITT Impact Estimates

Outcome

All UC

SUB

CBRR

PBTH

SUB

SUB

CBRR

Group

vs. UC

vs. UC

vs. UC

vs. CBRR

vs. PBTH vs. PBTH

Housing stability (intervention goal: lower values)

At least 1 night homelessa or doubled up in past 6 months or in shelter

38.3

– 20.9***

1.4

1.1

– 19.9***

– 25.7***

– 3.0

in past 12 monthsb (%) [confirmatory]

At least 1 night homelessa or doubled up in past 6 months (%)

34.4

– 18.2***

1.6

1.4

– 16.4***

– 23.8***

– 3.1

Number of places lived in past 6 months

1.58

– 0.23***

0.05

– 0.04

– 0.16**

– 0.33***

0.09

Any stay in emergency shelter in months 7 to 18 after RA (%)

17.1

– 14.1***

– 2.6

– 6.2**

– 11.9***

– 6.0**

1.8

Family preservation (intervention goal: lower values)

Family has at least one child separated in past 6 monthsc (%)

15.6

– 2.7

– 0.1

2.1

– 0.7

– 7.3*

– 3.6

Spouse/partner separated in past 6 months, of those with spouse/partner 38.4

12.9**

7.1

9.9

– 7.2

1.2

– 1.9

present at RAd (%) [limited base]

Family has no child reunified, of those families with at least one child

67.3

– 6.6

– 3.8

1.6

– 5.3

– 17.2

– 5.3

absent at RAe (%) [limited base]

Adult well-being (intervention goal: lower values)

Health in past 30 days was poor or fair (%)

31.0

2.7

0.8

– 2.7

– 1.2

– 2.2

– 0.8

Psychological distressf

0.00

– 0.13**

– 0.02

– 0.03

– 0.06

– 0.10

– 0.27**

Alcohol dependence or drug abuse in past 6 monthsg (%)

11.4

– 2.3

– 1.9

2.1

1.9

0.5

– 6.4*

Experienced intimate partner violence in past 6 months (%)

10.8

– 4.5**

– 1.8

– 2.1

– 1.8

– 0.9

– 0.6

Child well-being (intervention goal: lower values)

Number of schools attended since RAh

2.08

– 0.15**

0.04

0.05

– 0.23***

– 0.16

0.03

School absences in past month (ages 5 to 17 years)i

0.98

– 0.04

– 0.09

– 0.16

– 0.02

– 0.11

– 0.23

Poor or fair health (%)

5.8

1.1

– 0.8

– 0.4

0.8

0.7

– 0.7

Behavior problemsj

0.58

– 0.22***

– 0.21**

– 0.11

0.00

– 0.20

– 0.21*

Self-sufficiency (intervention goal: higher values)

Work for pay in week before survey (%)

37.8

– 0.7

0.5

0.0

0.1

– 1.5

– 3.6

Total family income ($)

12,117

– 853

– 758

– 234

– 409

– 1,324

– 1,850

Household is food secure (%)

53.4

8.9***

3.5

2.2

3.2

13.0***

11.6**

Number of families

556

895

868

551

652

414

363

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test (not adjusted for multiple comparisons).

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing. Additional impacts on the use of transitional housing are provided in Appendix E.

b After adjustment for multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB-versus-UC, SUB-versus-CBRR, and SUB-versus-PBTH comparisons.

c Percentage of families in which a child who was with the family at baseline became separated from the family in the 6 months before the 37-month survey.

d Percentage of families in which a spouse or partner who was with the family at baseline became separated from the family in the 6 months before the 37-month survey.

e Percentage of families in which at least one child was separated from the family at baseline and no child was reunited with the family at the time of the 37-month survey.

f Psychological distress is measured with the Kessler 6 (K6) scale and ranges from 0 to 24, with higher scores indicating greater distress. Impacts shown as standardized effect sizes. Effect sizes were standardized by dividing impacts by standard deviation for the UC group.

g Measures evidence of alcohol dependence or drug abuse using responses to the Rapid Alcohol Problems Screen (RAPS-4) and six items from the Drug Abuse Screening Test (DAST-10).

h Number of schools outcome is topcoded at 4 or more schools.

i Absences outcome is defined as 0 = no absences in past month; 1 = one to two absences; 2 = three to five absences; 3 = six or more absences. This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

j Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire, or SDQ.

Notes: Impact estimates are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

29

FAMILY OPTIONS STUDY

Appendix D. Analysis of 37-Month Survey Nonresponse

to coefficient signs, magnitudes, and statistical significance are

Changes of sign (that is, from positive to negative) were limited

minimal. In every case, the 95-percent confidence intervals

to coefficients that were both close in magnitude to zero and

associated with impact coefficients estimated without nonre-

estimated as statistically insignificant. Changes in statistical

sponse weights included the coefficient estimates in the main

significance were minor.

weighted specification, with intervals largely overlapping.

D

3-Year Impacts of Housing and Services Interventions for Homeless Families

30

FAMILY OPTIONS STUDY

APPENDIX E.

IMPACTS ON USE OF

TRANSITIONAL HOUSING

This appendix contains impacts on nine additional 7. Any use of emergency shelter or transitional housing in outcomes related to use of transitional housing as well

months 7 to 32 after random assignment (percentage of

as emergency shelter during the 37-month followup

families).

period. These outcomes more closely measure the impact of

8. Any use of emergency shelter in months 7 to 32 after

random assignment to contrasting interventions on homeless-

random assignment (percentage of families).

ness as defined in the McKinney-Vento Homeless Assistance

Act and Opening Doors: Federal Strategic Plan to Prevent and

9. Any use of transitional housing in months 7 to 32 after

End Homelessness. That Act includes residence in a transitional

random assignment (percentage of families).

housing program as one type of homelessness. The additional

outcomes are—

The new outcomes are measured with Program Usage Data.

They differ from outcomes in the analyses of housing stability

1. Any use of emergency shelter or transitional housing in

provided in Chapters 3 through 6, which did not consider use

months 0 to 32 after random assignment (percentage of

of transitional housing, either separately or in conjunction with

families).

the use of emergency shelter.

2. Any use of emergency shelter during months 0 to 32 after

This appendix includes six exhibits, one for each pairwise com-

random assignment (percentage of families).

parison: (1) priority access to permanent housing subsidy (SUB)

versus usual care (UC), (2) priority access to community-

3. Any use of transitional housing in months 0 to 32 after

based rapid re-housing (CBRR) versus UC, (3) priority access to

random assignment (percentage of families).

project- based transitional housing (PBTH) versus UC, (4) SUB

4. Number of months using emergency shelter or transitional

versus CBRR, (5) SUB versus PBTH, and (6) CBRR versus PBTH.

housing in months 0 to 32 after random assignment.

For comparisons involving PBTH (Exhibits E-3, E-5, and E-6),

impacts on the use of transitional housing in part reflects

5. Number of months using emergency shelter in months 0 to

takeup of the assigned intervention encouraged and facilitated

32 after random assignment.

for the PBTH group—but not the other group included in the

6. Number of months using transitional housing in months 0 to

comparison—by the study design.

32 after random assignment.

E

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix E. Impacts on Use of Transitional Housing

Exhibit E-1. SUB Versus UC: Impacts on Use of ES and TH in Months 0 to 32 After RA SUB

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Any use of ES or TH during months 0 to 32

Any use of emergency shelter or transitional housing during

501

85.6

(36.3)

395

91.9

(25.3)

– 6.3 ***

(2.1)

– 0.20

months 0 to 32 after RA (%)

Any use of emergency shelter during months 0 to 32 after RA (%)

501

84.5

(37.4)

395

88.5

(29.9)

– 4.0

*

(2.3)

– 0.11

Any use of transitional housing during months 0 to 32 after RA (%)

501

7.0

(26.2)

395

28.4

(44.2)

– 21.3 ***

(2.7)

– 0.41

Number of months of ES and TH during months 0 to 32

Number of months of emergency shelter and transitional housing

501

3.0

(4.5)

395

7.0

(8.0)

– 4.0 ***

(0.5)

– 0.43

use during months 0 to 32 after RA

Number of months of emergency shelter use during

501

2.5

(3.0)

395

3.9

(5.2)

– 1.4 ***

(0.2)

– 0.26

months 0 to 32 after RA

Number of months of transitional housing use during

501

0.5

(3.2)

395

3.1

(6.5)

– 2.6 ***

(0.4)

– 0.33

months 0 to 32 after RA

Any use of ES or TH during months 7 to 32

Any use of emergency shelter or transitional housing during

501

15.7

(36.5)

395

47.9

(50.1)

– 2.2 ***

(3.0)

– 0.56

months 7 to 32 after RA (%)

Any use of emergency shelter during months 7 to 32 after RA (%)

501

13.8

(34.1)

395

36.7

(48.6)

– 22.9 ***

(2.9)

– 0.41

Any use of transitional housing during months 7 to 32 after RA (%)

501

4.4

(21.4)

395

23.9

(41.7)

– 19.4 ***

(2.5)

– 0.39

SUB = priority access to permanent housing subsidy. UC = usual care.

ES = emergency shelter. ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. TH = transitional housing.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Source: Family Options Study Program Usage Data

Exhibit E-2. CBRR Versus UC: Impacts on Use of ES and TH in Months 0 to 32 After RA CBRR

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Any use of ES or TH during months 0 to 32

Any use of emergency shelter or transitional housing during

434

92.8

(24.6)

434

91.9

(26.5)

0.9

(1.9)

0.03

months 0 to 32 after RA (%)

Any use of emergency shelter during months 0 to 32 after RA (%)

434

90.4

(28.6)

434

89.4

(30.5)

0.9

(2.1)

0.03

Any use of transitional housing during months 0 to 32 after RA (%)

434

21.7

(41.6)

434

27.8

(44.9)

– 6.1

**

(3.0)

– 0.12

Number of months of ES and TH during months 0 to 32

Number of months of emergency shelter and transitional housing

434

5.9

(7.4)

434

7.0

(8.0)

– 1.1

**

(0.5)

– 0.12

use during months 0 to 32 after RA

Number of months of emergency shelter use during

434

3.7

(4.6)

434

4.0

(5.1)

– 0.3

(0.3)

– 0.05

months 0 to 32 after RA

Number of months of transitional housing use during

434

2.1

(5.5)

434

3.0

(6.6)

– 0.8

**

(0.4)

– 0.11

months 0 to 32 after RA

Any use of ES or TH during months 7 to 32

Any use of emergency shelter or transitional housing during

434

43.8

(49.6)

434

48.6

(50.0)

– 4.8

(3.6)

– 0.08

months 7 to 32 after RA (%)

Any use of emergency shelter during months 7 to 32 after RA (%)

434

35.7

(47.7)

434

38.0

(48.7)

– 2.4

(3.5)

– 0.04

Any use of transitional housing during months 7 to 32 after RA (%)

434

17.3

(38.1)

434

22.8

(42.0)

– 5.4

*

(2.8)

– 0.11

CBRR = priority access to community-based rapid re-housing. UC = usual care.

ES = emergency shelter. ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. TH = transitional housing.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Source: Family Options Study Program Usage Data

E

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix E. Impacts on Use of Transitional Housing

Exhibit E-3. PBTH Versus UC: Impacts on Use of ES and TH in Months 0 to 32 After RA PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Any use of ES or TH during months 0 to 32

Any use of emergency shelter or transitional housing during

293

95.1

(20.6)

259

92.9

(24.8)

2.2

(2.1)

0.07

months 0 to 32 after RA (%)

Any use of emergency shelter during months 0 to 32 after RA (%)

293

82.5

(37.7)

259

88.8

(32.1)

– 6.3

**

(3.0)

– 0.17

Any use of transitional housing during months 0 to 32 after RA (%)

293

52.3

(49.9)

259

33.9

(47.8)

18.3

***

(4.5)

0.35

Number of months of ES and TH during months 0 to 32

Number of months of emergency shelter and transitional housing

293

10.1

(9.6)

259

7.0

(8.5)

3.0

***

(0.8)

0.32

use during months 0 to 32 after RA

Number of months of emergency shelter use during

293

2.7

(3.5)

259

3.7

(4.9)

– 1.0

***

(0.3)

– 0.19

months 0 to 32 after RA

Number of months of transitional housing use during

293

7.4

(9.6)

259

3.3

(7.3)

4.1

***

(0.7)

0.52

months 0 to 32 after RA

Any use of ES or TH during months 7 to 32

Any use of emergency shelter or transitional housing during

293

58.3

(49.3)

259

46.9

(50.1)

11.4

**

(4.5)

0.20

months 7 to 32 after RA (%)

Any use of emergency shelter during months 7 to 32 after RA (%)

293

24.5

(43.1)

259

32.6

(47.3)

– 8.1

**

(4.0)

– 0.15

Any use of transitional housing during months 7 to 32 after RA (%)

293

43.2

(49.8)

259

26.6

(45.1)

16.6

***

(4.4)

0.33

PBTH = priority access to project-based transitional housing. UC = usual care.

ES = emergency shelter. ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. TH = transitional housing.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Source: Family Options Study Program Usage Data

Exhibit E-4. SUB Versus CBRR: Impacts on Use of ES and TH in Months 0 to 32 After RA SUB

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Any use of ES or TH during months 0 to 32

Any use of emergency shelter or transitional housing during

362

85.7

(35.9)

290

91.7

(26.5)

– 6.1

**

(2.6)

– 0.19

months 0 to 32 after RA (%)

Any use of emergency shelter during months 0 to 32 after RA (%)

362

84.6

(36.7)

290

90.5

(27.6)

– 5.9

**

(2.7)

– 0.16

Any use of transitional housing during months 0 to 32 after RA (%)

362

7.5

(25.4)

290

18.9

(39.5)

– 11.4 ***

(2.7)

– 0.22

Number of months of ES and TH during months 0 to 32

Number of months of emergency shelter and transitional housing

362

3.0

(4.1)

290

5.6

(7.0)

– 2.6 ***

(0.4)

– 0.28

use during months 0 to 32 after RA

Number of months of emergency shelter use during

362

2.5

(3.2)

290

4.0

(5.1)

– 1.5 ***

(0.3)

– 0.28

months 0 to 32 after RA

Number of months of transitional housing use during

362

0.5

(2.5)

290

1.6

(4.4)

– 1.1 ***

(0.3)

– 0.14

months 0 to 32 after RA

Any use of ES or TH during months 7 to 32

Any use of emergency shelter or transitional housing during

362

15.9

(37.0)

290

43.6

(49.8)

– 27.7 ***

(3.8)

– 0.48

months 7 to 32 after RA (%)

Any use of emergency shelter during months 7 to 32 after RA (%)

362

14.5

(35.4)

290

36.0

(48.2)

– 21.5 ***

(3.7)

– 0.39

Any use of transitional housing during months 7 to 32 after RA (%)

362

4.7

(20.6)

290

15.9

(37.2)

– 11.3 ***

(2.4)

– 0.23

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

ES = emergency shelter. ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. TH = transitional housing.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Source: Family Options Study Program Usage Data

E

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix E. Impacts on Use of Transitional Housing

Exhibit E-5. SUB Versus PBTH: Impacts on Use of ES and TH in Months 0 to 32 After RA SUB

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Any use of ES or TH during months 0 to 32

Any use of emergency shelter or transitional housing during

215

88.7

(33.2)

201

93.4

(20.7)

– 4.7

(3.2)

– 0.15

months 0 to 32 after RA (%)

Any use of emergency shelter during months 0 to 32 after RA (%)

215

88.4

(34.2)

201

81.3

(36.7)

7.1

*

(3.8)

0.20

Any use of transitional housing during months 0 to 32 after RA (%)

215

9.2

(29.1)

201

49.3

(50.1)

– 40.2 ***

(4.3)

– 0.77

Number of months of ES and TH during months 0 to 32

Number of months of emergency shelter and transitional housing

215

3.0

(5.1)

201

9.8

(9.7)

– 6.8 ***

(0.8)

– 0.72

use during months 0 to 32 after RA

Number of months of emergency shelter use during

215

2.1

(2.8)

201

2.8

(3.8)

– 0.7

**

(0.3)

– 0.12

months 0 to 32 after RA

Number of months of transitional housing use during

215

0.9

(4.1)

201

7.0

(9.8)

– 6.1 ***

(0.7)

– 0.77

months 0 to 32 after RA

Any use of ES or TH during months 7 to 32

Any use of emergency shelter or transitional housing during

215

11.3

(32.7)

201

58.5

(49.3)

– 47.2 ***

(4.5)

– 0.82

months 7 to 32 after RA (%)

Any use of emergency shelter during months 7 to 32 after RA (%)

215

7.0

(27.0)

201

26.8

(43.9)

– 19.9 ***

(4.0)

– 0.36

Any use of transitional housing during months 7 to 32 after RA (%)

215

6.3

(23.9)

201

40.4

(49.6)

– 34.1

**

(4.0)

– 0.69

PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

ES = emergency shelter. ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. TH = transitional housing.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Source: Family Options Study Program Usage Data

Exhibit E-6. CBRR Versus PBTH: Impacts on Use of ES and TH in Months 0 to 32 After RA CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Any use of ES or TH during months 0 to 32

Any use of emergency shelter or transitional housing during

180

92.3

(26.0)

184

94.9

(20.4)

– 2.6

(2.6)

– 0.08

months 0 to 32 after RA (%)

Any use of emergency shelter during months 0 to 32 after RA (%)

180

86.9

(32.2)

184

86.0

(37.0)

1.0

(3.7)

0.03

Any use of transitional housing during months 0 to 32 after RA (%)

180

30.9

(46.4)

184

51.4

(49.8)

– 20.4 ***

(5.7)

– 0.39

Number of months of ES and TH during months 0 to 32

Number of months of emergency shelter and transitional housing

180

6.1

(8.3)

184

9.6

(9.6)

– 3.4 ***

(1.0)

– 0.37

use during months 0 to 32 after RA

Number of months of emergency shelter use during

180

3.2

(4.0)

184

2.6

(3.4)

0.6

(0.4)

0.11

months 0 to 32 after RA

Number of months of transitional housing use during

180

2.9

(6.5)

184

6.9

(9.5)

– 4.0 ***

(0.9)

– 0.51

months 0 to 32 after RA

Any use of ES or TH during months 7 to 32

Any use of emergency shelter or transitional housing during

180

42.2

(49.5)

184

55.4

(49.4)

– 13.2

**

(5.5)

– 0.23

months 7 to 32 after RA (%)

Any use of emergency shelter during months 7 to 32 after RA (%)

180

30.1

(46.2)

184

25.0

(43.7)

5.1

(5.3)

0.09

Any use of transitional housing during months 7 to 32 after RA (%)

180

22.4

(42.1)

184

42.7

(50.0)

– 20.3 ***

(5.3)

– 0.41

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing.

ES = emergency shelter. ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. TH = transitional housing.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Appendix B for outcome definitions.

Source: Family Options Study Program Usage Data

E

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

APPENDIX F.

IMPACT ESTIMATES FOR POOLED

COMPARISONS

List of Exhibits

Exhibit F-1. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-13. SUB+PBTH Versus CBRR: Impacts on

Housing Stability at 37 Months ....................F-2

Housing Stability at 37 Months ..................F-10

Exhibit F-2. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-14. SUB+PBTH Versus CBRR: Impacts on

Family Preservation at 37 Months ................F-2

Family Preservation at 37 Months ..............F-10

Exhibit F-3. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-15. SUB+PBTH Versus CBRR: Impacts on

Adult Well-Being at 37 Months ....................F-3

Adult Well-Being at 37 Months ..................F-11

Exhibit F-4. SUB+CBRR+PBTH Versus UC: Impacts

Exhibit F-16. SUB+PBTH Versus CBRR: Impacts on

on Child Well-Being Across Age Groups at

Child Well-Being Across Age Groups at

37 Months ....................................................F-3

37 Months ..................................................F-11

Exhibit F-5. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-17. SUB+PBTH Versus CBRR: Impacts on

Child Well-Being Developmental Outcomes

Child Well-Being Developmental Outcomes

by Age Group at 37 Months .........................F-4

by Age Group at 37 Months .......................F-12

Exhibit F-6. SUB+CBRR+PBTH Versus UC: Impacts on

Exhibit F-18. SUB+PBTH Versus CBRR: Impacts on

Self-Sufficiency at 37 Months .......................F-5

Self-Sufficiency at 37 Months .....................F-13

Exhibit F-7. SUB+CBRR Versus PBTH: Impacts on

Exhibit F-19. CBRR+PBTH Versus SUB: Impacts on

Housing Stability at 37 Months ....................F-6

Housing Stability at 37 Months ..................F-14

Exhibit F-8. SUB+CBRR Versus PBTH: Impacts on

Exhibit F-20. CBRR+PBTH Versus SUB: Impacts on

Family Preservation at 37 Months ................F-6

Family Preservation at 37 Months ..............F-14

Exhibit F-9. SUB+CBRR Versus PBTH: Impacts on

Exhibit F-21. CBRR+PBTH Versus SUB: Impacts on

Adult Well-Being at 37 Months ....................F-7

Adult Well-Being at 37 Months ..................F-15

Exhibit F-10. SUB+CBRR Versus PBTH: Impacts on

Exhibit F-22. CBRR+PBTH Versus SUB: Impacts on

Child Well-Being Across Age Groups at

Child Well-Being Across Age Groups at

37 Months ....................................................F-7

37 Months ..................................................F-15

Exhibit F-11. SUB+CBRR Versus PBTH: Impacts on

Exhibit F-23. CBRR+PBTH Versus SUB: Impacts on

Child Well-Being Developmental Outcomes

Child Well-Being Developmental Outcomes

by Age Group at 37 Months .........................F-8

by Age Group at 37 Months .......................F-16

Exhibit F-12. SUB+CBRR Versus PBTH: Impacts on

Exhibit F-24. CBRR+PBTH Versus SUB: Impacts on

Self-Sufficiency at 37 Months .......................F-9

Self-Sufficiency at 37 Months .....................F-17

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-1. SUB+CBRR+PBTH Versus UC: Impacts on Housing Stability at 37 Months

SUB+CBRR+PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the follow up period

At least 1 night homelessb or doubled up in past 6 months or in

1,226

30.3

(46.0)

556

38.9

(48.7)

– 8.6

***

(2.6)

– 0.15

shelter in past 12 monthsc (%)

At least 1 night homelessb or doubled up in past 6 months (%)

1,226

27.7

(44.7)

556

34.8

(47.5)

– 7.1

***

(2.5)

– 0.13

At least 1 night homelessb in past 6 months (%)

1,227

13.8

(34.5)

556

18.0

(38.4)

– 4.2

**

(2.0)

– 0.10

At least 1 night doubled up in past 6 months (%)

1,227

22.0

(41.4)

556

28.4

(45.1)

– 6.4

***

(2.3)

– 0.12

Any stay in emergency shelter in past 6 months (%)

1,228

5.2

(22.2)

556

8.5

(27.8)

– 3.2

**

(1.4)

– 0.10

[Program Usage Data]

Any stay in emergency shelter in months 21 to 32 after RA (%)

1,228

9.6

(29.5)

556

17.6

(37.7)

– 8.1

***

(1.9)

– 0.19

[Program Usage Data]

Number of days homelessb or doubled up in past 6 months

1,224

36.1

(66.1)

553

47.5

(74.0)

– 11.4

***

(3.9)

– 0.13

Number of days homelessb in past 6 months

1,227

12.3

(39.2)

556

18.4

(48.2)

– 6.1

**

(2.4)

– 0.11

Number of days doubled up in past 6 months

1,225

26.2

(57.5)

553

33.1

(62.3)

– 6.8

**

(3.4)

– 0.09

Housing independence

Living in own house or apartment at followup (%)

1,228

74.3

(43.7)

556

69.1

(46.3)

5.2

**

(2.4)

0.10

Living in own house or apartment with no housing assistance (%)

1,224

29.2

(45.4)

553

41.4

(49.1)

– 12.2

***

(2.5)

– 0.21

Living in own house or apartment with housing assistance (%)

1,224

45.2

(49.8)

553

27.6

(45.2)

17.6

***

(2.4)

0.34

Number of places lived

Number of places lived in past 6 monthsd

1,225

1.5

(1.0)

556

1.6

(1.0)

– 0.1

(0.1)

– 0.06

Housing quality

Persons per room

1,166

1.5

(1.1)

526

1.6

(1.2)

– 0.1

**

(0.1)

– 0.10

Housing quality is poor or fair (%)

1,166

30.5

(45.8)

525

32.7

(46.8)

– 2.2

(2.6)

– 0.04

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c After adjustment of multiple comparisons, the impact on the confirmatory outcome is statistically significant at the .01 level for the SUB+CBRR+PBTH-versus-UC comparison.

d The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

Exhibit F-2. SUB+CBRR+PBTH Versus UC: Impacts on Family Preservation at 37 Months SUB+CBRR+PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

1,199

15.9

(36.5)

545

17.0

(36.3)

– 1.1

(2.0)

– 0.02

Family has at least one foster care placement in past 6 monthsb (%)

1,208

3.4

(18.1)

550

3.5

(17.3)

– 0.1

(1.1)

0.00

Spouse/partner separated in past 6 months, of those with spouse/

317

42.1

(49.0)

151

34.8

(48.8)

7.3

(5.0)

0.12

partner present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at least

231

40.6

(49.5)

107

35.7

(47.1)

4.9

(6.2)

0.09

one child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent at

120

28.1

(45.3)

55

22.1

(40.4)

5.9

(9.0)

0.13

RA (%)

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-3. SUB+CBRR+PBTH Versus UC: Impacts on Adult Well-Being at 37 Months

SUB+CBRR+PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

1,224

31.3

(46.3)

555

30.8

(46.3)

0.5

(2.4)

0.01

Adult mental health

Goal-oriented thinkingb

1,211

4.46

(1.02)

553

4.52

(0.98)

– 0.06

(0.05)

– 0.06

Psychological distressc

1,223

6.75

(5.57)

554

7.07

(5.79)

– 0.33

(0.29)

– 0.05

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past

1,217

20.9

(40.4)

552

22.5

(41.8)

– 1.6

(2.2)

– 0.03

30 days (%)

Adult substance use

Alcohol dependence or drug abused (%)

1,226

11.3

(31.8)

554

11.9

(31.8)

– 0.5

(1.7)

– 0.01

Alcohol dependenced (%)

1,227

8.6

(28.3)

555

8.8

(27.3)

– 0.2

(1.5)

– 0.01

Drug abused (%)

1,226

3.9

(19.4)

555

5.2

(21.9)

– 1.3

(1.2)

– 0.05

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

1,224

7.6

(26.5)

553

10.7

(31.1)

– 3.1 *

(1.6)

– 0.09

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler 6 (K6) scale and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit F-4. SUB+CBRR+PBTH Versus UC: Impacts on Child Well-Being Across Age Groups at 37 Months SUB+CBRR+PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

1,442

2.1

(0.9)

655

2.1

(0.9)

– 0.0

(0.1)

– 0.01

Grade completion (not held back) (%)

1,198

90.85 (29.47)

539

91.05 (28.24)

– 0.20

(1.72)

– 0.01

School gradesc

1,095

3.1

(0.9)

485

3.1

(0.9)

0.0

(0.1)

0.01

Child physical health

Poor or fair health in past 30 days (%)

1,670

6.3

(23.4)

743

5.7

(23.4)

0.6

(1.3)

0.02

Well-child checkup in past year (%)

1,669

89.5

(30.6)

741

91.0

(29.1)

– 1.5

(1.7)

– 0.04

Child has regular source of health care (%)

1,666

91.8

(26.2)

742

91.6

(27.1)

0.2

(1.8)

0.01

Sleep problemsd

1,669

2.06

(1.05)

744

2.20

(1.09)

– 0.14 ** (0.06)

– 0.10

Child behavioral strengths and challenges

Behavior problemse

1,589

0.45

(1.23)

709

0.59

(1.25)

– 0.15 ** (0.07)

– 0.09

Prosocial behaviorf

.

.

(.)

.

.

(.)

.

(.)

.

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-5. SUB+CBRR+PBTH Versus UC: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months

SUB+CBRR+PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

516

37.1

(48.7)

223

37.3

(49.6)

– 0.2

(4.4)

0.00

Child care or preschool absences in past monthd

195

0.73

(0.92)

93

0.76

(0.91)

– 0.03

(0.14)

– 0.03

Positive child care or preschool experiencese

205

0.83

(0.42)

97

0.82

(0.39)

0.01

(0.06)

0.02

Positive child care or preschool attitudesf

205

4.57

(0.90)

96

4.50

(0.69)

0.07

(0.11)

0.08

Child care or preschool conduct problemsg (%)

217

7.7

(30.3)

98

6.9

(24.1)

0.7

(3.5)

0.03

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

451

71.1

(45.1)

204

70.4

(45.7)

0.6

(5.0)

0.01

Ages 3 years, 6 months to 7 years

Verbal abilityi

582 – 0.23

(1.02)

246

– 0.34

(1.04)

0.11

(0.10)

0.08

Math abilityj

580 – 0.27

(0.94)

249

– 0.30

(0.95)

0.03

(0.09)

0.03

Executive functioningk (self-regulation)

554

16.00 (16.16)

239

18.41

(16.39)

– 2.41

**

(1.10)

– 0.12

Ages 5 to 17 yearsl

School enrollmentc (%)

1,162

97.9

(13.6)

525

97.6

(13.7)

0.3

(1.0)

0.02

School absences in past monthd,m

438

0.89

(0.93)

176

1.04

(0.98)

– 0.15

(0.10)

– 0.12

Positive school experiencese,m

441

0.54

(0.59)

178

0.47

(0.63)

0.08

(0.07)

0.09

Positive school attitudesf,m

441

4.17

(1.10)

178

4.06

(1.12)

0.11

(0.10)

0.07

School conduct problemsg,m (%)

442

25.0

(43.4)

180

30.7

(46.6)

– 5.7

(4.4)

– 0.09

Ages 8 to 17 years

Anxietyn

716

35.39

(7.45)

328

35.17

(7.53)

0.22

(0.56)

0.02

Fearso

724

63.29 (14.43)

329

62.95

(14.75)

0.34

(0.98)

0.02

Substance usep (%)

706

6.00 (24.95)

321

7.30

(27.80)

– 1.30

(2.01)

– 0.04

Goal-oriented thinkingq

693

22.39

(4.92)

316

22.22

(4.65)

0.17

(0.38)

0.03

School effort in past monthr

715

2.76

(0.81)

323

2.80

(0.78)

– 0.04

(0.06)

– 0.04

Arrests or police involvement in past 6 monthss (%)

386

9.99 (29.83)

181

8.36

(25.89)

1.63

(3.04)

0.04

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Sources: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-6. SUB+CBRR+PBTH Versus UC: Impacts on Self-Sufficiency at 37 Months

SUB+CBRR+PBTH

UC

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

1,227

38.5

(48.7)

556

37.2

(48.5)

1.2

(2.5)

0.02

Any work for pay since 20-month surveyb (%)

.

.

(.)

.

.

(.)

.

(.)

.

Months worked for pay since 20-month surveyb,c

.

.

(.)

.

.

(.)

.

(.)

.

Any work for pay since RA (%)

1,227

71.6

(44.4)

555

73.7

(45.0)

– 2.1

(2.2)

– 0.04

Months worked for pay since RAc

1,214

13.2

(13.5)

542

13.6

(13.5)

– 0.4

(0.6)

– 0.02

Hours of work per week at current main jobd

1,223

12.2

(17.0)

555

12.0

(16.8)

0.2

(0.9)

0.01

Income sources and amounts

Annualized current earnings ($)

1,204

6,810 (10,983)

542

6,531 (10,759)

279

(536)

0.02

Total family income ($)

1,175 11,762

(9,994)

536 12,314 (11,331)

– 552

(548)

– 0.04

Anyone in family had earnings in past month (%)

1,228

49.8

(50.0)

556

50.4

(50.0)

– 0.6

(2.6)

– 0.01

Anyone in family received TANF in past month (%)

1,227

27.0

(44.5)

556

22.7

(42.4)

4.3 **

(2.2)

0.09

Anyone in family received SSDI in past month (%)

1,225

8.7

(27.4)

556

8.7

(29.9)

– 0.1

(1.4)

0.00

Anyone in family received SSI in past month (%)

1,227

14.1

(34.4)

556

14.4

(36.0)

– 0.3

(1.5)

– 0.01

Anyone in family received SNAP/Food Stamps in past month (%)

1,227

82.1

(38.0)

556

81.8

(39.0)

0.4

(2.0)

0.01

Anyone in family received WIC in past month (%)

1,227

26.2

(44.5)

556

24.3

(42.5)

1.9

(2.2)

0.04

Education and training

Participated in 2 weeks or more weeks of any school or training

1,223

38.8

(48.9)

555

39.3

(48.8)

– 0.5

(2.5)

– 0.01

since RA (%)

Number of weeks in school/training programs since RA

1,210

7.1

(14.9)

546

8.8

(17.0)

– 1.7 *

(0.9)

– 0.08

Participated in 2 weeks or more weeks of school since RA (%)

1,222

10.6

(30.7)

555

11.8

(31.5)

– 1.3

(1.7)

– 0.03

Participated in 2 weeks or more weeks of basic education since RA (%) 1,222

2.7

(16.5)

555

2.1

(15.7)

0.6

(0.7)

0.03

Participated in 2 weeks or more weeks of vocational education

1,222

10.7

(31.5)

555

15.0

(34.6)

– 4.2 **

(1.8)

– 0.10

since RA (%)

Food security and hunger

Household is food insecure (%)

1,228

40.9

(49.1)

556

46.2

(49.9)

– 5.3 **

(2.7)

– 0.09

Food insecurity scalee

1,222

1.71

(2.03)

554

1.91

(2.08)

– 0.21 *

(0.11)

– 0.09

Economic stressors

Economic stress scalef

1,218 – 0.18

(0.48)

554

– 0.12

(0.49)

– 0.06 ** (0.02)

– 0.10

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability

Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-7. SUB+CBRR Versus PBTH: Impacts on Housing Stability at 37 Months

SUB+CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the follow up period

At least 1 night homelessb or doubled up in past 6 months or in

394

28.8

(45.7)

289

42.5

(49.5)

– 13.7 ***

(4.1)

– 0.24

shelter in past 12 months (%)

At least 1 night homelessb or doubled up in past 6 months (%)

394

26.2

(44.3)

289

38.9

(49.0)

– 12.7 ***

(4.1)

– 0.23

At least 1 night homelessb in past 6 months (%)

394

14.9

(35.5)

290

20.1

(40.1)

– 5.2

*

(3.2)

– 0.12

At least 1 night doubled up in past 6 months (%)

395

20.6

(41.0)

289

29.2

(46.0)

– 8.7

**

(3.9)

– 0.17

Any stay in emergency shelter in past 6 months (%)

395

5.8

(23.4)

290

7.4

(26.5)

– 1.5

(2.0)

– 0.05

[Program Usage Data]

Any stay in emergency shelter in months 21 to 32 after RA (%)

395

8.5

(27.7)

290

10.2

(29.6)

– 1.7

(2.4)

– 0.04

[Program Usage Data]

Number of days homelessb or doubled up in past 6 months

394

33.9

(65.8)

289

48.5

(72.8)

– 14.6

**

(6.1)

– 0.17

Number of days homelessb in past 6 months

394

12.1

(40.0)

290

17.0

(44.3)

– 4.9

(3.4)

– 0.09

Number of days doubled up in past 6 months

395

22.6

(55.4)

289

34.3

(64.5)

– 11.6

**

(5.4)

– 0.16

Housing independence

Living in own house or apartment at followup (%)

395

75.6

(42.9)

290

65.9

(47.5)

9.7

**

(3.9)

0.18

Living in own house or apartment with no housing assistance (%)

393

27.2

(44.2)

289

40.8

(49.3)

– 13.5 ***

(3.9)

– 0.24

Living in own house or apartment with housing assistance (%)

393

48.7

(50.1)

289

25.0

(43.1)

23.6 ***

(3.9)

0.46

Number of places lived

Number of places lived in past 6 monthsc

395

1.5

(1.0)

287

1.6

(1.0)

– 0.1

(0.1)

– 0.09

Housing quality

Persons per room

376

1.6

(1.2)

272

1.7

(1.3)

– 0.1

(0.1)

– 0.09

Housing quality is poor or fair (%)

378

27.4

(44.4)

270

33.2

(47.2)

– 5.8

(4.0)

– 0.11

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

Exhibit F-8. SUB+CBRR Versus PBTH: Impacts on Family Preservation at 37 Months

SUB+CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

386

15.0

(36.0)

280

21.5

(41.1)

– 6.5 **

(3.1)

– 0.15

Family has at least one foster care placement in past 6 monthsb (%)

388

4.8

(19.9)

283

3.1

(19.4)

1.7

(1.7)

0.08

Spouse/partner separated in past 6 months, of those with spouse/

127

33.3

(47.8)

81

36.6

(47.4)

– 3.3

(7.5)

– 0.06

partner present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at

93

48.7

(50.0)

56

38.6

(49.9)

10.1

(10.2)

0.18

least one child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent

42

31.0

(45.7)

27

35.4

(46.5)

– 4.4

(15.7)

– 0.09

at RA (%)

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-9. SUB+CBRR Versus PBTH: Impacts on Adult Well-Being at 37 Months

SUB+CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

393

31.9

(46.4)

288

30.8

(46.1)

1.1

(3.9)

0.02

Adult mental health

Goal-oriented thinkingb

387

4.45

(0.99)

284

4.39

(1.04)

0.06

(0.09)

0.05

Psychological distressc

393

6.38

(5.72)

289

6.95

(5.51)

– 0.57

(0.44)

– 0.08

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past

390

19.8

(39.9)

288

19.0

(39.4)

0.8

(3.3)

0.02

30 days (%)

Adult substance use

Alcohol dependence or drug abused (%)

394

11.4

(32.5)

290

14.0

(34.5)

– 2.6

(2.7)

– 0.07

Alcohol dependenced (%)

394

9.4

(29.6)

290

10.8

(31.0)

– 1.4

(2.4)

– 0.04

Drug abused (%)

394

3.3

(17.9)

290

5.5

(23.5)

– 2.3

(1.6)

– 0.08

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

393

8.9

(28.9)

289

7.7

(27.1)

1.2

(2.4)

0.03

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler 6 (K6) scale and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit F-10. SUB+CBRR Versus PBTH: Impacts on Child Well-Being Across Age Groups at 37 Months SUB+CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

457

2.1

(1.0)

336

2.1

(1.0)

– 0.1

(0.1)

– 0.05

Grade completion (not held back) (%)

384

93.01 (25.16)

284

87.60 (33.72)

5.42 ** (2.53)

0.14

School gradesc

351

3.1

(0.9)

251

3.0

(0.9)

0.1

(0.1)

0.09

Child physical health

Poor or fair health in past 30 days (%)

543

5.3

(23.2)

389

6.3

(21.6)

– 1.0

(2.1)

– 0.03

Well-child checkup in past year (%)

544

90.6

(29.4)

389

89.8

(32.3)

0.7

(2.5)

0.02

Child has regular source of health care (%)

544

93.4

(24.9)

385

88.2

(29.5)

5.1

(3.3)

0.14

Sleep problemsd

544

2.08

(1.04)

388

2.14

(1.10)

– 0.06

(0.08)

– 0.04

Child behavioral strengths and challenges

Behavior problemse

515

0.36

(1.17)

370

0.54

(1.19)

– 0.18

*

(0.10)

– 0.11

Prosocial behaviorf

516

– 0.11

(1.05)

370

– 0.25

(1.22)

0.14

(0.09)

0.10

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

7

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-11. SUB+CBRR Versus PBTH: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months

SUB+CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

178

36.3

(48.7)

114

33.8

(48.2)

2.6

(5.8)

0.04

Child care or preschool absences in past monthd

64

0.70

(1.01)

44

0.73

(0.81)

– 0.03

(0.22)

– 0.03

Positive child care or preschool experiencese

65

0.89

(0.36)

49

0.76

(0.54)

0.13

(0.09)

0.27

Positive child care or preschool attitudesf

66

4.60

(0.82)

48

4.52

(1.07)

0.08

(0.18)

0.10

Child care or preschool conduct problemsg (%)

70

4.1

(20.4)

53

8.9

(34.2)

– 4.8

(5.9)

– 0.17

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

148

69.4

(47.0)

95

68.8

(47.5)

0.6

(7.1)

0.01

Ages 3 years, 6 months to 7 years

Verbal abilityi

198

– 0.20

(1.10)

131

– 0.48

(1.06)

0.28

(0.18)

0.21

Math abilityj

195

– 0.30

(1.02)

131

– 0.36

(1.01)

0.06

(0.15)

0.05

Executive functioningk (self-regulation)

183

15.64 (16.28)

120

15.12 (15.80)

0.52

(1.98)

0.03

Ages 5 to 17 yearsl

School enrollmentc (%)

369

98.9

(11.6)

277

95.4

(14.6)

3.5

(2.5)

0.18

School absences in past monthd,m

136

0.79

(0.92)

102

0.95

(0.94)

– 0.16

(0.13)

– 0.13

Positive school experiencese,m

135

0.57

(0.54)

105

0.55

(0.64)

0.02

(0.09)

0.02

Positive school attitudesf,m

135

4.21

(1.08)

105

4.12

(1.09)

0.09

(0.15)

0.06

School conduct problemsg,m (%)

135

27.1

(44.4)

105

28.2

(45.8)

– 1.2

(6.8)

– 0.02

Ages 8 to 17 years

Anxietyn

221

35.14

(7.02)

166

35.02

(7.72)

0.12

(0.95)

0.01

Fearso

224

63.39 (14.83)

169

62.02 (14.76)

1.37

(1.45)

0.07

Substance usep (%)

217

4.11 (22.91)

161

10.74 (33.09)

– 6.63 ** (3.25)

– 0.18

Goal-oriented thinkingq

211

22.26

(4.62)

159

22.83

(5.00)

– 0.57

(0.56)

– 0.09

School effort in past monthr

219

2.59

(0.81)

168

2.71

(0.84)

– 0.12

(0.10)

– 0.11

Arrests or police involvement in past 6 monthss (%)

117

10.98 (30.47)

83

15.07 (36.57)

– 4.08

(5.76)

– 0.11

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Sources: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

8

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-12. SUB+CBRR Versus PBTH: Impacts on Self-Sufficiency at 37 Months

SUB+CBRR

PBTH

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

394

37.5

(48.3)

290

38.8

(48.9)

– 1.3

(4.0)

– 0.02

Any work for pay since 20-month surveyb (%)

369

60.6

(48.9)

256

57.9

(49.5)

2.7

(3.9)

0.06

Months worked for pay since 20-month surveyb,c

368

6.8

(7.7)

254

7.3

(8.3)

– 0.5

(0.6)

– 0.06

Any work for pay since RA (%)

394

72.4

(44.4)

290

72.1

(44.9)

0.3

(3.3)

0.01

Months worked for pay since RAc

389

13.3

(13.3)

287

13.8

(13.7)

– 0.5

(1.0)

– 0.03

Hours of work per week at current main jobd

393

12.0

(16.6)

289

12.7

(18.0)

– 0.7

(1.4)

– 0.04

Income sources and amounts

Annualized current earnings ($)

385

6,594 (10,302)

285

7,534 (12,128)

– 940

(969)

– 0.08

Total family income ($)

371

12,009 (9,631)

287

12,936 (11,711)

– 926

(812)

– 0.07

Anyone in family had earnings in past month (%)

395

49.2

(50.0)

290

53.3

(49.9)

– 4.1

(4.0)

– 0.07

Anyone in family received TANF in past month (%)

395

26.2

(44.8)

289

25.6

(43.1)

0.6

(3.6)

0.01

Anyone in family received SSDI in past month (%)

393

8.8

(26.2)

290

7.6

(26.5)

1.1

(2.1)

0.03

Anyone in family received SSI in past month (%)

395

13.3

(33.0)

289

12.8

(33.9)

0.6

(2.5)

0.01

Anyone in family received SNAP/Food Stamps in past month (%)

395

81.0

(38.4)

289

82.2

(39.0)

– 1.2

(3.0)

– 0.03

Anyone in family received WIC in past month (%)

395

27.7

(45.2)

289

29.6

(46.0)

– 1.9

(3.7)

– 0.04

Education and training

Participated in 2 weeks or more weeks of any school or training

393

40.4

(49.2)

287

36.9

(48.4)

3.5

(4.0)

0.06

since RA (%)

Number of weeks in school/training programs since RA

386

8.0

(16.2)

286

5.7

(12.4)

2.3 **

(1.2)

0.12

Participated in 2 weeks or more weeks of school since RA (%)

392

12.3

(32.2)

287

9.6

(30.6)

2.7

(2.5)

0.07

Participated in 2 weeks or more weeks of basic education since RA (%)

392

1.5

(13.3)

287

2.2

(13.1)

– 0.7

(1.2)

– 0.04

Participated in 2 weeks or more weeks of vocational education

392

8.1

(28.6)

287

11.0

(30.2)

– 2.8

(2.6)

– 0.07

since RA (%)

Food security and hunger

Household is food insecure (%)

395

34.4

(47.8)

290

46.7

(49.9)

– 12.3 ***

(4.1)

– 0.21

Food insecurity scalee

394

1.44

(1.89)

286

1.90

(2.03)

– 0.47 ***

(0.17)

– 0.19

Economic stressors

Economic stress scalef

390

– 0.19

(0.48)

287

– 0.14

(0.51)

– 0.05

(0.04)

– 0.09

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability

Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

9

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-13. SUB+PBTH Versus CBRR: Impacts on Housing Stability at 37 Months

SUB+PBTH

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the follow up period

At least 1 night homelessb or doubled up in past 6 months or in

545

25.8

(43.6)

382

40.4

(48.9)

– 14.6 ***

(3.4)

– 0.26

shelter in past 12 months (%)

At least 1 night homelessb or doubled up in past 6 months

545

23.3

(42.3)

382

35.5

(47.7)

– 12.2 ***

(3.3)

– 0.22

(%)

At least 1 night homelessb in past 6 months (%)

546

13.1

(34.1)

382

16.0

(36.2)

– 2.8

(2.6)

– 0.06

At least 1 night doubled up in past 6 months (%)

545

16.9

(37.7)

382

31.0

(45.8)

– 14.1 ***

(3.0)

– 0.27

Any stay in emergency shelter in past 6 months (%)

546

4.2

(20.5)

382

5.6

(23.8)

– 1.4

(1.6)

– 0.04

[Program Usage Data]

Any stay in emergency shelter in months 21 to 32 after RA (%)

546

7.4

(26.1)

382

16.8

(36.4)

– 9.4 ***

(2.5)

– 0.22

[Program Usage Data]

Number of days homelessb or doubled up in past 6 months

544

28.1

(59.8)

381

49.6

(73.6)

– 21.6 ***

(4.9)

– 0.25

Number of days homelessb in past 6 months

546

11.8

(39.6)

382

14.6

(41.1)

– 2.8

(3.1)

– 0.05

Number of days doubled up in past 6 months

544

18.5

(49.3)

381

38.7

(66.1)

– 20.2 ***

(4.3)

– 0.27

Housing independence

Living in own house or apartment at followup (%)

546

77.8

(41.4)

382

68.8

(46.8)

9.1 ***

(3.2)

0.17

Living in own house or apartment with no housing assistance (%)

545

24.8

(42.4)

380

36.9

(48.2)

– 12.1 ***

(3.2)

– 0.21

Living in own house or apartment with housing assistance (%)

545

52.9

(49.8)

380

32.4

(46.5)

20.5 ***

(3.4)

0.40

Number of places lived

Number of places lived in past 6 monthsc

543

1.4

(0.9)

382

1.6

(1.0)

– 0.1

**

(0.1)

– 0.11

Housing quality

Persons per room

521

1.3

(0.9)

362

1.6

(1.3)

– 0.3 ***

(0.1)

– 0.23

Housing quality is poor or fair (%)

520

30.1

(45.5)

365

30.2

(45.7)

– 0.1

(3.3)

0.00

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

Exhibit F-14. SUB+PBTH Versus CBRR: Impacts on Family Preservation at 37 Months

SUB+PBTH

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

530

15.9

(36.9)

379

15.6

(35.8)

0.3

(2.6)

0.01

Family has at least one foster care placement in past 6 monthsb (%)

535

3.2

(17.6)

379

4.5

(18.9)

– 1.2

(1.4)

– 0.06

Spouse/partner separated in past 6 months, of those with spouse/

133

37.3

(49.3)

106

43.9

(48.7)

– 6.6

(6.6)

– 0.11

partner present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at

109

40.6

(49.8)

72

36.0

(47.9)

4.6

(9.0)

0.08

least one child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent

55

32.8

(45.8)

35

23.7

(45.8)

9.2

(15.3)

0.20

at RA (%)

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

10

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-15. SUB+PBTH Versus CBRR: Impacts on Adult Well-Being at 37 Months

SUB+PBTH

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

546

31.1

(46.3)

381

32.8

(46.6)

– 1.7

(3.1)

– 0.03

Adult mental health

Goal-oriented thinkingb

540

4.43

(1.07)

379

4.47

(0.98)

– 0.04

(0.07)

– 0.03

Psychological distressc

544

6.87

(5.74)

381

6.67

(5.64)

0.20

(0.38)

0.03

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past

540

22.0

(41.6)

379

18.9

(38.9)

3.0

(2.8)

0.06

30 days (%)

Adult substance use

Alcohol dependence or drug abused (%)

545

12.5

(33.5)

381

9.4

(30.4)

3.1

(2.1)

0.08

Alcohol dependenced (%)

546

9.7

(30.1)

381

7.8

(28.2)

1.9

(1.9)

0.06

Drug abused (%)

545

4.5

(20.5)

381

2.7

(16.0)

1.8

(1.3)

0.07

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

545

7.7

(27.3)

382

8.8

(26.9)

– 1.1

(1.9)

– 0.03

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler 6 (K6) scale and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit F-16. SUB+PBTH Versus CBRR: Impacts on Child Well-Being Across Age Groups at 37 Months SUB+PBTH

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

629

2.0

(0.9)

444

2.1

(1.0)

– 0.1

**

(0.1)

– 0.12

Grade completion (not held back) (%)

515

90.48 (29.64)

370

92.16 (28.15)

– 1.68

(2.14)

– 0.04

School gradesc

466

3.1

(0.9)

347

3.0

(0.9)

0.1

(0.1)

0.05

Child physical health

Poor or fair health in past 30 days (%)

733

7.1

(23.8)

524

5.2

(22.5)

2.0

(1.8)

0.06

Well-child checkup in past year (%)

730

89.7

(30.0)

524

91.4

(28.9)

– 1.7

(2.2)

– 0.05

Child has regular source of health care (%)

730

89.9

(27.9)

525

94.5

(23.2)

– 4.5

**

(2.2)

– 0.12

Sleep problemsd

730

2.04

(1.05)

524

2.08

(1.03)

– 0.04

(0.07)

– 0.03

Child behavioral strengths and challenges

Behavior problemse

691

0.48

(1.24)

506

0.38

(1.20)

0.11

(0.09)

0.06

Prosocial behaviorf

692

– 0.21

(1.16)

507

– 0.18

(1.12)

– 0.04

(0.09)

– 0.02

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

11

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-17. SUB+PBTH Versus CBRR: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months

SUB+PBTH

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

235

35.1

(48.4)

168

34.3

(48.4)

0.8

(5.3)

0.01

Child care or preschool absences in past monthd

93

0.81

(0.88)

53

0.47

(0.87)

0.35 *

(0.19)

0.31

Positive child care or preschool experiencese

98

0.78

(0.44)

55

0.89

(0.36)

– 0.11

(0.09)

– 0.23

Positive child care or preschool attitudesf

97

4.64

(0.86)

55

4.49

(0.90)

0.15

(0.19)

0.17

Child care or preschool conduct problemsg (%)

103

10.5

(33.4)

59

6.7

(28.1)

3.8

(6.2)

0.14

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

205

64.9

(46.8)

149

72.9

(44.8)

– 8.0

(5.9)

– 0.14

Ages 3 years, 6 months to 7 years

Verbal abilityi

260

– 0.20

(1.00)

173

– 0.29

(1.02)

0.08

(0.12)

0.06

Math abilityj

258

– 0.28

(1.03)

173

– 0.38

(0.83)

0.10

(0.11)

0.09

Executive functioningk (self-regulation)

253

15.20 (15.77)

165

15.51

(16.35)

– 0.30

(1.28)

– 0.01

Ages 5 to 17 yearsl

School enrollmentc (%)

499

97.2

(14.7)

359

97.3

(16.5)

– 0.1

(1.7)

– 0.01

School absences in past monthd,m

219

0.95

(0.92)

141

0.76

(0.95)

0.19

(0.13)

0.15

Positive school experiencese,m

223

0.57

(0.59)

140

0.51

(0.58)

0.06

(0.08)

0.07

Positive school attitudesf,m

223

4.22

(1.05)

141

4.10

(1.13)

0.13

(0.12)

0.08

School conduct problemsg,m (%)

224

25.2

(43.1)

140

27.0

(44.6)

– 1.9

(5.2)

– 0.03

Ages 8 to 17 years

Anxietyn

310

35.38

(7.59)

230

35.18

(7.35)

0.20

(0.73)

0.02

Fearso

313

63.03 (14.49)

232

63.71

(14.52)

– 0.67

(1.24)

– 0.03

Substance usep (%)

306

4.44 (23.57)

226

5.94

(22.47)

– 1.50

(2.25)

– 0.04

Goal-oriented thinkingq

303

22.01

(5.08)

223

22.91

(4.73)

– 0.90 *

(0.52)

– 0.14

School effort in past monthr

312

2.78

(0.79)

228

2.71

(0.81)

0.07

(0.08)

0.07

Arrests or police involvement in past 6 monthss (%)

174

11.46 (30.54)

130

8.84

(29.06)

2.63

(4.14)

0.07

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Sources: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

12

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-18. SUB+PBTH Versus CBRR: Impacts on Self-Sufficiency at 37 Months

SUB+PBTH

CBRR

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Employment status

Work for pay in week before survey (%)

546

37.2

(48.1)

382

37.7

(48.9)

– 0.5

(3.3)

– 0.01

Any work for pay since 20-month surveyb (%)

505

58.7

(49.3)

358

62.7

(48.3)

– 3.9

(3.3)

– 0.08

Months worked for pay since 20-month surveyb,c

502

7.1

(8.2)

358

7.3

(8.1)

– 0.2

(0.5)

– 0.02

Any work for pay since RA (%)

546

69.7

(46.0)

382

74.5

(42.2)

– 4.9 *

(2.9)

– 0.09

Months worked for pay since RAc

540

13.1

(13.9)

379

14.1

(13.6)

– 1.0

(0.8)

– 0.07

Hours of work per week at current main jobd

546

11.5

(16.6)

381

12.4

(17.2)

– 0.9

(1.2)

– 0.05

Income sources and amounts

Annualized current earnings ($)

539

6,526 (10,676)

373

6,698 (11,127)

– 171

(743)

– 0.01

Total family income ($)

522

11,885 (10,057)

364

11,772

(9,436)

113

(694)

0.01

Anyone in family had earnings in past month (%)

546

49.7

(50.0)

382

48.1

(50.1)

1.5

(3.4)

0.03

Anyone in family received TANF in past month (%)

546

26.2

(44.2)

382

28.9

(44.8)

– 2.7

(3.1)

– 0.05

Anyone in family received SSDI in past month (%)

544

7.8

(27.3)

382

11.4

(28.9)

– 3.6 *

(2.1)

– 0.11

Anyone in family received SSI in past month (%)

546

13.4

(34.5)

382

14.1

(34.9)

– 0.7

(2.2)

– 0.02

Anyone in family received SNAP/Food Stamps in past month (%)

546

81.7

(39.3)

382

81.8

(37.2)

– 0.1

(2.7)

0.00

Anyone in family received WIC in past month (%)

546

26.5

(44.2)

382

27.7

(45.5)

– 1.3

(2.9)

– 0.03

Education and training

Participated in 2 weeks or more weeks of any school or training

544

37.4

(48.6)

382

40.7

(49.2)

– 3.3

(3.3)

– 0.06

since RA (%)

Number of weeks in school/training programs since RA

540

6.6

(13.2)

373

7.9

(17.0)

– 1.3

(1.1)

– 0.07

Participated in 2 weeks or more weeks of school since RA (%)

544

12.1

(32.5)

381

8.8

(27.4)

3.2

(2.0)

0.09

Participated in 2 weeks or more weeks of basic education since RA (%)

544

3.1

(16.9)

381

3.7

(20.1)

– 0.7

(1.3)

– 0.04

Participated in 2 weeks or more weeks of vocational education

544

10.3

(30.9)

381

11.1

(32.0)

– 0.8

(2.1)

– 0.02

since RA (%)

Food security and hunger

Household is food insecure (%)

546

42.1

(49.5)

382

40.0

(48.9)

2.1

(3.5)

0.04

Food insecurity scalee

542

1.69

(2.04)

382

1.66

(2.02)

0.03

(0.14)

0.01

Economic stressors

Economic stress scalef

543

– 0.20

(0.49)

379

– 0.17

(0.49)

– 0.03

(0.03)

– 0.05

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability

Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

13

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-19. CBRR+PBTH Versus SUB: Impacts on Housing Stability at 37 Months

CBRR+PBTH

SUB

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Homelessness or doubled up during the follow up period

At least 1 night homelessb or doubled up in past 6 months or in

490

38.6

(48.8)

462

17.4

(37.5)

21.2 ***

(2.9)

0.38

shelter in past 12 months (%)

At least 1 night homelessb or doubled up in past 6 months

490

34.6

(47.7)

462

15.8

(36.1)

18.8 ***

(2.9)

0.34

(%)

At least 1 night homelessb in past 6 months (%)

491

15.7

(36.8)

462

9.7

(28.8)

6.0 ***

(2.2)

0.13

At least 1 night doubled up in past 6 months (%)

490

28.3

(45.2)

463

11.0

(31.3)

17.3 ***

(2.6)

0.33

Any stay in emergency shelter in past 6 months (%)

491

5.7

(24.0)

463

3.9

(17.7)

1.8

(1.5)

0.06

[Program Usage Data]

Any stay in emergency shelter in months 21 to 32 after RA (%)

491

13.9

(34.6)

463

5.4

(21.8)

8.5 ***

(2.0)

0.20

[Program Usage Data]

Number of days homelessb or doubled up in past 6 months

489

46.1

(72.1)

461

18.1

(49.4)

27.9 ***

(4.2)

0.32

Number of days homelessb in past 6 months

491

13.7

(41.8)

462

9.5

(34.6)

4.2

*

(2.5)

0.07

Number of days doubled up in past 6 months

489

34.9

(64.7)

462

10.5

(39.3)

24.4 ***

(3.8)

0.33

Housing independence

Living in own house or apartment at followup (%)

491

68.8

(46.8)

463

83.8

(36.5)

– 15.1 ***

(2.9)

– 0.28

Living in own house or apartment with no housing assistance (%)

488

38.5

(48.5)

462

16.1

(36.1)

22.4 ***

(2.9)

0.39

Living in own house or apartment with housing assistance (%)

488

30.5

(46.0)

462

67.6

(46.4)

– 37.0 ***

(3.1)

– 0.72

Number of places lived

Number of places lived in past 6 monthsc

489

1.6

(1.0)

463

1.4

(0.9)

0.2 ***

(0.1)

0.17

Housing quality

Persons per room

465

1.6

(1.2)

442

1.2

(0.6)

0.4 ***

(0.1)

0.29

Housing quality is poor or fair (%)

464

32.0

(46.0)

441

27.6

(44.7)

4.4

(3.2)

0.08

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

c The number of places lived in past 6 months is topcoded at 6 places.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

Exhibit F-20. CBRR+PBTH Versus SUB: Impacts on Family Preservation at 37 Months

CBRR+PBTH

SUB

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Current or recent separations of family members present at baseline

Family has at least one child separated in past 6 months (%)

482

18.3

(38.3)

449

14.8

(34.8)

3.5

(2.6)

0.08

Family has at least one foster care placement in past 6 monthsb (%)

483

4.0

(19.0)

454

3.4

(17.9)

0.6

(1.3)

0.03

Spouse/partner separated in past 6 months, of those with spouse/

133

47.6

(49.6)

119

39.5

(49.9)

8.1

(6.6)

0.14

partner present at RA (%)

Reunification of family members reported as separated at baseline

Family has at least one child reunified, of those families with at

84

32.3

(47.8)

92

44.9

(50.1)

– 12.6

(9.3)

– 0.22

least one child absent at RA (%)

Spouse/partner reunified, of those with spouse/partner absent

43

25.0

(44.1)

49

31.9

(45.6)

– 6.9

(12.8)

– 0.15

at RA (%)

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Foster care placement outcome includes any children (present at baseline) who are placed in foster care or adopted by another family at the time of followup.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

14

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-21. CBRR+PBTH Versus SUB: Impacts on Adult Well-Being at 37 Monthss

CBRR+PBTH

SUB

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Adult physical health

Health in past 30 days was poor or fair (%)

488

30.6

(46.0)

462

30.9

(46.4)

– 0.3

(2.9)

– 0.01

Adult mental health

Goal-oriented thinkingb

482

4.47

(1.03)

457

4.48

(1.05)

– 0.01

(0.07)

– 0.01

Psychological distressc

489

6.93

(5.62)

461

6.59

(5.56)

0.33

(0.36)

0.05

Adult trauma symptoms

Post-traumatic stress disorder (PTSD) symptoms in past

486

20.1

(39.5)

458

22.3

(41.5)

– 2.2

(2.7)

– 0.04

30 days (%)

Adult substance use

Alcohol dependence or drug abused (%)

490

11.6

(32.3)

462

13.3

(32.7)

– 1.6

(2.3)

– 0.04

Alcohol dependenced (%)

490

8.8

(28.9)

463

10.1

(28.4)

– 1.2

(2.1)

– 0.04

Drug abused (%)

490

4.1

(19.8)

462

4.4

(19.9)

– 0.3

(1.4)

– 0.01

Experience of intimate partner violence

Experienced intimate partner violence in past 6 months (%)

490

8.8

(27.7)

460

8.2

(26.5)

0.7

(1.9)

0.02

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Goal-oriented thinking is measured with a modified version of the State Hope Scale and ranges from 1 to 6, with higher scores indicating higher levels of positive, goal-oriented thinking.

c Psychological distress is measured with the Kessler 6 (K6) scale and ranges from 0 to 24, with higher scores indicating greater distress.

d Alcohol dependence is measured with the Rapid Alcohol Problems Screen (RAPS-4), and drug abuse is measured with six items from the Drug Abuse Screening Test (DAST-10).

Both are measured for the 6 months before the 37-month survey.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

Exhibit F-22. CBRR+PBTH Versus SUB: Impacts on Child Well-Being Across Age Groups at 37 Months CBRR+PBTH

SUB

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Child education

Number of schools attended since RAb

554

2.1

(1.0)

533

1.9

(0.9)

0.2 ***

(0.1)

0.19

Grade completion (not held back) (%)

460

89.67 (30.60)

439

92.88 (26.76)

– 3.20

(2.08)

– 0.08

School gradesc

423

3.0

(0.9)

403

3.1

(0.9)

– 0.1

(0.1)

– 0.10

Child physical health

Poor or fair health in past 30 days (%)

659

7.0

(24.2)

620

7.5

(25.2)

– 0.5

(1.8)

– 0.02

Well-child checkup in past year (%)

658

90.7

(29.4)

619

89.3

(30.7)

1.4

(2.0)

0.04

Child has regular source of health care (%)

656

92.1

(26.1)

619

91.0

(26.5)

1.1

(2.2)

0.03

Sleep problemsd

659

2.10

(1.06)

620

2.00

(1.04)

0.10

(0.07)

0.07

Child behavioral strengths and challenges

Behavior problemse

628

0.49

(1.22)

582

0.48

(1.27)

0.02

(0.09)

0.01

Prosocial behaviorf

629

– 0.23

(1.19)

584

– 0.19

(1.12)

– 0.04

(0.08)

– 0.03

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Number of schools outcome is topcoded at 4 or more schools.

c School grades outcome is defined as 1 = mostly Ds or Fs, 2 = mostly Cs, 3 = mostly Bs, 4 = mostly As.

d Sleep problems outcome ranges from 1 to 5, with higher values indicating more frequent tiredness upon waking and during the day.

e Behavior problems outcome is measured as the standardized Total Difficulties score from the Strengths and Difficulties Questionnaire (SDQ).

f Prosocial behavior is measured as the standardized prosocial domain score from the SDQ.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey (parent report)

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

15

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-23. CBRR+PBTH Versus SUB: Impacts on Child Well-Being Developmental Outcomes by Age Group at 37 Months

CBRR+PBTH

SUB

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

Ages 2 to 5 yearsb

Preschool or Head Start enrollmentc (%)

211

31.6

(47.7)

199

35.4

(48.9)

– 3.8

(4.9)

– 0.06

Child care or preschool absences in past monthd

71

0.57

(0.83)

80

0.86

(0.94)

– 0.29

*

(0.16)

– 0.26

Positive child care or preschool experiencese

76

0.79

(0.48)

83

0.85

(0.38)

– 0.06

(0.08)

– 0.13

Positive child care or preschool attitudesf

75

4.53

(0.94)

84

4.73

(0.69)

– 0.20

(0.15)

– 0.24

Child care or preschool conduct problemsg (%)

81

9.3

(33.1)

87

9.8

(30.6)

– 0.5

(5.9)

– 0.02

Ages 2 years to 5 years, 6 months

Met developmental milestonesh (%)

186

72.1

(44.7)

173

68.3

(44.6)

3.8

(5.3)

0.07

Ages 3 years, 6 months to 7 years

Verbal abilityi

223

– 0.41

(1.07)

229

– 0.18

(1.03)

– 0.24

*

(0.12)

– 0.18

Math abilityj

222

– 0.37

(0.96)

228

– 0.24

(1.01)

– 0.13

(0.11)

– 0.11

Executive functioningk (self-regulation)

208

15.32 (16.02)

223

15.54 (16.06)

– 0.22

(1.30)

– 0.01

Ages 5 to 17 yearsl

School enrollmentc (%)

452

96.9

(17.3)

424

98.6

(11.8)

– 1.7

(1.2)

– 0.09

School absences in past monthd,m

173

0.87

(0.94)

195

0.93

(0.91)

– 0.06

(0.13)

– 0.05

Positive school experiencese,m

173

0.49

(0.61)

196

0.58

(0.58)

– 0.09

(0.07)

– 0.11

Positive school attitudesf,m

173

4.09

(1.14)

195

4.25

(1.09)

– 0.17

(0.13)

– 0.11

School conduct problemsg,m (%)

172

24.0

(44.4)

197

23.8

(41.1)

0.2

(5.2)

0.00

Ages 8 to 17 years

Anxietyn

281

34.92

(7.75)

259

35.18

(7.53)

– 0.25

(0.76)

– 0.03

Fearso

284

62.58 (14.69)

260

62.94 (14.48)

– 0.36

(1.39)

– 0.02

Substance usep (%)

272

9.83 (30.44)

257

2.94 (20.28)

6.89 *** (2.17)

0.19

Goal-oriented thinkingq

268

23.19

(4.90)

252

21.67

(5.11)

1.52 *** (0.55)

0.24

School effort in past monthr

279

2.78

(0.84)

258

2.76

(0.77)

0.02

(0.08)

0.02

Arrests or police involvement in past 6 monthss (%)

142

9.71 (29.92)

142

9.93 (27.91)

– 0.22

(4.02)

– 0.01

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes focal children who were ages 4 years or younger on the September 1 before the 37-month parent survey.

c Preschool or Head Start enrollment outcome is defined as enrollment in preschool, center-based child care, or school.

d Absences outcome is defined as 0 = no absences in past month, 1 = one to two absences, 2 = three to five absences, 3 = six or more absences.

e Positive child care, preschool, or school experiences outcome is defined as – 1 = mostly negative experiences, 0 = both positive and negative experiences, 1 = mostly positive experiences.

f Positive child care, preschool, or school attitudes outcome is parent report of how much child likes school and ranges from 1 to 5, with higher values indicating greater like of school.

g Child care, preschool, or school conduct problems outcome is defined as 0 = no conduct problems reported to parent, 1 = parent contacted about conduct problems or suspension or expulsion from school or child care center.

h Met developmental milestones outcome is defined as scoring above the typical development cutoffs in all domains of the Ages and Stages Questionnaire (ASQ-3).

i Verbal ability outcome is the nationally standardized score from the Woodcock-Johnson III (WJ III) letter-word identification test.

j Math ability outcome is the nationally standardized score from the WJ III applied problems test.

k Executive functioning outcome is the Head Toes Knees Shoulders (HTKS) score and ranges from 0 to 40, with higher scores indicating greater executive functioning.

l Includes focal children who were ages 5 to 17 years on the September 1 before the 37-month parent survey and no older than 17 years at the time of the survey.

m This parent-reported outcome was collected from only the first 38 percent of parents surveyed due to an error in data collection.

n Anxiety (child report) is measured using the A-Trait scale from the State-Trait Anxiety Inventory for Children (STAIC). Scores range from 20 to 60, with higher scores indicating greater anxiety.

o Fears outcome (child report) is the score from the Fears Scale and ranges from 33 to 99, with higher scores indicating more fear.

p Substance use (child report) is measured with 23 items from the Centers for Disease Control and Prevention 2011 Youth Risk Behavior Survey.

q Goal-oriented thinking (child report) is measured with a modified version of the Children’s Hope Scale and ranges from 6 to 30, with higher scores indicating higher levels of positive, goal-oriented thinking.

r School effort outcome (child report) ranges from 1 to 4, with higher scores indicating greater effort during school day and on homework.

s Arrest or police involvement in past 6 months is from parent report.

Sources: Family Options Study 37-month followup survey (parent report); Family Options Study 37-month child survey (child report); ASQ-3; WJ III; HTKS

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

16

FAMILY OPTIONS STUDY

Appendix F. Impact Estimates for Pooled Comparisons

Exhibit F-24. CBRR+PBTH Versus SUB: Impacts on Self-Sufficiency at 37 Months

CBRR+PBTH

SUB

ITT Impact

Outcome

Effect

Sizea

N

Mean

(SD)

N

Mean

(SD)

Im-

pact

(SE)

Employment status

Work for pay in week before survey (%)

491

39.2

(48.9)

462

36.5

(48.0)

2.7

(3.2)

0.05

Any work for pay since 20-month surveyb (%)

445

60.8

(48.7)

433

58.8

(49.4)

2.0

(3.2)

0.04

Months worked for pay since 20-month surveyb,c

444

7.4

(8.2)

431

6.8

(7.9)

0.6

(0.5)

0.07

Any work for pay since RA (%)

491

73.8

(42.9)

462

69.9

(46.3)

3.9

(2.7)

0.08

Months worked for pay since RAc

487

14.1

(13.6)

456

12.2

(13.5)

1.9

**

(0.8)

0.12

Hours of work per week at current main jobd

489

13.1

(17.8)

461

11.4

(16.5)

1.7

(1.1)

0.09

Income sources and amounts

Annualized current earnings ($)

483

7,511 (12,048)

454

6,355 (10,376)

1,156

(747)

0.09

Total family income ($)

476

12,632 (10,696)

434

11,430

(9,535)

1,202

*

(666)

0.09

Anyone in family had earnings in past month (%)

491

50.8

(50.0)

463

46.9

(49.9)

3.9

(3.3)

0.07

Anyone in family received TANF in past month (%)

490

28.2

(45.2)

463

28.4

(45.3)

– 0.2

(2.9)

0.00

Anyone in family received SSDI in past month (%)

491

9.1

(26.7)

460

7.0

(26.5)

2.1

(1.9)

0.06

Anyone in family received SSI in past month (%)

490

15.5

(35.4)

463

13.8

(35.0)

1.7

(2.1)

0.04

Anyone in family received SNAP/Food Stamps in past month (%)

490

81.1

(38.8)

463

80.7

(38.9)

0.5

(2.7)

0.01

Anyone in family received WIC in past month (%)

490

28.8

(45.6)

463

25.8

(43.6)

3.0

(2.9)

0.06

Education and training

Participated in 2 weeks or more weeks of any school or training

489

38.1

(48.7)

461

38.5

(48.9)

– 0.4

(3.3)

– 0.01

since RA (%)

Number of weeks in school/training programs since RA

481

7.0

(15.3)

458

7.6

(15.0)

– 0.6

(1.0)

– 0.03

Participated in 2 weeks or more weeks of school since RA (%)

489

9.3

(28.9)

461

13.4

(33.7)

– 4.1

*

(2.2)

– 0.11

Participated in 2 weeks or more weeks of basic education since RA (%)

489

3.3

(17.8)

461

2.3

(15.9)

0.9

(1.2)

0.05

Participated in 2 weeks or more weeks of vocational education

489

12.2

(33.1)

461

10.5

(31.4)

1.7

(2.1)

0.04

since RA (%)

Food security and hunger

Household is food insecure (%)

491

44.5

(49.5)

463

37.3

(48.7)

7.2

**

(3.3)

0.12

Food insecurity scalee

488

1.89

(2.04)

461

1.52

(1.99)

0.37 ***

(0.13)

0.15

Economic stressors

Economic stress scalef

486

– 0.17

(0.49)

460

– 0.24

(0.45)

0.07

**

(0.03)

0.12

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. RA = random assignment. SD = standard deviation. SE = standard error. SNAP = Supplemental Nutrition Assistance Program. SSDI = Social Security Disability

Insurance. SSI = Supplemental Security Income. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

a Effect size column shows standardized effect sizes, which were calculated by dividing impact by standard deviation for the entire UC group.

b Includes only families who responded to both 20-month and 37-month followup surveys; not weighted for survey nonresponse.

c Number of months worked for pay includes partial calendar months.

d Hours of work per week includes those not currently working (that is, those with 0 hours of work per week).

e Food insecurity scale ranges from 0 to 6, with higher values indicating higher food insecurity.

f Economic stress scale ranges from – 1 to 1, with higher values indicating higher economic stress.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics and are weighted to adjust for survey nonresponse. See Chapter 2 and Appendix B

for outcome definitions.

Source: Family Options Study 37-month followup survey

F

3-Year Impacts of Housing and Services Interventions for Homeless Families

17

FAMILY OPTIONS STUDY

APPENDIX G.

INTERVENTION COSTS—

METHODOLOGY, SITES,

AND PROGRAMS

This appendix provides greater detail on the cost analysis We attempted to collect cost data from all programs in which at methodology and the sites and programs in the cost

least one study family enrolled and either (1) at least five study

analysis. The study team calculated costs using prima-

families were referred to the program or (2) the number of fam-

ry data collected from transitional housing, rapid re-housing,

ilies referred was among the top three for the program type in

and emergency shelter programs. The study team sought to

the site. As such, the estimates were more likely to include larger

capture costs of all program inputs consistently across each

programs that were able to make spaces available to study fam-

program type. The study team used administrative data to esti-

ilies and that study families were willing to accept. Collectively,

mate costs for vouchers and public housing offered to families

the programs selected for the cost analysis represented more

assigned to the permanent housing subsidy (SUB) intervention.

than 85 percent of study families who accepted a study referral

to CBRR and PBTH programs, and more than 90 percent of

The first four sections of this appendix describe the process

families assigned to usual care (UC) from emergency shelters.

of determining the per-family monthly program cost for each

program in the cost study, referred to as the program-level per-

Cost estimates are based on detailed reviews of 81 CBRR, PBTH,

family monthly program cost. Section G.1 reviews the programs

and emergency shelter programs providing housing or services

in the cost study. Section G.2 describes the elements of the

to homeless families across the 12 sites, plus administrative

per- family monthly program cost relevant to the transitional

data (both at the household and public housing agency [PHA]

housing, rapid re-housing, and emergency shelter programs.

level) covering the 10 sites providing permanent housing sub-

Section G.3 reviews the approach used to calculate the per-family

sidies offered through the SUB intervention. The data collected

monthly program cost for the SUB intervention, and section G.4

from these programs represent a valuable contribution to the

reviews the calculation for the other program types. Section G.5

understanding of the cost of providing services to homeless

reviews how the study team averaged these program-level per-

families using each of the interventions studied and emergency

family monthly program costs to derive the average per-family

shelter. Exhibit G-1 reports the number of programs in which

monthly program cost for each program type for the entire study

the study team collected cost data. Sites that offered the SUB

and for study sites. This section also reviews how the study

intervention are also indicated in Exhibit G-1.

team combined per-family monthly program costs with study

As described in The Family Options Study Interim Report, families

families’ observed program usage to calculate the other two types

were enrolled in the study from September 2010 through

of costs addressed in Chapter 9 of the report: (1) cost of all pro-

January 2012. Then, depending on which group a family was

gram use during the followup period, and (2) monthly cost of all

assigned to, families may have received housing and services

program use in the month of the 37- month followup survey.

from the program associated with their intervention for either

days or weeks (shelter), months (rapid re-housing and transi-

G.1. Programs in the Cost Study

tional housing), or years (transitional housing or permanent

subsidies). We designed the cost analysis to be representative

The study team collected cost data at a large subset of rapid

of the approximate timeframe when families were most likely

re-housing, transitional housing, and emergency shelter pro-

to be enrolled in the programs. The study team collected cost

grams that participated in the study in the 12 study sites. The

data for the full-year period that best aligned with the time

study team purposefully selected programs from each site in

period in which most families were referred to the program and

which a substantial number of study families had enrolled or,

that overlapped with the program’s financial recordkeeping.

in the case of emergency shelter, programs with a high number

Most often (48 programs) this period was fiscal year 2011. We

of families recruited for the study.

adjusted all costs using local consumer price index measures of

inflation so that all estimates are reported in 2013 dollars.

G

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix G. Intervention Costs—Methodology, Sites, and Programs

Exhibit G-1. Number of Programs From Which Cost Data

collected costs in five high-level cost categories: (1) overhead,

Were Collected and Presence of SUB Intervention by

(2) rental assistance, (3) facility operations, (4) supportive

Study Site

services, and (5) capital costs. Exhibit G-2 shows the cost cate-

Number of Numberof Number of

gories and how typical line items were sorted within categories.

Site

CBRR

PBTH

ES

SUB Site

Programs

Programs

Programs

The study team collected cost data for CBRR, PBTH, and

Alameda County

1

3

7

Yes

emergency shelter programs from the programs directly. The

Atlanta

1 *

1

2

No

study team reviewed these categories and all program activities

Baltimore

1

2

3

No

with key program operations and agency accounting staff using

Boston

1

—

5

Yes

Connecticut*

1

2

7

Yes

a standardized protocol and data collection tools that were

Denver

1

2

5

Yes

adapted from previous studies, including HUD’s Cost of Home-

Honolulu

1

5

5

Yes

lessness Study (Spellman et al., 2010). Our primary source of

Kansas City

1 *

3

3

Yes

cost information for all inputs except capital costs and in-kind

Louisville

1

1

2

Yes

Minneapolis

1

—

1

Yes

and partner costs was audited expense statements. These

Phoenix

1

4

4

Yes

statements were supplemented by program budgets, staffing

Salt Lake City

1

1

1

Yes

lists, partner commitment letters, and program staff estimates

Total

12

24

45

10

of labor and material costs of any services not reflected in

CBRR = community-based rapid re-housing. ES = emergency shelter. PBTH =

expense statements. To further ensure all program services

project-based transitional housing. SUB = permanent housing subsidy.

* This study site comprises four Continuums of Care in New Haven and Bridgeport,

were recognized as costs, interviews also reviewed program

Connecticut.

calendars and case management approaches for each age group

Notes: Pooled data from multiple CBRR programs were reported for Atlanta and Kan-

sas City. PBTH was not offered in Boston, and enrollment in PBTH in Minneapolis did

of program residents.

not support cost-data collection. SUB was not offered in Atlanta and Baltimore.

Source: Family Options Study cost data

To clarify and simplify comparisons across intervention pro-

grams for reporting our findings in Chapter 9, we collapsed the

five cost-data collection categories into two broad designations:

G.2. Elements of Program-Level

(1) housing or shelter and (2) supportive services.

per-Family Monthly Program Cost

1. Housing or shelter refers to the rental cost—either observed

The methodology was designed to collect comprehensive

or estimated—of the space used to provide housing or

program costs. Both to ensure that the study team collected all

shelter and program services and also any maintenance or

costs associated with proving program services and to allow

other facility operation costs (including durable items such

for analysis comparing program structure, the study team

as furnishings). This rental cost is net of any rent payments

made by the family.

Exhibit G-2. Cost Data Collection Categories and Associated Item Prompts

Supportive Services

Housing or Shelter

Facility Operating

Agency Overhead

Program/Assistance

Rental Assistance

Property Value or

Expenses

for Client Housing

Costs (project-based

programs)

Lease Expenses

Administrative staff

Supportive services staff

Lease of client housing

Housing operations staff

Market rate lease of client,

• Salaries

• Salaries

Electric

• Salaries

program, and administrative

• Fringe

• Fringe

Gas

• Fringe

property

• Other staff-related costs

• Other staff-related costs

Water/sewer

• Other

Estimate of rental rate of

Advertising

Staff transportation

Other

Trash removal

owned or donated property

Audit

Program supplies

Family contributions

Landscaping

Estimate of rental rate of

Accounting

Activities

Exterminating

owned or donated property-

Legal

Housing placement

Painting and decorating

Facility rent (or capital cost)

Management fee

Employment search

Property Insurance

for space used for support-

Leasing & utilities

Direct support

Real estate taxes

ive services

(prorated agency)

• Food

Repairs

Miscellaneous office

• Clothing

Supplies

expenses

• Furniture

Furnishing equipment

Payroll taxes

• Transportation

Other

Indirect or allocated costs

• Education

• Cash

• Other

G

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix G. Intervention Costs—Methodology, Sites, and Programs

2. Supportive services costs refer to any services other than

In-Kind and Partner Costs

shelter or housing provided as an integral part of the

The study team reviewed all services that were provided to fam-

program, including case management, and any cash or in-

ilies because they were enrolled in the program to ensure that

kind assistance (for example, meals provided in emergency

we were accounting for all inputs consistently across programs

shelters).

and across interventions, whether the assistance was provided

Additional detail is reported in Chapter 9 for two other

directly by the program or in kind by a partner or volunteer.

categories—(1) administrative and overhead costs and (2) in-kind

The analysis, however, does not include the costs of every

and partner costs—because they provide information on typical

social service accessible to or encountered by program

program structures.

participants. The study team did not gather costs for services

1. Administrative and overhead costs include management

provided or made available by virtue of a families’ housing or

salaries; legal, accounting, and other professional services;

socioeconomic status alone, regardless of their enrollment in

and program support costs, such as insurance premiums and

a particular program. Rather, costs were included for housing

agency and association fees. Administrative and overhead

or shelter, goods, and services provided to families explicitly

costs are divided among supportive services and housing

because they were enrolled in the study program. The analysis

and shelter costs according to the cost types’ relative share

included only the partner services and in-kind assistance that

of total costs so that they are included in the two broad

resulted from a dedicated relationship with the program and

categories.

were accessed by at least 20 percent of eligible families (as

determined by interviews with program staff). In addition, the

2. In-kind and partner costs include any costs of housing or

analysis included any services that programs cited as critical to

shelter or supportive services provided to families because

the mission or core approach, regardless of participation rates.

they participate in a program. These costs are not provided

Exhibit G-3 provides examples of services that our approach

by the program, and, as a result, are not included in program

did and did not include when identifying partner and in-kind

financial statements. Common examples include onsite

costs.

health or mental health providers funded by an outside

agency, community volunteers providing a variety of

When inputs were provided by an external partner or through

services, and consumer goods donated to program clients.

in kind donations, the study team estimated the value of the

The importance of these costs varies widely from program to

program input using the following resources, as available—

program. When present, they typically are part of the cost of

• Program documentation (such as an audit estimate of

supportive services provided by a program. In some cases,

in-kind services value).

however, housing or shelter costs include the costs of labor,

such as handyman services, or of facilities used regularly

• Costs of a similar service or item paid for by the program or

for program activities that were provided in kind. In other

by another program at the site.

cases, accounting, legal, or administrative services were

• External documentation, such as Bureau of Labor Statistics

provided in kind or by partners. In each case, the study team

data on local wages or publicly listed costs for the goods or

apportioned the cost to the appropriate category.

services.

The next section provides additional detail regarding the assess-

Examples of partner and in-kind services and resources used to

ment of in-kind and partner costs and capital costs and detail

determine an associated cost are listed in Exhibit G-4.

about our treatment of participant contributions.

Exhibit G-3. Determining Inclusion of External Services

Included—On Site

Not Included—On Site

A health clinic sends a doctor and a nurse practitioner to the program site to

A volunteer organization holds parenting classes in a common area,

conduct weekly screenings and checkups.

but less than 20 percent of parents attend.

Included—Off Site

Not Included—Off Site

A Head Start school reserves and guarantees priority slots for all appropriately

If a client expresses interest, case managers refer her to an external

aged children from the program.

job training program run by the local workforce board.

G

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix G. Intervention Costs—Methodology, Sites, and Programs

Exhibit G-4. Valuing In-Kind Services

Example

Estimation Approach

Partnering organization sends substance-abuse counselor to program to hold Program has an equivalent social worker with a substance-abuse weekly meetings with residents.

specialization on staff. This hourly wage rate is applied to level of

effort by partner staff.

Health clinic sends team of nurse practitioners monthly to offer basic

Clinic sends leverage letter detailing the estimated costs of the nurses’

preventative health care and checkups.

time and medical supplies.

Local church contributes a move-in packet for each family, including furniture

Program estimates the value of each packet at $500 per client.

and kitchenware.

YMCA nearby holds five slots in its summer camp for children staying in

YMCA website lists costs for summer camp activities.

program’s shelter.

Area nonprofit organization sends a dentist to conduct exams for all parents

Bureau of Labor Statistics’ median wage for a dentist in that state is

and children once per year.

applied to the level of effort by the dentist.

YMCA = Young Men’s Christian Association.

The approach used to valuing partner and in-kind staff is

• Program’s or donor’s stated annual value of the occupied space.

conservative in that it applies estimated labor rates to partner

and in-kind level of effort, but it does not apply an overhead

• A 5-percent annual cost of capital applied to a total property

cost multiplier to that rate. The logic of this approach is an

value estimate; for example, a property value estimate from a

assumption that the program’s existing overhead infrastructure

recent appraisal or sales price-based insurance estimate.

could absorb any additional administrative cost associated with

• HUD’s published Fair Market Rent (FMR) for a comparable

directly funding this partner or in-kind service were the service

number of units as the space used by the program, adjusted

to be provided by the program.

for maintenance costs.

Capital Costs

• Readily available estimates of market rent for similar nearby

properties, adjusted for maintenance costs; for example,

Accounting for capital costs associated with the physical space

similar units or properties listed publicly for rent next door

used to provide client housing and program services is critical

or reasonable rental estimates from similar neighboring

to establishing comparable costs within and across programs.

properties provided by Zillow.

For some programs this accounting was fairly straightforward.

A few programs rent client housing units, program services,

When rental value was imputed, either using comparable

space, or administrative office space at market rates, providing

properties or local FMR, an estimate of net rent—the opportu-

direct market-rate estimates of the cost of facilities used by

nity cost of interest—was imputed as 55 percent of the gross

the program. At other programs, where client client-housing

rent. This deflation was based on an analysis of data from the

facilities are unique and different from typical housing stock,

Residential Housing Finance Survey (RHFS; HUD, 2014b). Our

estimating a cost of ongoing occupancy of the space used for the

analysis of the RHFS indicated an average 45-percent expense

program was more nuanced. For example, a congregate shelter

ratio for market-rate rental properties with between 5 and 49

in an otherwise industrial neighborhood or in the basement of

units—the relevant size properties for the size of buildings used

a downtown historic religious sanctuary is not comparable with

by programs in our analysis.

any space that would be rented at a market rate. Other space is

provided at an in-kind discount; for example, office space owned

Participant Contributions

by a municipality is leased at effectively no cost to the program.

Of the 45 shelters in the cost analysis, 8 required shelter partic-

For cases in which market rates were not paid by the organiza-

ipants to pay some set amount or percent of income as rent. Of

tion directly, the study team took an opportunity-cost approach

the 24 transitional housing programs, 21 required rental pay-

to valuing space used to provide housing and services. We

ments.2 This analysis reports program costs net of these con-

specifically relied on the following resources to derive annual

tributions. Annual program costs were reduced specifically by

capital cost estimates, as available1—

the amount of any participant contributions before determining

1 In developing this approach, we used multiple methods for facilities where data were available. Estimates for the same property resulting from different methods were remarkably similar (typically within 5 to 10 percent), which gives us confidence that a particular choice of property valuation method does not introduce material variation into our cost estimates.

2 A few programs return mandatory participant savings as cash on their exit. Because funds dispersed are exactly offset by the participant contributions, we do not include these transactions as program costs.

G

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix G. Intervention Costs—Methodology, Sites, and Programs

per-family averages, which conceptually is consistent with

contain move-in dates and housing assistance payments (HAPs)

reporting the net costs to society of providing assistance.

made on behalf of the family. To determine site-level and over-

From a practical standpoint, this procedure allows for a closer

all average HAP, we averaged these household HAPs, weighting

comparison across program types. The actual family rental

households by the number of days of assistance received. These

payments for families receiving rapid re-housing or subsidy

site-level HAPs were then inflated to account for administrative

assistance were unknown (For SUB, we know households’ ex-

costs. Annual PHA financial reports provided information on

pected rent contribution, but we do not observe actual payment

the total administrative costs for the voucher program in each

to landlords.) Reducing costs by tenant rent for project-based

of the 18 PHAs that participated in the Family Options Study.3

transitional housing and emergency shelter programs makes the

resulting estimates more comparable with rapid re-housing and

G.4. Calculating Costs

permanent subsidy costs. As such, costs reported in Chapter 9

For CBRR, PBTH, and emergency shelter programs, the study

were net of any participant contributions.

team collected or calculated costs for the entire program,

The pattern of participant contributions was similar across sites,

typically for a fiscal year. Translating this total program cost

with the exception of project-based transitional housing in

into a per-family unit cost required determining the number

Hawaii, where family contributions averaged slightly less than

of families assisted at a time for PBTH and emergency shelter

$480 per family per month across the five programs. Among

programs, and, for CBRR programs, the average number of

the 8 remaining sites (16 programs) in which families in

months of assistance provided to each family. For PBTH and

project-based transitional housing programs pay some rent, the

emergency shelter programs,

average monthly per-family rent is $144. For the 8 emergency

shelters at which families pay some rent or program fee, the

average per-family per-month contribution is $105.

.

All costs were reported net of any family contributions.

The number of families served at a time is determined from

program reports of the number of units and occupancy rate for

G.3. Costs of SUB Intervention

unit-based facilities and the number of beds—typical family

size—and occupancy rate for congregate facilities.

The approach to collecting cost data for the SUB intervention

differed from that outlined previously for the CBRR and PBTH

For rapid re-housing programs, the study team divided total

programs and for emergency shelters. Because the SUB inter-

program costs by the total number of months of rent supported

vention offered priority access to permanent housing assistance

by the program for all families in the program during the peri-

provided by PHAs, mostly in the form of a housing choice

od for which costs were collected. In some cases, we received

voucher, administrative data sources available to HUD were an

program administrative data from which we calculated the

efficient source of cost information.

number of months of assistance directly. For other providers,

programs provided us total program expenditures, total

The cost of housing assistance was calculated directly from

number of households assisted, and average number of months

household-level administrative data for each study family

of housing provided by the assistance. Exhibit G-5 reviews the

assigned to the SUB group who received SUB assistance. These

cost calculation for each program type.

HUD Public and Indian Housing Information Center data

Exhibit G-5. Program Level Average per Family Cost Calculations

Program Type

Program Level Average per-Family Monthly Cost Calculation

Emergency shelter

((Total annual program cost)/12)/typical number of families assisted at a time

CBRR programs

(Total annual program costs)/total family-monthsa of rent subsidized with funds

PBTH programs

((Total annual program cost)/12)/typical number of families served at a time

Voucher program assistance offered to the

Average observed per family per month rental subsidy for sample families scaled by PHA administrative SUB group

cost rate

Public housing program assistance offered

Average per family per month costs with imputed rental value for sample families scaled by PHA to the SUB group

administrative cost rate

CBRR = community-based rapid re-housing. PBTH = project-based transitional housing. SUB = permanent housing subsidy.

PHA = public housing authority.

a “Family-month” means a rent is a subsidy for one family for 1 month.

3 For families in Honolulu who were assigned to SUB and received permanent subsidies from public housing programs, the study team imputed the cost of providing the public housing unit using the Honolulu FMR discounted to reflect typical maintenance costs together with average observed maintenance and actual administrative costs.

G

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

Appendix G. Intervention Costs—Methodology, Sites, and Programs

This per-family monthly program cost for each program in the

housing assistance that was not provided to families assigned to

cost study is used to calculate each of our four cost concepts;

the SUB intervention nor included in the cost analysis. Under

(1) average per-family monthly program cost, (2) program cost

the assumption that they have similar program and cost struc-

per stay during the followup period, (3) cost of all program use

tures, the estimates reported in this section use site-level PBTH

during the followup period, and (4) monthly program cost of

program costs as a proxy for the cost of PSH and SUB costs

all program use at the 18-month followup survey.

as a proxy for the costs of public housing and project-based

housing assistance.

Per-Family Monthly Program Cost

To calculate the average cost of all program use during the

Chapter 9 reports per-family monthly program cost averaged

followup period for the families on each side of each pairwise

across all programs included in the cost analysis. As noted

comparison, we first multiplied each family’s observed duration

previously, this group of programs is a subset of all programs in

in each program type with the site-level average per-family

the Family Options Study. One aim in selecting programs was

monthly program cost of providing that type of assistance. After

to produce cost estimates that reflected the assistance that study

summing all program types a family uses during the followup

families actually received. To be consistent with this aim—to cal-

period, we averaged this family-level cost of all program use

culate average per-family monthly program costs—we weighted

during the followup period over all families in each assignment

program-level per-family monthly program costs by the number

group, using the same nonresponse weights used in the impact

of study families who actually enrolled in the program after being

analysis.

referred to the program by the study for PBTH and CBRR pro-

grams. To be consistent with this approach for emergency shelter

Cost of All Program Use at the Time of the

programs, we weighted program-level costs by the number of

Followup Survey

families assigned to the UC group at each shelter in the cost

study. SUB costs were averaged directly from family-level data,

To calculate the cost of all program use at the time of the

weighted by the number of days a family received assistance.

followup survey, we average the site-level average per-family

monthly program cost for the program type in the site in which

Cost of All Program Use During the

a family was receiving assistance (if any) at the time of the

Followup Period

37-month followup survey. As with the per-family monthly

cost measure, site-level average per-family monthly program

The per-family monthly program cost is calculated from

costs are calculated with weights for the number of study fam-

program-level data points. By contrast, the study team esti-

ilies who accepted an assignment to the programs. Associating

mated the cost of all program use during the followup period

site-level program type costs with assistance study families are

from family-level data points multiplied by site-level per-family

receiving at the time of the followup survey requires the same

monthly program cost estimates. This approach is used because

two assumptions described previously for cost of all program

average costs of all program use during the followup period

use during the followup period. First, site-level per-family

are calculated for each of the six pairwise comparisons in the

monthly program costs for each of our four program types are

study—a concept that is based on families assigned to interven-

used as cost estimates for a month of assistance at any program

tion rather than families referred to particular programs.

of that type. Second, the study data track families’ use of PSH,

public housing, and project-based housing assistance that was

This approach requires two additional assumptions. First,

not associated with the study or included in the cost analysis.

site-level per-family monthly program costs for each of our

four program types are used as cost estimates for a month of

We calculate the cost of all program use at the time of the

assistance at any program of that type.4 For example, all tran-

followup survey by averaging these site-level cost estimates

sitional housing programs in a site have the same per-family

that we associate with the assistance families are receiving.

monthly program cost as the site-level average PBTH program

These averages are calculated over families on each side of each

cost estimated using program-level costs reported in Chapter

impact comparison using the same survey nonresponse used in

9. Second, the study data track families’ use of permanent

the impact analysis.

supportive housing (PSH), public housing, and project-based

4 Atlanta and Baltimore did not offer the SUB intervention and per-family monthly program costs were not calculated in these sites for SUB programs. An additional site (Minneapolis) did not have adequate takeup of PBTH programs to support cost data collection. In these sites, the study team uses study-level average per-family monthly program costs as a proxy to allow for cost of all program use since random assignment estimates to include the families who found their way to these program types without study assistance.

G

3-Year Impacts of Housing and Services Interventions for Homeless Families

6

FAMILY OPTIONS STUDY

APPENDIX H.

SUPPLEMENTAL ANALYSES

This appendix provides exhibits from supplemental Exhibits H-7, H-8, and H-9 provide updates to three nonex-analyses conducted for the Family Options Study.

perimental analyses presented in the Short-Term Impacts report .

Exhibits H-1 through H-6 show information about the

Exhibits 6-6, 7-7, and 8-10 in the Short-Term Impacts report showed

use of homeless and housing assistance programs in the latter

comparisons of outcomes for families assigned to priority access

part of the 37-month followup period, from months 21 to 32

to (1) permanent housing subsidy (SUB), (2) community-based

after the month of random assignment. The analysis of program

rapid re-housing (CBRR), and (3) project-based transitional

use presented in Chapters 2 through 6 examined the extent to

housing (PBTH) who did and did not use the programs to

which families in each of the policy comparisons used seven

which they received priority access. The study team also tabu-

types of homeless and housing assistance programs at any time

lated 37-month outcomes for these groups of families as shown

during the entire 37-month followup period. For completeness,

in Exhibits H-7, H-8, and H-9. Neither the differences nor

and to supplement the program use analysis reported in

the similarities between the groups’ outcomes can be causally

Family Options Study: Short-Term Impacts of Housing and Services

attributed to the use of the SUB, CBRR, or PBTH programs

Interventions for Homeless Families for the first 20 months after

because use of the programs was not randomly assigned.

random assignment, the study team also tabulated program use

for months 21 to 32 after random assignment.

Exhibit H-1. SUB Versus UC: Program Use From 19th Month After RA Until 37-Month Survey Response Month Percent Ever Used

From 19th Month

Number of Months Used From 19th Month

Percent Used in

After RA to 37-Month

After RA to 37-Month Followup Survey, if

Month of Followup

Type of Housing Assistance

Followup Survey

Ever Used Type of Housing Assistance

Survey Response

a

SUB

UC

SUB

UC

SUB

UC

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

78.5

12.0

17.9

19.0

15.7

18.0

68.4

10.8

SUB groupb

Rapid re-housing (CBRR)

1.3

6.1

3.7

3.5

5.6

4.5

0.3

1.8

Transitional housing

3.8

17.7

7.9

8.0

8.2

7.5

0.9

3.7

Permanent supportive housing

2.8

8.5

10.1

9.5

12.1

12.5

2.3

6.9

Public housing

1.5

9.8

17.9

18.0

13.4

16.0

1.4

7.6

Project-based vouchers/Section 8 projects

1.3

5.9

9.3

14.5

13.7

15.5

0.8

5.4

Any form of permanent housing subsidy c

83.9

34.5

17.6

18.0

14.6

17.0

73.0

30.5

Emergency shelterd

6.4

22.8

3.6

2.4

3.5

2.4

1.7

5.4

No use of homeless or housing programse

11.2

42.1

—

—

—

—

24.1

59.1

N

501

395

—

—

—

—

501

395

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the 19th calendar month after the month of RA to the month of the 37-month followup survey response. Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at RA. Percentages less than 100 are because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

H

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

FAMILY OPTIONS STUDY

Appendix H. Supplemental Analyses

Exhibit H-2. CBRR Versus UC: Program Use From 19th Month After RA Until 37-Month Survey Response Month Percent Ever Used

From 19th Month

Number of Months Used From 19th Month

Percent Used in

After RA to 37-Month

After RA to 37-Month Followup Survey, if

Month of Followup

Type of Housing Assistance

Followup Survey

Ever Used Type of Housing Assistance

Survey Response

a

CBRR

UC

CBRR

UC

CBRR

UC

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

9.8

11.5

15.6

18.0

15.0

17.0

8.5

10.2

SUB groupb

Rapid re-housing (CBRR)

7.1

5.7

7.4

7.5

6.4

6.5

2.3

1.1

Transitional housing

12.9

15.5

7.7

6.5

8.2

6.5

4.2

3.3

Permanent supportive housing

8.9

9.9

12.8

11.5

13.3

14.5

6.7

8.0

Public housing

10.7

9.3

14.1

16.5

13.7

15.5

9.4

8.2

Project-based vouchers/Section 8 projects

5.4

5.7

14.4

17.0

12.1

14.0

5.3

4.8

Any form of permanent housing subsidy c

34.4

34.8

14.4

16.5

14.4

16.5

29.7

31.1

Emergency shelterd

20.9

22.8

3.2

2.0

3.4

2.2

2.6

5.2

No use of homeless or housing programse

44.0

42.6

—

—

—

—

61.3

59.9

N

434

434

—

—

—

—

434

434

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy. UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the 19th calendar month after the month of RA to the month of the 37-month followup survey response. Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at RA. Percentages less than 100 are because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

Exhibit H-3. PBTH Versus UC: Program Use From 19th Month After RA Until 37-Month Survey Response Month Percent Ever Used

From 19th Month

Number of Months Used From 19th Month

Percent Used in

After RA to 37-Month

After RA to 37-Month Followup Survey, if

Month of Followup

Type of Housing Assistance

Followup Survey

Ever Used Type of Housing Assistance

Survey Response

a

PBTH

UC

PBTH

UC

PBTH

UC

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

8.6

9.7

10.6

9.5

12.9

17.0

7.4

7.9

SUB groupb

Rapid re-housing (CBRR)

6.2

6.4

5.3

3.5

5.2

4.5

0.7

1.8

Transitional housing

25.8

19.1

9.6

8.5

7.9

6.5

9.3

6.2

Permanent supportive housing

10.1

9.6

13.3

14.0

14.4

17.0

7.6

8.3

Public housing

8.0

8.5

14.3

17.0

13.2

15.5

7.1

5.8

Project-based vouchers/Section 8 projects

5.3

6.1

13.1

12.5

13.2

13.5

3.9

5.4

Any form of permanent housing subsidy c

31.1

31.5

13.1

14.5

14.5

17.0

26.0

27.1

Emergency shelterd

16.2

18.9

2.0

1.1

3.5

2.2

3.5

4.3

No use of homeless or housing programse

43.4

44.0

—

—

—

—

61.5

61.5

N

293

259

—

—

—

—

293

259

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the 19th calendar month after the month of RA to the month of the 37-month followup survey response. Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at RA. Percentages less than 100 are because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

H

3-Year Impacts of Housing and Services Interventions for Homeless Families

2

FAMILY OPTIONS STUDY

Appendix H. Supplemental Analyses

Exhibit H-4. SUB Versus CBRR: Program Use From 19th Month After RA Until 37-Month Survey Response Month Percent Ever Used

From 19th Month

Number of Months Used From 19th Month

Percent Used in

After RA to 37-Month

After RA to 37-Month Followup Survey, if

Month of Followup

Type of Housing Assistance

Followup Survey

Ever Used Type of Housing Assistance

Survey Response

a

SUB

CBRR

SUB

CBRR

SUB

CBRR

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

77.4

9.6

18.3

19.0

16.6

18.0

67.4

8.4

SUB groupb

Rapid re-housing (CBRR)

1.9

7.4

3.8

5.0

7.9

8.0

0.3

2.6

Transitional housing

3.9

11.4

7.7

8.0

7.3

7.5

0.9

4.1

Permanent supportive housing

3.6

11.3

10.1

9.5

12.6

11.5

3.1

8.0

Public housing

1.1

10.9

16.8

15.0

15.7

17.0

1.2

10.0

Project-based vouchers/Section 8 projects

0.8

6.7

11.9

14.5

13.2

17.0

0.6

6.4

Any form of permanent housing subsidy c

83.0

37.9

17.9

19.0

14.8

17.0

72.3

32.6

Emergency shelterd

7.6

21.7

3.7

2.7

3.5

2.3

1.8

2.5

No use of homeless or housing programse

11.9

42.2

—

—

—

—

24.7

58.3

N

362

290

—

—

—

—

362

290

CBRR = priority access to community-based rapid re-housing. SUB = priority access to permanent housing subsidy.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the 19th calendar month after the month of RA to the month of the 37-month followup survey response. Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at RA. Percentages less than 100 are because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

Exhibit H-5. SUB Versus PBTH: Program Use From 19th Month After RA Until 37-Month Survey Response Month Percent Ever Used

From 19th Month

Number of Months Used From 19th Month

Percent Used in

After RA to 37-Month

After RA to 37-Month Followup Survey, if

Month of Followup

Type of Housing Assistance

Followup Survey

Ever Used Type of Housing Assistance

Survey Response

a

SUB

PBTH

SUB

PBTH

SUB

PBTH

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

76.1

7.1

18.1

19.0

10.6

8.5

65.8

6.5

SUB groupb

Rapid re-housing (CBRR)

0.5

6.7

3.5

3.5

5.5

2.5

0.4

0.9

Transitional housing

5.3

23.2

7.5

3.5

9.4

8.5

1.0

7.3

Permanent supportive housing

2.1

10.8

7.2

8.0

13.6

12.5

1.2

8.6

Public housing

1.6

7.8

18.5

20.5

14.5

15.5

1.5

6.9

Project-based vouchers/Section 8 projects

1.7

5.8

9.0

14.5

12.8

15.0

1.2

4.6

Any form of permanent housing subsidy c

81.2

30.1

17.7

19.0

13.6

15.5

69.7

26.7

Emergency shelterd

6.5

18.7

4.6

2.4

2.1

1.2

2.5

5.2

No use of homeless or housing programse

13.4

45.2

—

—

—

—

26.4

60.7

N

215

201

—

—

—

—

215

201

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the 19th calendar month after the month of RA to the month of the 37-month followup survey response. Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at RA. Percentages less than 100 are because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

H

3-Year Impacts of Housing and Services Interventions for Homeless Families

3

FAMILY OPTIONS STUDY

Appendix H. Supplemental Analyses

Exhibit H-6. CBRR Versus PBTH: Program Use From 19th Month After RA Until 37-Month Survey Response Month Percent Ever Used

From 19th Month

Number of Months Used From 19th Month

Percent Used in

After RA to 37-Month

After RA to 37-Month Followup Survey, if

Month of Followup

Type of Housing Assistance

Followup Survey

Ever Used Type of Housing Assistance

Survey Response

a

CBRR

PBTH

CBRR

PBTH

CBRR

PBTH

Mean

Median

Mean

Median

Permanent housing subsidies offered to the

6.8

7.0

17.4

19.0

12.7

15.0

6.8

6.2

SUB groupb

Rapid re-housing (CBRR)

6.9

5.6

6.6

6.5

6.6

4.0

2.7

0.4

Transitional housing

15.5

27.2

8.2

7.5

9.3

7.5

5.1

9.1

Permanent supportive housing

8.7

9.6

13.7

15.5

13.1

14.0

7.6

6.8

Public housing

11.5

9.1

13.7

16.5

14.6

18.0

10.1

8.2

Project-based vouchers/Section 8 projects

4.9

4.7

16.6

18.0

14.1

12.5

4.9

3.8

Any form of permanent housing subsidy c

31.8

30.4

14.9

17.0

13.6

14.5

29.3

25.1

Emergency shelterd

17.6

15.5

3.1

2.0

2.2

1.1

2.8

2.0

No use of homeless or housing programse

46.0

42.8

—

—

—

—

60.1

63.4

N

180

184

—

—

—

—

180

184

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

RA = random assignment.

a Percentage of families who ever used a type of assistance program during the period from the 19th calendar month after the month of RA to the month of the 37-month followup survey response. Percentages do not add to 100 because some families used more than one program type during the followup period.

b Subsidies offered to the SUB group are housing choice vouchers plus site-specific programs offered to families assigned to the SUB group in Bridgeport, Connecticut, and Honolulu, Hawaii.

c Any form of permanent housing subsidy includes the types of permanent subsidy offered to the SUB group, permanent supportive housing, public housing, and project-based vouchers/Section 8 projects.

d All families were in emergency shelter at RA. Percentages less than 100 are because of missing data on shelter use.

e Indicates no use of the six program types in this table during any of the followup period and no use of emergency shelter after the first 6 months after RA. No use in the month of followup survey response indicates no use of any of these seven program types.

Notes: Percentages are regression adjusted, controlling for site and randomization ratio. Percentages, means, and medians are weighted for survey nonresponse to represent full comparison sample.

Source: Family Options Study Program Usage Data

Exhibit H-7. Housing Stability Outcomes for the SUB RA Group by Use of the Offered Permanent Subsidy Families Assigned to SUB

Families Assigned to SUB

Who Never Used the Offered

Who Ever Used the Offered

Outcome

Permanent Subsidy

Permanent Subsidy

N = 82

N = 419

Homelessness or doubled up during the followup period

At least 1 night homelessa or doubled up in past 6 months or in shelter in past

40.7

11.2 †

12 months (%)

At least 1 night homelessa or doubled up in past 6 months (%)

38.3

10.0 †

At least 1 night homelessa in past 6 months (%)

24.7

5.3 †

At least 1 night doubled up in past 6 months (%)

26.8

7.4 †

Any stay in emergency shelter in past 6 months (%)

12.2

1.2 †

Any stay in emergency shelter in months 21 to 32 after RA (%)

20.7

1.4 †

Number of days homelessa or doubled up in past 6 months

45.6

11.9 †

Number of days homelessa in past 6 months

26.4

4.9 †

Number of days doubled up in past 6 months

27.9

7.5 †

Housing independence

Living in own house or apartment at followup (%)

72.0

87.4 †

Living in own house or apartment with no housing assistance (%)

46.3

7.9 †

Living in own house or apartment with housing assistance (%)

25.6

79.4 †

Number of places lived

Number of places lived in past 6 monthsb

1.8

1.2 †

Housing quality

Persons per room

1.4

1.2 †

Housing quality is poor or fair (%)

32.9

25.1

SUB = priority access to permanent housing subsidy.

RA = random assignment.

† Difference in means is statistically significant at 0.10 level.

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

b The number of places lived in past 6 months is topcoded at 6 places.

Notes: Outcome means are unweighted. See Chapter 2 and Appendix B for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

H

3-Year Impacts of Housing and Services Interventions for Homeless Families

4

FAMILY OPTIONS STUDY

Appendix H. Supplemental Analyses

Exhibit H-8. Housing Stability Outcomes for the CBRR RA Group by Use of Rapid Re-Housing Families Assigned to CBRR Who

Families Assigned to CBRR Who

Outcome

Never Used Rapid Re-housing

Ever Used Rapid Re-housing

N = 182

N = 252

Homelessness or doubled up during the followup period

At least 1 night homelessa or doubled up in past 6 months or in shelter

34.6

42.1

in past 12 months (%)

At least 1 night homelessa or doubled up in past 6 months (%)

31.9

36.9

At least 1 night homelessa in past 6 months (%)

14.8

16.7

At least 1 night doubled up in past 6 months (%)

26.9

31.7

Any stay in emergency shelter in past 6 months (%)

6.6

6.0

Any stay in emergency shelter in months 21 to 32 after RA (%)

13.2

17.1

Number of days homelessa or doubled up in past 6 months

44.3

51.3

Number of days homelessa in past 6 months

16.0

13.0

Number of days doubled up in past 6 months

32.9

40.2

Housing independence

Living in own house or apartment at followup (%)

62.1

72.2 †

Living in own house or apartment with no housing assistance (%)

27.5

46.0 †

Living in own house or apartment with housing assistance (%)

34.6

26.8 †

Number of places lived

Number of places lived in past 6 monthsb

1.5

1.6

Housing quality

Persons per room

1.7

1.6

Housing quality is poor or fair (%)

26.1

31.0

CBRR = priority access to community-based rapid re-housing.

RA = random assignment.

† Difference in means is statistically significant at 0.10 level.

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

b The number of places lived in past 6 months is topcoded at 6 places.

Notes: Outcome means are unweighted. See Chapter 2 and Appendix B for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

Exhibit H-9. Housing Stability Outcomes for the PBTH RA Group by Use of Transitional Housing Families Assigned to PBTH Who

Families Assigned to PBTH Who

Outcome

Never Used Transitional Housing

Ever Used Transitional Housing

N = 133

N = 160

Homelessness or doubled up during the followup period

At least 1 night homelessa or doubled up in past 6 months or in shelter

40.2

43.1

in past 12 months (%)

At least 1 night homelessa or doubled up in past 6 months (%)

35.6

41.9

At least 1 night homelessa in past 6 months (%)

16.5

22.5

At least 1 night doubled up in past 6 months (%)

30.3

29.4

Any stay in emergency shelter in past 6 months (%)

7.5

7.5

Any stay in emergency shelter in months 21 to 32 after RA (%)

9.8

9.4

Number of days homelessa or doubled up in past 6 months

46.0

50.1

Number of days homelessa in past 6 months

13.0

19.1

Number of days doubled up in past 6 months

35.4

33.1

Housing independence

Living in own house or apartment at followup (%)

72.9

60.0 †

Living in own house or apartment with no housing assistance (%)

50.8

36.6 †

Living in own house or apartment with housing assistance (%)

22.0

29.7

Number of places lived

Number of places lived in past 6 monthsb

1.5

1.7

Housing quality

Persons per room

1.8

1.7

Housing quality is poor or fair (%)

31.1

31.6

PBTH = priority access to project-based transitional housing.

RA = random assignment.

† Difference in means is statistically significant at 0.10 level.

a The definition of “homeless” in this report includes stays in emergency shelters and places not meant for human habitation. It excludes transitional housing.

b The number of places lived in past 6 months is topcoded at 6 places.

Notes: Outcome means are unweighted. See Chapter 2 and Appendix B for outcome definitions.

Sources: Family Options Study 37-month followup survey; Program Usage Data

H

3-Year Impacts of Housing and Services Interventions for Homeless Families

5

FAMILY OPTIONS STUDY

APPENDIX I.

LENGTH OF BASELINE STAY IN

EMERGENCY SHELTER

This appendix updates analysis presented in Family Op- wherein each dummy variable indicated a calendar month tions Study: Short-Term Impacts of Housing and Services

during which at least 1 night of shelter stay was observed for

Interventions for Homeless Families about the relative

a family.

impacts of the interventions on the length of time families re-

main in emergency shelter after random assignment. Exhibit I-1

3. The Program Usage Data were entirely recreated for the

presents this information for each of the six policy comparisons

37-month analysis in order to make use of updated extracts

(SUB versus UC, CBRR versus UC, PBTH versus UC, SUB versus

from the Homeless Management Information System

CBRR, SUB versus PBTH, and CBRR versus PBTH).

(HMIS), Public and Indian Housing Information Center,

and Tenant Rental Assistance Certification System and

The results presented in this appendix reflect three important

newly available data from the 27-month tracking survey and

revisions from the analysis conducted for the Short-Term

37-month followup survey. Updated data (particularly from

Impacts report.

HMIS) and revisions to data cleaning procedures may have

resulted in some changes to the Program Usage Data used to

1. The findings presented here are for the full sample of 2,282

measure length of baseline stay.

families. Updated Program Usage Data permit examination

of the length of baseline stay outcome for the full sample,

The Short-Term Impacts report presented results showing

whereas the Short-Term Impacts report included findings

that assignment to priority access to community-based rapid

only for the sample that responded to the 20-month survey

re-housing (CBRR) led to more rapid departures from emer-

(1,857 families). The updated findings are thus more robust,

gency shelter than usual care (UC) by about 2 weeks, but it did

because they account for the entire study sample.

not lead to more rapid departures than for families assigned

to priority access to permanent housing subsidy (SUB) or to

2. The outcome, length of baseline stay in emergency shelter,

project-based transitional housing (PBTH). Revised analysis

has been measured more precisely than was done in the

using updated Program Usage Data on length of emergency

Short-Term Impacts report. As described in Appendix B,

shelter stays shown in this appendix has resulted in changes to

the outcome analyzed is measured as the count of days in

the findings about length of initial shelter stay, however. For

emergency shelter converted to months and is based on exit

the full study sample, families assigned to CBRR left shelter

dates in the source data. In the previous analysis, measures

on average 1 week faster than families assigned to UC. This

were based on adjusted counts of monthly dummy variables,

difference is not statistically significant.

Exhibit I-1. Impacts on Length of Baseline Stay in Emergency Shelter

First Assignment Group

Second Assignment Group

ITT Impact

Comparison

N

Mean

(SD)

N

Mean

(SD)

Impact

(SE)

SUB vs. UC

466

2.54

(2.57)

430

3.12

4.27

– 0.58 ***

(0.20)

CBRR vs. UC

453

2.87

(4.31)

456

3.09

4.24

– 0.23

(0.24)

PBTH vs. UC

281

2.51

(3.35)

272

2.74

3.40

– 0.23

(0.26)

SUB vs. CBRR

336

2.56

(2.75)

306

3.27

4.84

– 0.70 **

(0.28)

SUB vs. PBTH

198

1.99

(2.26)

183

2.71

3.69

– 0.71 **

(0.29)

CBRR vs. PBTH

181

2.43

(3.53)

185

2.69

3.24

– 0.26

(0.32)

CBRR = priority access to community-based rapid re-housing. PBTH = priority access to project-based transitional housing. SUB = priority access to permanent housing subsidy.

UC = usual care.

ITT = intention-to-treat. SD = standard deviation. SE = standard error.

*/**/*** Impact estimate is significantly different from 0 at the .10/.05/.01 levels, respectively, using a two-tailed t-test.

Notes: Impact estimates and outcome means are regression adjusted for baseline characteristics. See Appendix B for outcome definitions. All study sample families with an observed emergency shelter stay that includes random assignment date are included in the analysis.

Source: Family Options Study Program Usage Data

I

3-Year Impacts of Housing and Services Interventions for Homeless Families

1

Document Outline

Cover

Title page

ACKNOWLEDGMENTS

FOREWORD

TABLE OF CONTENTS

List of Exhibits

LIST OF ABBREVIATIONS AND ACRONYMS

EXECUTIVE SUMMARY

CHAPTER 1. INTRODUCTION

CHAPTER 2. DESCRIPTION OF USUAL CARE (UC) AND OUTCOMES MEASURED IN THE STUDY

CHAPTER 3. IMPACTS OF PERMANENT HOUSING SUBSIDY (SUB) COMPARED WITH USUAL CARE (UC)

CHAPTER 4. IMPACTS OF COMMUNITY-BASED RAPID RE-HOUSING (CBRR) COMPARED WITH USUAL CARE (UC)

CHAPTER 5. IMPACTS OF PROJECT-BASED TRANSITIONAL HOUSING (PBTH) COMPARED WITH USUAL CARE (UC)

CHAPTER 6. IMPACTS OF PERMANENT HOUSING SUBSIDY (SUB) COMPARED WITH COMMUNITY-BASED RAPID RE-HOUSING (CBRR), SUB COMPARED WITH PROJECT-BASED TRANSITIONAL HOUSING (PBTH), AND CBRR COMPARED WITH PBTH

CHAPTER 7. IMPACTS OF POOLED COMPARISONS

CHAPTER 8. DO CERTAIN INTERVENTIONS WORK BETTER WHEN OFFERED TO FAMILIES WHO FACE GREATER DIFFICULTIES?

CHAPTER 9. INTERVENTION COSTS

CHAPTER 10. CONCLUSIONS

CHAPTER 11. REFERENCES

CHAPTER 12. ADDITIONAL READING

APPENDIX A. DATA SOURCES AND DATA SET CONSTRUCTION

APPENDIX B. CONSTRUCTION OF OUTCOMES

APPENDIX C. ANALYSIS METHODS

APPENDIX D. ANALYSIS OF 37-MONTH SURVEY NONRESPONSE

APPENDIX E. IMPACTS ON USE OF TRANSITIONAL HOUSING

APPENDIX F. IMPACT ESTIMATES FOR POOLED COMPARISONS

APPENDIX G. INTERVENTION COSTS—METHODOLOGY, SITES, AND PROGRAMS

APPENDIX H. SUPPLEMENTAL ANALYSES

APPENDIX I. LENGTH OF BASELINE STAY IN EMERGENCY SHELTER