Short-Term

Impact Report

The HUD First-Time Homebuyer

Education and Counseling

Demonstration

Preliminary Findings

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

Short-Term Impact Report

The HUD First-Time Homebuyer Education

and Counseling Demonstration

Preliminary Findings

Prepared for

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Prepared by

Laura R. Peck

Shawn R. Moulton

Debbie Gruenstein Bocian

Donna DeMarco

Nichole Fiore

Abt Associates

Disclaimer

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. i

Acknowledgments

The authors of this report gratefully acknowledge the efforts of many individuals who have assisted in carrying out the HUD First-Time Homebuyer Education and Counseling

Demonstration since it began in 2011. We particularly thank the Government Technical Reviewer, Marina Myhre, for her guidance and support throughout all phases of the research.

Other staff members at the U.S. Department of Housing and Urban Development (HUD), Office of Policy Development and Research (PD&R) have provided valuable guidance and technical oversight for the study. We also extend our appreciation to Sarah Gerecke, Deputy Assistant Secretary for Housing Counseling for HUD’s Office of Housing Counseling.

The study would not be possible without the enthusiasm and dedication of the participating lenders and housing counseling agencies. The study team is grateful for their many contributions and continued commitment to the study. Specifically, we would like to thank NeighborWorks America, ClearPoint, Community Ventures Corporation, and the UnidosUS (formerly the National Council of La Raza).

This study has benefited from the contributions of a number of technical experts. The study design reflects guidance from experts in evaluation design methodology and from subject matter experts knowledgeable about and close to the operations of homebuyer education and counseling services. The authors are especially grateful to Jonathan Spader (Harvard Joint Center for Housing Studies), Roberto Quercia (University of North Carolina), Stephanie Moulton (Ohio State University), Doug Dylla (Doug Dylla Consulting), and Lucy Gorham (University of North Carolina). We also thank the members of our Expert Panel, Kenneth Brevoort (Consumer Financial Protection Bureau), J. Michael Collins (University of Wisconsin-Madison), Chris Herbert (Harvard Joint Center for Housing Studies), Genevieve Melford (Consumer Financial Protection Bureau), Carolina Reid (University of California-Berkeley), Kris Rengert (FDIC), and Peter Zorn (Freddie Mac).

We thank the many project staff at Abt Associates who helped to implement the study, and the Abt survey research staff who carried out the survey data collection. We recognize several staff who contributed to the analysis reported here: Utsav Kattel, Micah Villarreal, Michael Meneses, and Sarah Wolfe. We are grateful for Bry Pollack’s careful editing of this report.

At Abt Associates, the following staff have key roles in the study: Debbie Gruenstein Bocian and Laura Peck (Co-Principal Investigators), Donna DeMarco (Project Director), Nichole Fiore (Deputy Project Director), Shawn Moulton (Director of Analysis), Jill Khadduri (Project Quality Advisor), and Meryl Finkel (management oversight).

Finally, we are deeply grateful to the HUD First-Time Homebuyer Education and Counseling Demonstration participants who have continued to share their experiences of the homebuying process with the study team.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. ii

CONTENTS

Table of Contents

List of Exhibits ............................................................................................................................. vi

List of Boxes....................................................................................................................................x

Foreword ....................................................................................................................................... xi

Executive Summary ................................................................................................................... xiii

1.

Introduction ........................................................................................................................1

1.1 Research Question and Outcome Domains .................................................................3

1.2 Two Perspectives on Impacts......................................................................................5

1.3 Report Objectives and Study Timeline .......................................................................5

1.4 Organization of This Report .......................................................................................7

2.

Evaluation Design and Analysis .....................................................................................11

2.1 Enrollment and Characteristics of Study Participants ..............................................11

2.2 The Evaluation Design ...............................................................................................14

2.2.1 Initial Study Design: Control Group Plus Remote and In-

Person Treatment Groups .............................................................................14

2.2.2 Modified Study Design: Control Group Plus Remote and

Choice Treatment Groups .............................................................................15

2.3 Service Take-up and Completion Rates ....................................................................17

2.4 Impact Evaluation Questions and Contrasts .............................................................17

2.4.1 Primary Research Questions .........................................................................17

2.4.2 Impacts for Subgroups ..................................................................................18

2.4.3 Impacts by Delivery Mode............................................................................21

2.5 The Analytic Process for Computing Impacts ..........................................................23

2.6 Data Sources .............................................................................................................24

2.7 Outcomes for the Evaluation’s Impact Analyses ......................................................25

3.

The Intervention: In-Person and Remote Homebuyer Education and

Counseling ........................................................................................................................30

3.1 Content and Quality of Study’s Homebuyer Education and Counseling .................30

3.2 In-Person Service Delivery .......................................................................................32

3.2.1 In-Person Education ......................................................................................32

3.2.2 In-Person One-on-One Counseling ...............................................................32

3.2.3 Advantages/Disadvantages of In-Person Service Delivery ..........................33

3.3 Remote Service Delivery ..........................................................................................33

3.3.1 Online Education Course ..............................................................................33

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. i i

CONTENTS

3.3.2 Telephone One-on-One Counseling .............................................................34

3.3.3 Advantages/Disadvantages of Remote Service Delivery .............................34

3.4 Comparison of In-Person and Remote Services .......................................................34

3.5 Participant Experiences With Services .....................................................................37

3.6 Key Take-Aways About the Intervention .................................................................38

4.

Prospective Homebuyers’ Experiences ..........................................................................40

4.1 Homeowner Status and Housing Arrangements ..........................................................41

4.2 Homebuying Experience of Purchasers ....................................................................45

5.

Impacts on Preparedness and Search ............................................................................50

6.

Impacts on Financial Capability.....................................................................................58

6.1 Impacts on Financial Knowledge..............................................................................59

6.2 Impacts on Financial Behavior .................................................................................63

6.3 Impacts on Financial Indicators ................................................................................65

7.

Impacts on Sustainable Homeownership .......................................................................72

8.

Conclusion and Implications ...........................................................................................77

8.1 Key Findings .............................................................................................................77

8.2 Implications of Findings ...........................................................................................80

8.2.1 Impact on Access to Homeownership...........................................................80

8.2.2 Impact on Sustainable Homeownership ........................................................81

8.3 Insights into Service Delivery Modes .......................................................................82

8.4 Generalizability of Findings .....................................................................................84

8.4.1 Housing and Credit Environment .................................................................84

8.4.2 Study Sample ................................................................................................85

8.5 Next Steps .................................................................................................................86

Appendix A: Analytic Methods ..................................................................................................87

A.1 Estimating the Impact of Being Offered Services ....................................................87

A.2 Estimating the Impact of Taking Up Services ..........................................................88

A.3 Samples Used to Answer Evaluation Questions .......................................................94

A.4 Methods for Handling Missing Baseline and Outcome Data ...................................97

A.5 Balance Testing on Baseline Characteristics of the Analytic Sample ..........................97

A.6 Survey Nonresponse Weighting Methods ..............................................................101

A.7 Ability to Detect Impacts: Minimum Detectable Effects .......................................104

Appendix B: Data Sources and Measures ...............................................................................108

B.1 Data Sources ...........................................................................................................108

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. iv

CONTENTS

B.2 Combining Data Sources for Impact Analysis ........................................................111

B.3 Measure Construction .............................................................................................112

Appendix C: Expanded Results for the Overall Impact of Services .....................................123

C.1 Overall Impact (and How to Read the Impact Exhibits in This Appendix) ............123

C.2 Alternative Estimates of Overall Impact .................................................................129

Appendix D: Expanded Results for Impacts by Service Delivery Mode ..............................133

D.1 Expanded Results by Service Delivery Mode: In-Person, Remote, and Choice ....133

D.2 Comparison of Impact of Taking Up In-Person Services with Impact of Taking

Up Remote Services................................................................................................143

D.3 Comparison of Impact of Choice of Service Modes with Impact of Remote

Services ...................................................................................................................145

D.4 Sensitivity of Impacts of In-Person and Remote Services to Using Alternative

Samples ...................................................................................................................149

Appendix E: Impacts on Subgroups Defined by Baseline Characteristics ...........................157

E.1 Differences in Subgroup Impacts: Gender ..............................................................158

E.2 Differences in Subgroup Impacts: Age ...................................................................161

E.3 Differences in Subgroup Impacts: Area Housing Affordability .............................163

E.4 Additional Subgroup Results ..................................................................................165

Appendix F: Impacts on Subgroups Defined by Likelihood of Service Participation ........178

F.1 Method for Analyzing the Role of Service Participation ........................................180

F.2 What Characteristics Predict Participation in Services? .........................................182

F.3 What Is the Impact on Those Most Likely to Take Up Any Services? ..................183

F.4 What Is the Impact on Those Most Likely to Complete the Education

Curriculum? ............................................................................................................185

F.5 What Is the Impact on Those Most Likely to Complete One-on-One

Counseling? ............................................................................................................187

F.6 What Is the Impact on Those Most Likely to Complete All Services? ..................188

F.7 Discussion ...............................................................................................................190

Appendix G: Impacts on Subgroups Defined by Likelihood of Home Purchase .................193

G.1 Method for Analyzing the Role of Home Purchase ................................................194

G.2 What Characteristics Predict Home Purchase? .......................................................196

G.3 What Is the Impact on Those Most Likely to Purchase a Home? ...........................199

G.4 Discussion ...............................................................................................................200

Appendix H: Short-Term Follow-Up Survey ..........................................................................202

Works Cited ................................................................................................................................261

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. v

CONTENTS

List of Exhibits

Exhibit 1.1: Logic of the Intervention’s Influence on Outcomes ....................................................4

Exhibit 1.2: Timeline of the HUD First-Time Homebuyer Education and Counseling

Demonstration ..................................................................................................................................6

Terminology Used in This Report ...................................................................................................9

Exhibit 2.1: Study Locations and Sample Sizes ............................................................................12

Exhibit 2.2: Study Sample Characteristics at Baseline ..................................................................13

Exhibit 2.3: Study Sample, by Group and Time Period ................................................................16

Exhibit 2.4: Take-Up and Completion Rates, by Offer of In-Person or Remote Services ............17

Exhibit 2.5: Experimental Samples Used to Answer Each Impact Evaluation Question ..............22

Exhibit 2.6: Outcomes for Impact Analysis: Confirmatory, Secondary, and Exploratory ............27

Exhibit 3.1: National Industry Standards, Homebuyer Education’s Core Content .......................31

Exhibit 3.2: Differences Between In-Person and Remote Homebuyer Education and

Counseling .....................................................................................................................................36

Exhibit 3.3: Satisfaction With Services and Most “Helpful” Topics Covered, by Service

and Delivery Mode ........................................................................................................................38

Exhibit 4.1: Stage in the Homebuying Process at Short-Term Followup, by Baseline

Stage in the Homebuying Process ..................................................................................................42

Exhibit 4.2: Reasons for Postponing Purchase, Among Nonpurchasers Who Postponed

Their Home Search ........................................................................................................................44

Exhibit 4.3: Housing Arrangements at Short-Term Followup, Purchasers versus

Nonpurchasers ................................................................................................................................44

Exhibit 4.4: Housing Costs at Short-Term Followup, Purchasers versus Nonpurchasers .............45

Exhibit 4.5: Home Purchase Experience at Short-Term Followup, Among Purchasers ...............46

Exhibit 4.6: Home Purchase Price and Financing, Among Purchasers .........................................47

Exhibit 4.7: Loan Performance, Among Purchasers with a Mortgage Loan .................................48

Exhibit 5.1: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling on Preparedness and Search ........................................................................................53

Exhibit 5.2: Impact by Service Delivery Mode on Preparedness and Search ...............................57

Exhibit 6.1: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling on Financial Knowledge .............................................................................................60

Exhibit 6.2: Impact by Service Mode on Financial Knowledge ....................................................61

Exhibit 6.3: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling on Financial Behaviors ...............................................................................................64

Exhibit 6.4: Impact by Service Mode on Financial Behaviors ......................................................64

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. vi

CONTENTS

Exhibit 6.5: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling on Financial Indicators ...............................................................................................65

Exhibit 6.6: Impact by Service Mode on Financial Indicators ......................................................66

Exhibit 7.1: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling on Sustainable Homeownership .................................................................................73

Exhibit 7.2: Impact by Service Delivery Mode on Sustainable Homeownership .........................74

Exhibit 8.1: Summary of Selected Impacts ....................................................................................79

Exhibit A.1: Take-Up of Services Based on Short-Term Follow-Up Survey Responses .............93

Exhibit A.2: Defining Samples Used to Answer Evaluation Questions ........................................95

Exhibit A.3: Detail on Samples Used to Answer Evaluation Questions .......................................96

Exhibit A.4: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents,

Demographic Characteristics of Study Participants ......................................................................99

Exhibit A.5: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents,

Measures of Homebuying Stage, Financial Capability, and Creditworthiness of Study

Participants ...................................................................................................................................100

Exhibit A.6: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents

and Nonrespondents, Demographic Characteristics of Study Participants ..................................101

Exhibit A.7: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents

and Nonrespondents, Measures of Homebuying Stage, Financial Capability, and

Creditworthiness of Study Participants ........................................................................................103

Exhibit B.1: Timing of Short-Term Follow-Up Survey Response ..............................................109

Exhibit B.2: Timing of Followup Credit Data .............................................................................110

Exhibit B.3: Operationalization of Baseline Covariates ..............................................................113

Exhibit B.4: Operationalization of Subgroup Identifiers .............................................................116

Exhibit B.5: Operationalization of Outcome Measures ...............................................................118

Exhibit C.1: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling, Expanded Results.....................................................................................................126

Exhibit C.2: Overall Impact of the Demonstration’s Homebuyer Education and

Counseling, Excluding the In-Person Treatment Group ..............................................................130

Exhibit D.1: Impact of In-Person Homebuyer Education and Counseling, Expanded

Results ..........................................................................................................................................134

Exhibit D.2: Impact of Remote Homebuyer Education and Counseling, Expanded Results ......137

Exhibit D.3: Impact of Choice of In-Person or Remote Homebuyer Education and

Counseling ...................................................................................................................................140

Exhibit D.4: Comparison of Impact of Taking Up In-Person Services with Impact of

Taking Up Remote Services ........................................................................................................144

Exhibit D.5: Comparison of Impact of Being Offered Choice of In-Person or Remote

Services with Impact of Being Offered Remote Services ...........................................................146

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. vii

CONTENTS

Exhibit D.6: Comparison of Impact of Taking Up Services When Given Choice of

Service Modes With Impact of Taking Up Services When Offered Remote Services ...............148

Exhibit D.7: Impact of In-Person Homebuyer Education and Counseling, Excluding

Choice Group ...............................................................................................................................151

Exhibit D.8: Impact of Remote Homebuyer Education and Counseling, Excluding Choice

Group ...........................................................................................................................................154

Exhibit E.1: Comparison of Impacts on Subpopulations Defined by Gender .............................160

Exhibit E.2: Comparison of Impacts on Subpopulations Defined by Age at Baseline................162

Exhibit E.3: Comparison of Impacts on Subpopulations Defined by Area Housing

Affordability ................................................................................................................................164

Exhibit E.4: Comparison of Impacts on Subpopulations Defined by Race/Ethnicity:

White Non-Hispanic Versus All Other Race/Ethnicities .............................................................166

Exhibit E.5: Comparison of Impacts on Subpopulations Defined by Race/Ethnicity:

White Non-Hispanic Versus African-American Non-Hispanic ..................................................167

Exhibit E.6: Comparison of Impacts on Subpopulations Defined by Educational

Attainment at Baseline .................................................................................................................168

Exhibit E.7: Comparison of Impacts on Subpopulations Defined by Marital Status at

Baseline ........................................................................................................................................169

Exhibit E.8: Comparison of Impacts on Subpopulations Defined by English Language

Preference ....................................................................................................................................170

Exhibit E.9: Comparison of Impacts on Subpopulations Defined by Stage in Homebuying

Process at Baseline .......................................................................................................................171

Exhibit E.10: Comparison of Impacts on Subpopulations Defined by Credit Score at

Baseline ........................................................................................................................................172

Exhibit E.11: Comparison of Impacts on Subpopulations Defined by Cash on Hand for

Downpayment and Closing Costs at Baseline .............................................................................173

Exhibit E.12: Comparison of Impacts on Subpopulations Defined by Savings at Baseline ........174

Exhibit E.13: Comparison of Impacts on Subpopulations Defined by Borrower Income

Relative to Area Median Income at Baseline ..............................................................................175

Exhibit E.14: Comparison of Impacts on Subpopulations Defined by Nonhousing Debt at

Baseline ........................................................................................................................................176

Exhibit F.1: Impact of In-Person Homebuyer Education and Counseling, Expanded

Results: Comparison of Impacts on Subpopulations Defined by Likelihood of Taking Up

Any Services ................................................................................................................................184

Exhibit F.2: Comparison of Impacts on Subpopulations Defined by Likelihood of

Completing Education Curriculum ..............................................................................................186

Exhibit F.3: Comparison of Impacts on Subpopulations Defined by Likelihood of

Completing One-on-One Counseling ..........................................................................................187

Exhibit F.4: Comparison of Impacts on Subpopulations Defined by Likelihood of

Completing All Services ..............................................................................................................189

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. viii

CONTENTS

Exhibit G.1: Impact of In-Person Homebuyer Education and Counseling, Expanded

Results: Predictors of Home Purchase for Control Group ...........................................................196

Exhibit G.2: Comparison of Impacts on Subpopulations Defined by Likelihood of

Purchasing a Home ......................................................................................................................199

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. ix

CONTENTS

List of Boxes

The Intervention .......................................................................................................................... xivv Terminology Used in This Report ...................................................................................................9

Calculating Impact Two Ways: Understanding Intent-to-Treat (ITT) versus Treatment-on-the-Treated (TOT) ..........................................................................................................................19

Key Findings: Prospective Homebuyers’ Experiences ..................................................................40

Key Findings: Impacts on Preparedness and Search .....................................................................50

How to Read the Impact Exhibits in This Report ..........................................................................51

Understanding Null Effects: Minimum Detectable Effects and Why They Matter .......................52

Key Findings: Impacts on Financial Capability .............................................................................58

Mortgage Literacy Quiz .................................................................................................................63

Key Findings: Impacts on Sustainable Homeownership ...............................................................72

Key Findings: Expanded Results for the Overall Impact of Services .........................................123

Key Findings: Expanded Results for Impacts by Service Delivery Mode ..................................133

Key Findings: Impacts on Subgroups Defined by Baseline Characteristics ................................157

Key Findings: Impacts on Subgroups Defined by Likelihood of Service Participation .............178

Key Findings: Impacts on Subgroups Defined by Likelihood of Home Purchase ......................193

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. x

Foreword

The U.S. Department of Housing and Urban Development (HUD) has sought to measure the impact of pre-purchase counseling for almost 40 years. While several recent studies have shown positive or neutral results of homebuyer education or counseling on outcomes such as personal budgeting, use of credit, mortgage delinquency, and foreclosure avoidance, none of these studies is a large-scale, long-term random assignment trial.

Congress and researchers have long recognized the importance of a true randomized research demonstration to isolate the effect of housing education and counseling on first-time homebuyer outcomes from other effects. This report provides a window into the rigorously designed demonstration now underway to provide insight into the value of homebuyer education and counseling that meet HUD Housing Counseling Program standards and are delivered both remotely (through the Internet and telephone) and in person to potential first-time homebuyers.

In 2014, HUD successfully launched a large-scale, randomized experiment to reliably assess the impact of homebuyer education and counseling for a diverse sample of over 5,800 low-to-moderate and middle-income prospective first-time homebuyers in 28 U.S. metropolitan areas.

This report comes with two important caveats—

• First, this is not a study of who normally receives housing counseling. In general, households who receive housing counseling do so in order to qualify for special

programs, such as down payment assistances. Instead, this is a study of a broader set of households who had inquired with a lender about their eligibility for getting a loan to buy a home. That is, this study is looking at a bigger policy question of the value of homebuyer education and counseling to a larger universe of prospective homebuyers.

• Second, this is an interim study. This means that most of the data we are presenting should be seen as interim tracking indicators and not final impact indicators.

Here is what the study says are important at this time: the short-term findings—12 to 18 months after random assignment—on the effect of homebuyer education and counseling are most relevant as related to the effects of housing counseling on preparedness, search, and financial capability. Specifically, the report shows members of the treatment group who were offered the education and counseling were—

• More likely to be confident they could find information they needed about the

homebuying process.

• More likely to report being very satisfied with the homebuying process.

• More likely to have their mortgage payments automatically deducted from their bank account.

• More likely to say they would contact a counseling agency or nonprofit prior to missing a mortgage payment.

As noted earlier, the report also provides data on metrics that are not meaningful at 12 to 18

months after random assignment but we present them to show that these are the longer-term

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xi

FOREWORD

measures we are tracking. Specifically, 12 to 18 months after random assignment is insufficient to measure sustainable homeownership and financial wellbeing.

For that reason, we caution against over interpreting any results in this short-term report on long-term impacts such as loan performance measures that require more time for measuring meaningful impacts. The same is true with the relationship between the education and counseling intervention and non-housing debt. This interesting result deserves further examination and the Department maintains that it would be too soon to draw any conclusions about the relationship between education and counseling and non-housing debt.

HUD is committed to providing interim reports for our long-term research studies. This interim report contributes to our knowledge on housing counseling; there is more to learn as we continue this important research over the next few years.

Seth D. Appleton

Assistant Secretary for Policy Development and Research

U.S. Department of Housing and Urban Development

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xii

Executive Summary

Homeownership has been a cornerstone of the American Dream for decades. However, the recent collapse and uneven recovery of the housing market not only forced Americans to confront the risks that homeownership can pose to their financial lives but also threatened to undermine the perception of homeownership as a stepping stone to the middle class and a symbol of economic stability.

Homebuyer education and counseling are designed to help prospective first-time homebuyers understand the benefits and risks of homeownership, choose an affordable home and an appropriate mortgage, and build the financial knowledge, resources, and behaviors needed for sustainable homeownership and long-term financial health. One, homebuyer education and counseling involve helping people decide whether homeownership is appropriate for them. Two, homebuyer education and counseling help prospective homebuyers understand the homebuying process and the responsibilities of homeownership. These two focuses—helping people decide whether to purchase, and if they do, helping them navigate the purchase process and prepare for homeownership—are, in theory, how homebuyer education and counseling help provide stability and security over the long term.

A body of prior research suggests that education and counseling are helpful for homebuyers, but conclusive evidence that homebuyer education and counseling are the cause of the desired outcomes is lacking. 1 Homebuyer education and counseling are commonly provided remotely (that is, online and by telephone) or in person, but the relative impacts of these alternative modes of service delivery have not been systematically evaluated.

To fill this research gap, in 2011, the U.S. Department of Housing and Urban Development (HUD) launched the First-Time Homebuyer Education and Counseling Demonstration under a contract with Abt Associates. As the first such large-scale national experimental evaluation of homebuyer education and counseling, the Demonstration is poised to provide rigorous, experimental evidence about the effectiveness of homebuyer education and counseling overall and the relative effectiveness of the two service delivery modes.

Research Question and Outcomes of Interest

The primary research question of the study is this: What are the impacts of homebuyer

education and counseling on low-, moderate-, and middle-income prospective first-time

homebuyers? 2

1 See Collins and O’Rourke (2011) and DeMarco et al. (2017) for a summary of the literature.

2 Low-, moderate-, and middle-income homebuyers have incomes at or below 120 percent of their local area median income.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xiii

EXECUTIVE SUMMARY

In answering the study’s primary

The Intervention

question, we consider two variants of

The intervention evaluated in this study consists of the

it: What are the impacts of being

offer of free homebuyer education and counseling.

offered homebuyer education and

Homebuyer education is training about buying a home

and financial management; homebuyer counseling is

counseling? and What are the impacts

one-on-one guidance tailored to the particular needs of the

of taking up the offer of homebuyer

individual homebuyer. The study sample includes people

who approached a lender about a mortgage, were not

education and counseling? Each of

otherwise required to receive counseling, agreed to be in

these two distinct questions and their

the study, and met some other selection criteria (see

associated impact estimates is relevant

section 2.1). Within the treatment group, 55 percent used

any services.

its own way. The first question is

The homebuyer education and counseling services in the

about the offer of services (which

study were provided through two service modes: in-

includes the whole treatment group,

person services provided at a local counseling agency or

remote services provided through online education and

including those who took up the offer

telephone one-on-one counseling. These two modes

and those who did not); 3 the relevant

reflect current practices in the housing counseling industry.

impact estimate is called the “intent-

To provide services, the study team partnered with 63

HUD-approved local housing counseling agencies

to-treat” (or ITT) impact. The second

across 28 large metropolitan areas throughout the United

question is about taking up that offer

States and two HUD-approved national agencies

(which focuses on just those who used

providing remote services. The local agencies delivered

in-person homebuyer education in group workshops and

the services); the relevant impact

in-person homebuyer counseling in one-on-one, face-to-

estimate is called the “treatment-on-

face sessions. For remote services, the study team

partnered with eHome America for online homebuyer

the-treated” (or TOT) impact.

education and ClearPoint Credit Counseling Solutions for

telephone counseling.

The study also examines how the

Al agencies participating in the study adhere to the

mode of service delivery—in-person

National Industry Standards for Homeownership

Education and Counseling and are HUD-approved.

or remote—influences the

These requirements ensured that the intervention services

effectiveness of the intervention. In

provided through the study were reasonably consistent in

addition, we consider how impacts

structure and content and were administered by programs

reviewed by HUD to meet its standards for quality.

vary for subgroups of the study

Findings generated from this study are most relevant to

sample as defined by their baseline

homebuyer education and counseling that is offered after a

demographic and socioeconomic

prospective homebuyer first communicates with a lender.

Findings may not be applicable to other types of programs

characteristics and by area housing

(for example, foreclosure counseling) or to services

affordability. The specific subgroups

provided by other types of agencies (for example, those

that do not adhere to the National Industry Standards or

were chosen because the study team,

are not approved by HUD).

HUD, and the study’s expert panel

The study’s Baseline Report (DeMarco et al., 2017) and

identified them as having particular

this report’s chapter 3 provide additional detail on the

intervention’s implementation and operations, as well as

policy interest.

participants’ experiences with services and the market in

which the Demonstration takes place.

3 To “take up” means prospective homebuyers actually use the services offered: by talking by phone with a counselor and visiting the online education materials (remote mode) or by attending a face-to-face counseling session and a group education workshop (in-person mode).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xiv

EXECUTIVE SUMMARY

The study examines impacts on a series of outcomes organized into three outcome domains: (1) preparedness and search, (2) financial capability, 4 and (3) sustainable homeownership.

Study Design and Methods

The First-Time Homebuyer Education and Counseling Demonstration uses a randomized experimental design to evaluate the effectiveness of homebuyer education and counseling services. This experimental evaluation design provides strong evidence of the effectiveness of these services: differences between the treatment group and the control group outcomes can be interpreted as the causal impact of being offered homebuyer education and counseling services.

Between September 2013 and February 2016, the study randomized more than 5,800

prospective first-time homebuyers from 28 large metropolitan areas either into a control group or into one of three treatment groups defined by the mode of services offered. Study participants assigned to a treatment group were offered free in-person services, free remote services, or their choice of free in-person or free remote services. Those study participants randomly assigned to the control group were not offered any homebuyer education or counseling services.

Generally speaking, the difference between treatment and control group outcomes is a program’s impact. As noted, this study computes impact in two ways, in order to report the impact of making services available (intent-to-treat, or ITT, impact estimate) and the impact of taking up those services (treatment-on-the-treated, or TOT, impact estimate).

• The study’s ITT impact estimate reflects the impact of the offer of services. This encompasses both the degree to which treatment group members decide to use those services and the effectiveness of the services.

• The study’s TOT impact estimate reflects the impact of taking up services, which analyzes the data using the assumption that those who did not use any services (“no-shows”) experience no impact.

Importantly, the ITT impact analysis and the TOT impact analysis by definition yield the same pattern of results—positive or negative impact, large or small impact, statistically significant or not. 5 It is that pattern that we emphasize in discussing this study’s results. 6

4 Financial capability refers to the capacity (that is, having the skills, attitudes, behaviors, and knowledge) to manage financial resources to achieve one’s financial goals.

5 This is the case because the TOT estimate is essentially computed by dividing the ITT estimate by the take-up rate.

Similarly, the TOT estimate’s standard error is essentially computed by dividing the ITT estimate’s standard error by the take-up rate. As a result, the TOT impact will always be larger in magnitude than the ITT estimate (because the take-up rate is always less than one); and the level of statistical significance remains constant between the two estimates (because the TOT impact estimate and standard error are both scaled up by the same amount).

6 See appendix section A.2 for further discussion.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xv

EXECUTIVE SUMMARY

Although we focus on the overall pattern of results, we leave readers to choose which result, ITT

or TOT, is more relevant to them. For example, for national policy or for lenders interested in the effect of making services available, the ITT estimate may be more relevant because it reflects the impact of the general availability of such services. For housing counseling agencies or for lenders requiring certain potential borrowers to participate in services, the TOT estimate may be more relevant because it reflects the impact of the intervention on those who show up for counseling (whether in person or virtually).

Data Sources

These impact analyses are conducted using followup data on key outcomes that come from a number of sources. The study collected administrative data from the Federal Housing Administration (FHA), a credit bureau, three national mortgage lenders, and service provider agencies. The study also collected survey data via its Short-Term Follow-Up Survey, which achieved a 79-percent response rate.

As of the fall of 2017, the study had administrative and survey followup data for all of the study’s participants. Those data reflect outcomes as of about 12 to 18 months after study enrollment.

Short-Term Impact Report’s Timeframe

This report provides the Demonstration’s first complete look at the short-term impacts of homebuyer education and counseling on prospective first-time homebuyers. At this time point, between 12 and 18 months into the followup period, it seems likely that most of any detected impacts would be in the two domains of preparedness and search and financial capability, with impacts in the sustainable homeownership domain requiring longer to materialize.

Summary of Key Findings

As of the short-term followup period, we observe mixed evidence of the impact of homebuyer education and counseling. Although no impact was detectable on the study’s main gauge of the intervention’s “success”—the 60-day mortgage delinquency rate—we do find impacts on a several other outcomes, particularly on those in the domains of preparedness and search and financial capability. 7 In these two domains, we find some impacts of homebuyer education and counseling on the treatment group that are encouraging.

7 Although we report on the 60-day delinquency rate in this report, the outcome is not the report’s primary focus as it will be in the study’s longer term-followup analysis and report. Because it is too early to judge the success of the intervention with respect to sustainable homeownership, this report instead focuses on outcomes related to preparedness and search and to financial capability.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xvi

EXECUTIVE SUMMARY

• Greater confidence in ability to find needed information. Members of the treatment group were more likely than the control group to be confident that they could find information they needed about the homebuying process.

• Higher rates of home purchase for young adults. The full treatment group and full control group had similar rates of home purchase, but for those age 29 or younger at the time they enrolled in the study, the treatment group had higher rates of homeownership than their control group counterparts.

• Greater satisfaction with the homebuying process. The treatment group was more likely to report being “very satisfied” with the homebuying process than the control group.

• Greater likelihood of having their mortgage payments automatically deducted from their bank account. The treatment group was more likely than the control group to have arranged for their mortgage to be paid monthly through automatic deduction.

• Greater likelihood of seeking assistance in times of financial distress. The treatment group was more likely than the control group to say they would contact a counseling agency or another nonprofit for assistance prior to missing a mortgage payment.

On the other hand, we find that homebuyer education and counseling had no detectable impact or even some unexpected impacts on a few select outcomes:

• No detectable impact on loan performance measures. No impact was detectable on the 30-, 60-, or 90-day delinquency rates. This may be due, in part, to typically low delinquency and default rates of newly-originated mortgages that would be captured during a short followup period of 12 to 18 months.

• Higher levels of debt. The treatment group experienced higher levels of nonhousing debt—

primarily student loan debt—than their control group counterparts. In addition, the treatment group had a slightly higher monthly debt-to-income ratio than the control group (0.27 versus 0.26), although the treatment and control groups were equally likely to have debt-to-income ratios that exceeded 0.43, the upper limit specified by FHA guidelines.

• Greater prevalence of high monthly housing costs relative to income. Overall, treatment group members more often had housing costs that exceeded 30 percent of their household income than did the control group. However, the impact of homebuyer education and counseling on housing costs is not robust to alternative measures of housing costs. We do not detect an impact on a continuous housing-cost-to-income ratio or on other ratio thresholds such as 25, 35, or 40 percent of income.

• Lower reported ability to cover all bills. Overall, a higher proportion of the treatment group than the control group reported occasionally not having enough money to cover bills.

Mode-Specific Impacts

On a small number of outcomes, we find differences in the impact of in-person services as compared to the impact of remote services, as noted below. As always, the pattern of results is the same whether we consider ITT or TOT impacts. Relative to the control group:

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xvii

EXECUTIVE SUMMARY

• Treatment group members offered in-person services reported higher satisfaction with the homebuying process and were more likely to achieve credit scores above 620, but they were also more likely to have high monthly housing costs.

• Treatment group members offered remote services reported greater confidence in their ability to correct inaccuracies in their credit reports and higher likelihood that they would reach out to housing counseling agencies or other organizations for help if they were in financial distress.

• Treatment group members offered remote services were more likely to say they would contact their lender if in financial distress, prior to missing a payment.

• Treatment group members offered remote services reported higher student loan balances.

Although this study is able to assess whether differential mode effects occurred, it is not designed to identify the specific reasons for those differentials. As a result, although we can offer possible explanations for the differential impacts that we observe, we do not have direct evidence linking mode-specific practices to these impacts. We must also note that, for these mode effect analyses, the mode-specific samples are smaller in size than the sample as a whole, which makes it harder to detect impacts. As such, only those impacts that are of relatively larger magnitude are detectable; smaller impacts may exist, but the study did not detect them.

Implications of Findings

What do these short-term findings tell us about how well homebuyer education and counseling achieve their primary goal of improving homeownership-related outcomes for low- to middle-income prospective first-time homebuyers?

Sustainable Homeownership

We find some encouraging evidence that homebuyer education and counseling might affect the likelihood of success for people who become homeowners. In particular, the intervention had favorable impacts on study participants having their mortgage payments automatically deducted from their bank account and on their understanding that they should proactively contact housing counseling agencies or another nonprofit in times of financial distress. Both could translate into fewer mortgage delinquencies and fewer defaults and foreclosures down the road. Our findings also suggest that the treatment group is both better informed (that is, better equipped to find needed information) and more satisfied with the homebuying process.

That said, some of the findings could lead to unfavorable effects on sustainable homeownership later on. In particular, evidence of relatively higher levels of nonhousing debt among the treatment group as compared to the control group could be cause for concern. This finding is driven by a positive impact on student loan debt (one component of nonhousing debt): treatment group members had higher levels of student loan debt at followup compared to their control group counterparts. One explanation for this finding might be that borrowers with lower credit scores are shifting resources from paying down student debt (or are taking on increased levels of student debt) in order to build savings—for example, for a downpayment. In some

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xviii

EXECUTIVE SUMMARY

circumstances, this could be a rational choice; for example, if debt-to-income ratios are lesser barriers to homeownership than are accumulation of savings for downpayments and closing costs. However, the higher levels of nonhousing debt could impair loan performance. This is a finding that we will follow to see whether the pattern holds at longer-term followup.

We also found that homebuyer education and counseling increases the incidence of study participants paying more than 30 percent of their income on housing. If the intervention is somehow encouraging people—particularly those with lower incomes—to spend more of their income on housing, this could have potentially negative effects on sustainable homeownership.

On the other hand, it could be that the intervention is simply encouraging people to consider the transaction costs related to a “trade-up” home purchase later on and is leading some buyers to purchase a home that could meet their needs for a longer time period. If so, this could be a rational decision with no adverse consequences for homeownership.

Home Purchase Rates

In the short term, we observe a higher rate of home purchase among study participants age 29 or younger in the treatment group, relative to their control group counterparts. This finding suggests that homebuyer education and counseling might be supporting younger people—whose

homeownership rates are at historic lows—to achieve homeownership. This finding implies that lenders, housing counseling agencies, and even society more broadly might consider how to capitalize on the possibility of increasing rates of purchase among younger people, providing them with greater opportunity to experience the benefits of homeownership.

Service Delivery Mode

When offered a choice of service delivery mode, a large majority (about three-fourths) of study participants stated a preference for remote services over in-person services. Beyond this descriptive finding, we find mixed evidence on the impact of the in-person and remote services modes, with each mode having its own set of favorable and unfavorable impacts (although most outcomes show no detectable impact). These mode-specific effects could reflect fundamental differences in how service recipients experience in-person versus remote services. For example, it might be that in-person services have the advantage of fostering a sense of support and community, whereas remote services help participants become comfortable conducting online research on their own. We therefore encourage further exploration of possible reasons for differential impacts between modes, which might inform service delivery of each mode in practice.

Study Context

This study has a large, diverse sample, but it is important to note our study selection processes make that sample somewhat particular. Participants were referred to the study by their lender and not via their contact with housing counseling agencies. We also screened out individuals for whom homeownership education and counseling were mandated under the requirements of a mortgage or downpayment assistance program. As a result, our study sample includes individuals who were willing to engage in homebuyer education and counseling services, but The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xix

EXECUTIVE SUMMARY

may not reflect the typical pool of housing counseling clients who have already taken the step to participate in those services.

In addition, many key outcomes considered in this study—such as the home purchase decision and delinquency rates—are sensitive to prevailing economic and other conditions. It is, therefore, important to consider the economic and credit climate under which this Demonstration takes place. During the study’s enrollment period of September 2013 through February 2016, interest rates were low, the economy was growing, and the housing market was appreciating, all of which would generally support people entering the housing market. On the other hand, this period coincided with a period of extremely tight credit conditions relative to historical standards and rising house prices. 8 As a result, although some potential homebuyers might be encouraged to take steps to become homeowners, others might be discouraged by the tight credit market and therefore less interested in homebuyer education and counseling.

Homebuyer education and counseling inevitably will be mediated by current market conditions, regardless of when the effects are measured. Readers should keep this in mind when translating findings to periods with markedly different market settings. Still, the study findings represent experimental evidence that will improve on available evidence on the impacts of homebuyer education and counseling.

Next Steps

This report presents impacts on short-term outcomes observed 12 to 18 months after study participants enrolled in the study. At this short-term followup point, we would expect to see the most pronounced impacts, if any, on those outcomes expected to arise soon after the intervention. This would be outcomes related to homeownership preparedness and search and outcomes (knowledge, behaviors, indicators) related to the financial capability tools developed during homebuyer education and counseling.

We would not expect to detect impacts on many of the longer-term sustainable homeownership measures, because mortgage delinquencies are rare events within the first year of a mortgage.

The study’s Long-Term Impact Report, scheduled for 2020, will analyze outcomes at about 3½

to 5 years after study enrollment, emphasizing those related to sustainable homeownership.

Moreover, the study’s mixed findings on short-term outcomes flag the importance of longer-term followup and analysis. It will be important to observe how overall, subgroup, and service delivery mode effects evolve over time.

8 For more information on the economic, housing, and credit market conditions that prevailed during this time period, see DeMarco et al. (2017).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. xx

1. Introduction

For generations, homeownership has been a gateway to the middle class and a cornerstone of the American Dream. Homeownership helps households build wealth in a variety of ways. First, homeowners automatically build wealth through the accumulation of home equity, as a portion of each regularly scheduled mortgage payment is applied to reducing the principal owed. As long as homes do not depreciate in value, these mortgage payments work, in part, as an “automatic savings” mechanism. 9 Second, homeownership is a “leveraged” investment: homebuyers typically commit just a small fraction of the home’s value as a downpayment but realize returns on the entire value of the property if it appreciates. Finally, for many homeowners, homeownership comes with tax advantages. 10 Homeownership also has risks. Unaffordable mortgage payments, the opportunity costs of not investing in better-performing assets, home maintenance responsibilities, and the transaction costs of moving can make homeownership more of a burden than a benefit.

Homebuyer education and counseling programs are designed to help prospective homebuyers think about the benefits and risks of homeownership, understand how to choose a home and mortgage, and build the financial knowledge and behaviors needed for sustainable

homeownership and financial health. These services are especially relevant to first-time homebuyers.

Despite substantial research to date on homebuyer education and counseling programs, the nonexperimental nature of those studies limits our ability to determine whether homebuyer education and counseling cause the desired outcomes. Without a randomized experiment, it is difficult to isolate the impact of homebuyer education and counseling from influences of other factors, such as borrower self-selection (for example, more self-motivated homebuyers might be more likely to use counseling services) or lender practices (for example, certain types of mortgages and downpayment assistance programs are available only to borrowers who have completed counseling).

In addition to lacking conclusive evidence on the causal impacts of homebuyer education and counseling, the field also lacks rigorous evidence on the relative effectiveness of service delivery modes: are education and counseling provided in person more (or less) effective than services provided remotely? In the world of housing counseling practice, some are strong proponents for in-person service provision: they insist that having the in-person experience is essential and 9 Homeowners must stay in their home for a minimum amount of time for these accrued savings to compensate for the transaction costs associated with purchasing.

10 The Tax Cuts and Jobs Act of 2017 decreases the tax advantages of homeownership by raising the standard deduction and reducing the mortgage interest and property tax deductions. However, this law had not gone into effect as of the time period of our analysis.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 1

1. INTRODUCTION

much more likely to generate the desired impacts because it allows for personalized hands-on attention. 11 That said, the availability of remotely provided services increases access to homebuyer education and counseling for people who live in areas without local housing counseling agencies, who otherwise face barriers to being able to participate in person or who prefer that mode of learning. If remote services are determined to be as effective as in-person services, then there could be opportunities for increasing the efficiency of the housing counseling industry by shifting resources to remote delivery modes. Remote services generally can reach more people at lower costs.

In order to fill these information gaps, the U.S. Department of Housing and Urban Development (HUD) established the First-Time Homebuyer Education and Counseling Demonstration, a large-scale, multi-site experimental study. The Demonstration uses a randomized experimental design to evaluate the effectiveness of homebuyer education and counseling services for low-, moderate-, and middle-income prospective first-time homebuyers. These are homebuyers whose incomes fall at or below 120 percent of the local median. 12 The study began enrollment in September 2013 and randomly assigned (through February 2016) more than 5,800 prospective first-time homebuyers in 28 large metropolitan areas. Study participants were randomized either into a control group (who were not offered any services) or into one of three treatment groups , where treatment group members were offered access to free homebuyer education and counseling services (“the intervention”).

Members of one treatment group were offered in-person services; another treatment group, remote services; and a third treatment group, their choice of remote or in-person services. The goal was to produce evidence on the differences in the impact of being offered in-person services relative to being offered remote services. These two service modes reflect the predominant models of homebuyer education and counseling in the field. The experimental evaluation design means that differences between the treatment group and the control group outcomes can be interpreted as the causal impact of being offered homebuyer education and counseling services.

This interpretation applies both to the overall impact of being offered homebuyer education and counseling services (which is computed by comparing outcomes for the pooled sample of all treatment group members to the control group) as well as for the impact of being offered each 11 As described by Collins and O’Rourke (2011: 44), “A frequent assumption in the financial education field is that face-to-face delivery methods are more effective than other modes of delivery. The current literature does not support this assumption.” In addition, the authors note that “variations in the form of service such as group education and online or telephone procedures, and even combinations of services, should be evaluated compared to each other.”

12 Low-, moderate-, and middle-income homebuyers are those who have incomes at or below 120 percent of their local area median income (AMI). Specifically, those whose incomes are less than 50 percent of AMI are classified as “low” income; those whose incomes are between 50 and 80 percent of AMI are classified as “moderate” income; and those whose incomes are between 80 and 120 percent of AMI are classified as “middle” income. Those whose incomes are above 120 percent of AMI are considered “upper” income and are not targets of this study.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 2

1. INTRODUCTION

service delivery mode (in-person and remote). The evaluation also estimates the impacts for those who take up homebuyer education and counseling.

1.1 Research Question and Outcome Domains

The primary question guiding this evaluation is, What are the impacts of homebuyer education and counseling on low-, moderate-, and middle-income prospective first-time homebuyers?

In answering the study’s primary question, we consider two variants of the question: What are the impacts of being offered homebuyer education and counseling? What are the impacts of taking up the offer of homebuyer education and counseling? The first question is about the offer of services, which includes the whole treatment group, including those who took up the offer and those who did not, and the relevant impact estimate is called the “intent-to-treat” (or ITT) impact. The second question is about taking up that offer, which focuses on just those who took up the offer, and the relevant impact estimate is called the “treatment-on-the-treated” (or TOT) impact.

We consider the intervention’s impact on study participants in three broad domains of outcomes:

• Preparedness and search—These outcomes are related to the decision of whether to purchase a home or not, the search for affordable homes, and selection of appropriate mortgages.

• Financial capability—These outcomes are related to participants’ general financial knowledge, behavior and traditional financial markers such as debts and savings, access to affordable credit, and credit profile.

• Sustainable homeownership—These outcomes are related to homebuyers’ mortgage payment behaviors, including those behaviors that can play a role in avoiding foreclosure and accruing and protecting home equity.

Exhibit 1.1 depicts the mechanisms through which homebuyer education and counseling are expected to affect outcomes in these domains over the short term (within 1 year of being offered services) and the longer term (3 or more years after being offered services).

Within the domain of preparedness and search, homebuyer education and counseling should increase service recipients’ knowledge and skills in the short term. This includes recipients’

awareness and knowledge of the pros and cons of homeownership, the responsibilities of homeownership, mortgages and terms, and underwriting criteria. In addition, education and counseling should enhance a recipient’s ability to determine the affordability of homes and the appropriateness of financing options. The added knowledge and skills are expected to inform a host of behaviors and decisions, starting with whether or not to purchase a home. For service recipients who decide to purchase a home, the intervention should help them search for and select affordable homes and to select and qualify for appropriate mortgages.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 3

1. INTRODUCTION

Exhibit 1.1: Logic of the Intervention’s Influence on Outcomes

Within the domain of financial capability, homebuyer education and counseling should, in the short term, improve recipients’ knowledge of financial terminology and the importance of good credit. It also should improve recipients’ ability to create budgets, track expenses, and correct credit reports. These enhanced skills and knowledge are expected to translate into better financial behaviors such as improved budgeting practices and better money and debt management. In turn, these favorable behaviors should lead to a better financial situation, as proxied by such markers as savings.

These two domains (preparedness and search; financial capability) are the most central to this report’s analysis of short-term impacts. Because of their temporal proximity to the intervention, we are most likely to find a statistically significant impact on outcomes in these domains in the short term. We do not have explicit hypotheses regarding any potential differential effects of services delivered remotely as opposed to in person, but we do test whether any differences arise.

In the longer term, the central goal of homebuyer education and counseling is sustainable homeownership—helping people decide whether homeownership is right for them and, for those who do purchase homes, helping them avoid foreclosure and build wealth. Homebuyer education and counseling can increase sustainable homeownership in several ways: (1) by helping people make good tenure decisions (that is, whether and when to purchase a home); (2) by helping people choose homes and financing options that are appropriate given their financial situation, goals, and priorities; and (3) by promoting behaviors that lead to timely mortgage payments.

If homebuyer education and counseling services improve recipients’ home preparedness and search capabilities (including better understanding the risks, benefits, and responsibilities of homeownership and being better able to navigate the homebuying process), then recipients should be better equipped to make good tenure, purchase, and financing decisions. In addition, The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 4

1. INTRODUCTION

for those people who do choose to become homeowners, the homebuyer education and

counseling should improve their financial capability (for example, budgeting and money management), thereby enhancing their ability to make timely mortgage payments, avoid foreclosure, and build home equity. Homebuyer education and counseling services should also improve the financial capability of those who choose to not to become homeowners, enhancing their ability to meet their financial goals.

However, these sustainable homeownership outcomes are best observed over a longer-term time horizon than this Short-Term Impact Report provides. Mortgage defaults tend to be low during the first year after origination, rising between years two and five (Stein et al., 2010). As a result, although we report on a small subset of possible sustainable homeownership measures, they are not the focus of this report. Rather, sustainable homeownership outcomes will be fully analyzed in the Long-Term Impact Report, looking at study participant outcomes at about 3½ to 5 years after study enrollment.

Homeownership education and counseling have another longer-term goal. For all service recipients, regardless whether they decide to purchase a home, the intervention should lead to enhanced long-term financial health by improving financial management, budgeting, and saving decisions. Indeed, for some people, the “right” choice might be not to buy a home, or to postpone buying a home until certain conditions are met. For that group, what is a good long-term outcome might be slightly different from those who do purchase; regardless, financial health is the goal.

Although we hypothesize that the intervention can have these favorable longer-term outcomes, this report focuses on short-term outcomes. The longer-term outcomes will be the subject of the study’s future, long-term followup analysis and report.

1.2 Two Perspectives on Impacts

In answering the primary research question, we consider two types of impacts.

• What are the impacts of being offered homebuyer education and counseling?

• What are the impacts of taking up the offer of homebuyer education and counseling?

The first question is about the offer of services (which includes the whole treatment group, including those who took up the offer and those who did not), and the relevant impact estimate is called the “intent-to-treat” (or ITT) impact. The second question is about taking up that offer (which focuses on just those treatment group members who took up the offer and participated in the study’s set of services), and the relevant impact estimate is called the “treatment-on-the-treated” (or TOT) impact.

1.3 Report Objectives and Study Timeline

This Short-Term Impact Report provides the first complete examination of impacts on outcomes since study participants enrolled in the First-Time Homebuyer Education and Counseling Demonstration. The report’s main objective is to present the short-term impacts of homebuyer The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 5

1. INTRODUCTION

education and counseling, both overall and for key subgroups, and by service delivery mode. The report also describes the study sample, the treatment group’s experiences with homeownership education and counseling services, and the sample’s outcomes at 12 to 18 months after study enrollment.

The Short-Term Impact Report follows two earlier reports on the Demonstration.

• The First-Time Homebuyer Education and Counseling Demonstration Baseline Report: Study Design and Implementation (the “Baseline Report”; DeMarco et al., 2017) provided a complete documentation of the Demonstration’s implementation, including detailing the evaluation design, describing the intervention’s operations, describing the study participants, and describing treatment group members’ experiences with the intervention and with the study.

• The First-Time Homebuyer Education and Counseling Demonstration: Early Insights (the

“Early Insights Report”; DeMarco et al., 2016) reported impacts on four outcomes observed 12 to 18 months after study enrollment for early entrants into the study. (Three of those impacts were statistically significant.) We revisit the impacts on these four outcomes for the full sample in this Short-Term Impact Report. What we found in the Early Insights Report is reinforced not for the overall impacts but for the specific service modes, as elaborated where we discuss the impact findings in chapter 6.

In addition, a special topics report is slated for release in the spring of 2018.

• Who Participates in Homebuyer Education and Counseling Services and Why? Insights from HUD’s First-Time Homebuyer Education and Counseling Demonstration (the “Participation Report”; Moulton et al., forthcoming) considers the characteristics of treatment group members that associate with their take-up and completion of services.

Exhibit 1.2 provides a timeline of the Demonstration from its inception through its Long-Term Impact Report.

Exhibit 1.2: Timeline of the HUD First-Time Homebuyer Education and Counseling Demonstration The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 6

1. INTRODUCTION

In September 2011, HUD contracted with Abt Associates to implement the Demonstration and conduct the evaluation. During the next 2 years, HUD and the Abt study team identified and recruited the necessary organizational partners—3 mortgage lenders, 63 local housing counseling agencies to provide in-person education and counseling services, and 2 national agencies to provide remote services. In the fall of 2013, the study piloted recruitment and random assignment in three sites. Full study enrollment across all 28 sites began in January 2014 and was completed in February 2016, enrolling a total of more than 5,800 participants referred to the study by the cooperating lenders. All of these study participants completed a baseline survey at the time of their study enrollment.

The study collects financial information through credit bureau data, lender loan and servicing data, and Federal Housing Administration (FHA) data on all enrolled participants every 6

months. In addition, we administered a followup survey (the “Short-Term Follow-Up Survey”) to study participants beginning at 12 months after their enrollment. The study also engaged 64

study participants in a series of 14 focus groups across four study sites to understand their experiences with participation.

We plan to conduct a second followup survey (a “Long-Term Follow-Up Survey”) between January 2019 and June 2020, subject to funding availability. A Long-Term Impact Report will be produced thereafter, providing analysis of impacts and outcomes measured about 3½ to 5 years after study enrollment.

1.4 Organization of This Report

The next three chapters provide context and lay the analytic foundation for reporting the impact findings:

• Chapter 2: Evaluation Design and Analysis describes the sample intake process and resulting study sample, the experimental evaluation design, service take-up rates, the impact evaluation questions, the analytic models and data sources, and the list of outcomes examined.

• Chapter 3: The Intervention: In-Person and Remote Homebuyer Education and Counseling describes the homebuyer education and counseling services offered to members of the treatment groups.

• Chapter 4: Prospective Homebuyers’ Experiences describes the sample’s home search and home purchase experiences and related outcomes.

The next three chapters report the findings on the impacts of homebuyer education and counseling. The findings include the overall impacts (both of being offered services and of taking up those services), impacts for selected subgroups, and impacts by service mode. Each chapter covers an outcome domain:

• Chapter 5: Impacts on Preparedness and Search.

• Chapter 6: Impacts on Financial Capability.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 7

1. INTRODUCTION

• Chapter 7: Impacts on Sustainable Homeownership.

The final chapter is a conclusion.

• Chapter 8: Conclusion and Implications summarizes the findings, considers the policy implications and generalizability of findings, and outlines the study’s future steps.

Detailed appendix material presents more detail on the study’s analytic methods, data sources and measures; expanded results beyond those presented in chapters 5, 6, and 7; additional subgroup analyses; and the question wording and response categories for the study’s the Short-Term Follow-up Survey.

• Appendix A: Analytic Methods.

• Appendix B: Data Sources and Measures.

• Appendix C: Expanded Results for the Overall Impact of Services.

• Appendix D: Expanded Results for Impacts by Service Delivery Mode.

• Appendix E: Impacts on Subgroups Defined by Baseline Characteristics.

• Appendix F: Impacts on Subgroups Defined by Likelihood of Service Participation.

• Appendix G: Impacts on Subgroups Defined by Likelihood of Home Purchase.

• Appendix H : Short-Term Follow-Up Survey.

The textbox on the next two pages defines some key terms used throughout the report.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 8

1. INTRODUCTION

Terminology Used in This Report

Terms Related to the Study’s Intervention

Intervention: Access to free homebuyer education and counseling services, either in-person or remote, offered to treatment group members.

Homebuyer education and counseling: These same services are sometimes called “pre-purchase” services because clients usually participate in them prior to purchasing a home and to differentiate these services from “post-purchase” services (for example, foreclosure prevention counseling). Because this study’s participants were at various stages in the homebuying process when they enrol ed, and because post-purchase services are not part of the study (and therefore there is no need to distinguish between pre- and post-purchase services), the study uses the broader “homebuyer” modifier.

• Education: Homebuyer education provides general information about buying a home either in a classroom workshop format or via an online program; and the content aligns with both the National Industry Standards and HUD’s standards .

• Counseling: Homebuyer counseling provides one-on-one guidance, either in-person or by telephone, tailored to the particular needs of the individual homebuyer; and its content aligns with both the National Industry Standards and HUD’s standards.

Housing counseling agencies: The HUD-approved agencies that provide the homebuyer education and counseling services.

Modes of service delivery: The two means by which homebuyer education and counseling could be accessed by study participants in one of the treatment groups—that is, either in person at a local housing counseling agency or remotely through the Internet and telephone.

Terms Related to Carrying Out the Study

Study participants: This is the label for all individual prospective first-time homebuyers who are enrolled in the study, regardless of the experimental group (treatment or control) to which they were randomly assigned.

• Treatment group members: The study participants who were offered the intervention (access to free homebuyer education and counseling) as part of study participation.

• Control group members: The study participants who were not offered the intervention (access to homebuyer education and counseling) as part of study participation. They represent the

“counterfactual,” which is what happens in the absence of the intervention.

Service recipients or recipients: The members of a treatment group who took up the offer of and used the homebuyer education and counseling services.

Outcomes: The specific constructs of interest that the intervention aims to influence.

Outcome domain: A category of outcomes. Each specific outcome in the study is part of one of three domains—preparedness and search, financial capability, or sustainable homeownership.

Terms Related to the Study’s Data Col ection

Baseline survey: Administered at study enrol ment, the survey that captured initial information about the study participants and their households. This survey’s response rate was 100%: that is, all study participants completed the survey.

Short-Term Follow-Up Survey: The survey fielded beginning 12 months after study participants enrol ed in the study. The average response time was 13.1 months. The survey covers a wide variety of topics that are the outcomes of interest to the study and also collected additional descriptive information. This survey’s response rate was 79%.

Follow-up period: For this Short-Term Impact Report, outcomes analyzed are measured—whether from survey or administrative data sources—to represent study participants’ experiences as of about 12 to 18

months after they enrol ed in the study.

Long-Term Follow-Up Survey: A second followup survey is slaed to be fielded in 2019. A Long-Term Impact Report will analyze impacts and outcomes measured about 3½ to 5 years after study enrollment.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 9

1. INTRODUCTION

Terminology Used in This Report (continued)

Terms Related to the Study’s Analyses

Impact: The change in outcomes that arises because of the intervention. Impacts reported in chapters 5

through 7 marked with one or more asterisks are statistically significant, indicating that it is unlikely that the impact is due to chance. Unless noted otherwise, we discuss only impacts that are statistically significant.

• ITT effect: The “intent-to-treat” (ITT) is the impact of being offered access to the intervention. It is computed by comparing the mean outcome for the entire treatment group (regardless whether they took up services) versus the mean outcome for the entire control group.

• TOT effect: The “treatment-on-the-treated” (TOT) is the impact of taking up the intervention. It is computed by rescaling the ITT estimate by the participation rate, and it relies on the assumption that any impact of the intervention on those who were offered services but did not use any (“no-shows”) was zero.

See the textbox Calculating Impact Two Ways (beginning on page 19) for additional detail on the ITT

and TOT terms.

Take-up: Treatment group members’ use of homebuyer education and counseling services. The “take-up rate”

is the proportion of each treatment group who used the services they were offered—meaning they talked by phone with a counselor or visited the online education materials (remote mode), or they attended a one-on-one, face-to-face counseling session or a group education workshop (in-person mode).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 10

2. Evaluation Design and Analysis

The First-Time Homebuyer Education and Counseling Demonstration evaluates the impact of homebuyer education and counseling on a wide variety of outcomes for low- to middle-income prospective first-time homebuyers. This chapter presents the intake of study participants and the characteristics of the study sample; the design of this experimental evaluation; the sample’s service take-up and completion rates; the impact evaluation’s research questions and how the design supports answering them; the analytic methods used to estimate the intervention’s impact overall, by mode, and by subgroup; the data sources; and the outcomes on which the impact estimates focus. Additional technical detail on analytic methods and data sources and measures can be found in appendixes A and B, respectively.

2.1 Enrollment and Characteristics of Study Participants

Study participants were recruited into the study via three major, national mortgage lenders. From among prospective homebuyers inquiring about a home purchase loan, those lenders made referrals to the study if the applicant reported being a low- to middle-income first-time homebuyer, lived in one of the study’s 28 sites, 13 and agreed that the lender could provide his/her contact information to the study team. The lenders contacted 136,874 customers and referred 18,279 to the study team. Of those referred to the study team, 79 were duplicate records or had insufficient data; 4,825 customers were screened out of the study; 5,702 were unreachable; 1,819 refused; and 5,854 consented to be in the study.

Exhibit 2.1 shows the 28 metropolitan areas selected for the study and the numbers of study participants ultimately recruited from each. Five metropolitan areas—Atlanta, Chicago, Dallas, Miami, and New York—each recruited 400 or more study participants, which combined account for about 40 percent of the total study sample.

These metropolitan areas span the country and offered a diverse range of housing affordability during the study period. For example, the average sales price of an existing home in San Jose in 2015 was $940,000, the highest of all U.S. metropolitan areas during the study period. However, the price in Atlanta was $173,600, well below the national average of $223,900 (National Association of REALTORS®, 2018b). The difference in these housing market conditions is also reflected in the National Association of Realtors Housing Affordability Index, which measures whether or not a “typical” family earns enough to qualify for a typical mortgage (higher index values signify greater affordability). 14 The 2015 index for the metropolitan statistical areas in our 13 These sites were chosen based on the three lenders’ loan application volume.

14 Specifically, a value of 100 means that the median income is exactly enough to qualify for a mortgage on a median-priced home, whereas an index value greater than 100 means that the median income is more than sufficient to qualify for a mortgage loan on a median-priced home (National Association of REALTORS®, 2018a,

“Methodology”).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 11

2. EVALUATION DESIGN & ANALYSIS

study ranges from a low of 63.1 in San Jose, CA to 250.7 in St. Louis, MO (National Association of REALTORS®, 2018a), indicating that housing is much more affordable in St. Louis than San Jose even though 2015 median family incomes were higher in the San Jose-Sunnyvale-Santa Clara, CA MSA ($106,300) as compared to the St. Louis, MO-IL MSA ($72,200) (FFIEC, 2015).

Exhibit 2.1: Study Locations and Sample Sizes

In addition to the income limit and the limit to first-time homebuyers, the evaluation imposed additional eligibility criteria during study enrollment. For example, we screened out referrals who did not have Internet access or access to transportation, or who stated they would not be willing to participate in services if assigned to a treatment group. We also screened out referrals who previously participated in homebuyer education and counseling or were participating in a downpayment assistance program that required homebuyer education and counseling. These factors—in addition to the fact that our participants’ first point of contact on the way to receiving homebuyer education and counseling was a lender, not the agency providing the education and counseling services—could result in a study sample that looks, on average, different from typical low-, moderate-, and middle-income prospective first-time homebuyers.

The sample also may differ from the typical clients of homebuyer education and counseling services. 15 For example, the sample for this study has a larger share of prospective homebuyers who are White and non-Hispanic than are typical clients served by the three national homebuyer 15 For a discussion of the potential generalizability of the study’s findings, given the sample composition and context, see DeMarco et al. (2017, chapter 5).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 12

2. EVALUATION DESIGN & ANALYSIS

education and counseling service providers (DeMarco et al., 2017), and the study sample has somewhat higher average incomes than those typical clients.

This study’s findings will be most relevant to populations that are similar to the study sample at baseline: low-, moderate-, and middle-income households that have contacted a mortgage lender about acquiring a mortgage for a first-time home purchase. The study sample includes a large number of participants, who vary in their sociodemographic composition and represent prospective homebuyers with a wide variety of traits. Exhibit 2.2 summarizes the sample’s baseline characteristics; they are detailed in the study’s Baseline Report (DeMarco et al., 2017).

Exhibit 2.2: Study Sample Characteristics at Baseline

Baseline Variable

Study Sample

Race/Ethnicity of Study Participant

Hispanic

25.1

White non-Hispanic

38.5

African-American non-Hispanic

20.5

Asian non-Hispanic

12.1

Other

3.9

Male

60.2

Marital Status of Study Participant

Married

38.2

Divorced, widowed, or separated

14.8

Single and never married

47.0

Plans to purchase the home with a co-borrower

26.2

Household Size

One

22.7

Two

32.0

Three

19.8

Four or more

25.5

Education of Study Participant

Bachelor’s degree or higher

53.4

Associate’s degree

12.9

Some college, but no degree

16.1

High school diploma or less

17.6

Employment

Full-time employment (30+ hours per week)

89.9

Part-time employment (1-29 hours per week)

4.1

Unemployed and looking for work

0.5

Not working, homemaker, retired, student, or other

5.5

Income Received by Study Participant and Any Co-Borrowers in Last 12 Months

$24,999 or less

8.1

$25,000 to $49,999

34.0

$50,000 to $74,999

32.7

$75,000 to $99,999

14.6

$100,000 or more

10.6

Mean ($)

59,510

Median ($)

54,000

Notes: Al measures are shown for the full sample of 5,770 study participants (after excluding study participants who withdrew from the study). Measure-specific sample sizes may vary due to item nonresponse. Due to rounding, not al reported percentages precisely equal 100.0 percent.

Source: Baseline survey of study participants

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 13

2. EVALUATION DESIGN & ANALYSIS

The study sample is racial y and ethnical y diverse, with 12.1 percent self-identifying as Asian; 20.5

percent as African American, non-Hispanic; 25.1 percent as Hispanic; and 38.5 percent as White non-Hispanic. Approximately three-fifths (60.2 percent) of study participants are men, and two-fifths (39.8 percent) are women.

The study participants reflect a wide range of educational attainment, with a slight majority of participants holding bachelor’s degrees (53.4 percent). Most participants (89.9 percent) were working full-time (that is, at least 30 hours per week) as of the study’s baseline survey. The median income for study participants and their co-borrowers was $54,000 in the 12 months prior to study enrol ment, with 10.6 percent making more than $100,000 and 8.1 percent making less than $25,000. Relatedly, the median income for study participants and their co-borrowers varied notably across the study’s 28

sites. For instance, study participants who enrol ed from Orlando, FL had a median income of $36,000, whereas those who enrol ed from San Jose, CA had a median income of $101,000.

2.2 The Evaluation Design

The study used a randomized experimental design to answer the study’s research questions. Eligible prospective first-time homebuyers were randomly assigned to a control group or to a treatment group. Members of the treatment group were offered free homebuyer education and counseling services (“the intervention”), the details of which are the subject of chapter 3. Members of the control group were not offered services through the study.

The randomization process ensures that there are no systematic differences between the treatment group and the control group, 16 except for the treatment offer. As such, differences in the mean outcomes between the groups can be at ributed to the intervention as its “impact.”

The study had two phases that affected the intervention for treatment group members—the Initial Study Design and the Modified Study Design, as elaborated next.

2.2.1 Initial Study Design: Control Group Plus Remote and In-Person Treatment Groups Starting in September 2013, the study began enrolling eligible prospective first-time homebuyers and randomly assigning them into one of three groups:

• Control group—Not offered homebuyer education or counseling services through the study.

• Remote treatment group—Offered the study’s free online homebuyer education and telephone counseling.

• In-person treatment group—Offered the study’s free in-person homebuyer education and counseling.

16 DeMarco et al. (2017) reports a baseline balance test, confirming that this is indeed the case.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 14

2. EVALUATION DESIGN & ANALYSIS

Those eligible had a 42-percent chance of being randomized into the control group and a 29-percent chance of being randomized into one of the two treatment groups. 17

Study participants assigned to a treatment group were referred to one of the study’s participating housing counseling agencies that provided the assigned mode of services (in-person or remote); overal , 55.1 percent took up that offer of services. The study did not refer control group members to homebuyer education and counseling services, but members were not prevented from accessing similar services on their own. Some control group members reported accessing homebuyer education and counseling, but it is not clear that those services are comparable to what they would have received had they gained access to the services that the treatment groups had (points on which we elaborate in section 2.3 and appendix A).

In the first year of the Demonstration’s implementation, the study team monitored the rate at which treatment group members took up the free homebuyer education and counseling services offered to them (their “take-up rate”). The team found that a relatively smal proportion (about one-quarter) of treatment group members offered in-person services took up services. Having such a large share of no-shows in this treatment group implied low power to detect the effect of being offered in-person services. In response, HUD and the study team decided to modify the study design.

2.2.2 Modified Study Design: Control Group Plus Remote and Choice Treatment Groups In September 2014, the protocol for assignment to the treatment groups was modified, replacing the in-person treatment group with a “choice” treatment group. As its name implies, study participants assigned to the choice treatment group would be able to choose between accessing services remotely through online education and telephone counseling or accessing them in person at one of the study’s local housing counseling agency.

Study participants enrol ed in the study on or after September 16, 2014, were randomly assigned to one of these three groups:

• Control group—Not offered homebuyer education or counseling services through the study.

• Remote treatment group—Offered the study’s free online homebuyer education and telephone counseling.

• Choice treatment group—Offered either the study’s free remote homebuyer education and counseling services or its free in-person services.

In addition, the baseline survey was amended to add a question asking all study participants whether they would prefer to receive services remotely or in person, should they be randomized into treatment. Their response did not influence how they were assigned to the control and treatment 17 This ratio was chosen to balance the study’s ability to detect differences (1) between the pooled treatment group and the control group and (2) between each treatment group and the control group.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 15

2. EVALUATION DESIGN & ANALYSIS

groups, however. That is, regardless of their stated preference, study participants were assigned into one of the three groups randomly. Among those randomized into the choice treatment group, we used stated baseline preference data to determine which mode of services to offer them: choice treatment group members who indicated on the survey that they would prefer to receive services remotely were offered free online homebuyer education and telephone counseling. Choice group members who indicated on the survey that they would prefer to receive services in person were offered in-person education and counseling. As such, the choice treatment group represents a service provision world in which people are offered services in line with their stated baseline preference. 18

Exhibit 2.3 displays the study’s sample according to timing of randomization to the experimental groups and shows how the control group was matched to the segmentation of the groups according to stated baseline preference. Under the Modified Study Design, we have baseline preference data for al study participants, including those in the control and remote treatment groups, as well as the choice treatment group. Having these preference data for all study participants allows us to conduct an experimental comparison between participants in the choice treatment group with a preference for in-person services versus participants in the control group with a preference for in-person services.

The same comparison was possible for those with a preference for remote services.

Having the preference data also al ows us to conduct an experimental comparison between members of the remote treatment group assigned after the study design was modified who indicated a preference for remote services versus their counterparts in the modified study design control group with a preference for remote services.

Exhibit 2.3: Study Sample, by Group and Time Period

a Study participants whose stated baseline preference on the amended survey was for in-person services.

b Study participants whose stated baseline preference on the amended survey was for remote services.

c Group total includes all study participants, including those for whom a stated baseline preference is missing.

Notes: This exhibit excludes study withdrawals, and as such, reflects the sample available for analysis. Purple shading indicates “no services offered”; teal shading indicates “in-person services offered”; and grey shading indicates “remote services offered.”

Source: Study’s random assignment and service tracking system

18 Choice treatment group members were permitted to enroll in their choice of service mode, regardless of their originally stated preference. In total, 39 choice treatment group members enrolled in remote services after random assignment having previously expressed a preference for in-person services; and one choice treatment group member enrolled in in-person after random assignment having previously stated a preference for remote services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 16

2. EVALUATION DESIGN & ANALYSIS

2.3 Service Take-up and Completion Rates

Overall, 55.1 percent of treatment group members offered the study’s free homebuyer education and counseling services took up services—that is, they used either the education curriculum or one-on-one counseling (exhibit 2.4).

The rates at which treatment group members took up services and completed services differ meaningfully between the in-person and remote service modes, with both take-up and completion rates much higher for those offered remote services than for those offered in-person services. Almost two-thirds of those offered remote services (63.8 percent) took up online education or telephone counseling. In contrast, about one-quarter (28.0 percent) of those offered in-person services took up an in-person education workshop or in-person counseling. Moreover, 28.5 percent of those offered remote services completed all homebuyer education and counseling services, whereas 14.6 percent of those offered in-person services completed all services.

Exhibit 2.4: Take-Up and Completion Rates, by Offer of In-Person or Remote Services Took Up Any

Completed All

Services

Services

(%)

(%)

Sample Size

In-person treatment group

25.8

14.2

515

Choice treatment group (in-person preference)a

32.0

15.5

291

In-person treatment group and choice treatment

group (in-person preference)

28.0

14.6

806

Remote treatment group

63.4

27.1

1,665

Choice treatment group (remote preference)b

64.6

31.1

851

Remote treatment group and choice treatment group

(remote preference)b

63.8

28.5

2,516

Full treatment group sample

55.1

25.1

3,322

a Includes choice treatment group members who stated a baseline preference for in-person services. Choice treatment group members were permitted to change their preference after random assignment. In total, 39 choice treatment group members changed preference from in-person to remote after random assignment.

b Includes choice treatment group members who stated a baseline preference for remote services. Choice treatment group members were permitted to change their preference after random assignment. In total, one choice treatment group member changed preference from remote to in-person after random assignment.

Note: Study participants who withdrew from the study are excluded.

Sources: Take-up data from eHome America, ClearPoint, and local housing counseling agencies 2.4 Impact Evaluation Questions and Contrasts

This section summarizes three main sets of evaluation questions—overall, subgroup, and delivery mode—and then identifies which samples we used to create the evaluation contrasts to answer these questions.

2.4.1 Primary Research Questions

A primary objective of this report is to answer the research question: what are the impacts of homebuyer education and counseling on low-, moderate-, and middle-income prospective first-time homebuyers? In answering this overarching research question, we consider two types of impacts: the impacts of being offered homebuyer education and counseling; and the impacts of The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 17

2. EVALUATION DESIGN & ANALYSIS

taking up the offer of homebuyer education and counseling. The impact of being offered services (which includes the whole treatment group, including those who took up the offer and those who did not) is called the “intent-to-treat” (or ITT) impact; and the impact of taking up services (which focuses on just those who took up the offer) is called the “treatment-on-the-treated” (or TOT) impact. The textbox Calculating Impact Two Ways beginning on page 19 provides additional detail on these alternative estimates.

Regardless of whether we are considering the ITT impact or the TOT impact, to asses overall impacts, we pool the treatment groups and compare overall treatment group mean values and control group mean values for the outcomes of interest. Given our experimental research design, if we find that study participants who were offered homebuyer education and counseling services have, for example, higher average homeownership rates than do participants not offered services, then we can confidently conclude that the difference in homeownership rates is the causal impact of being offered homebuyer education and counseling services. (That is, the difference was caused by the intervention.)

Chapters 5, 6, and 7 report the intervention’s overall impact in each outcome domain: preparedness and search; financial capability; and sustainable homeownership.

2.4.2 Impacts for Subgroups

This report also considers the research question: what are the impacts of offering homebuyer education and counseling on select subgroups? To address this question, we estimate the intervention’s impacts for subgroups defined by baseline demographic and socioeconomic characteristics of the individual study participants, as well as by housing characteristics of the area in which they lived at baseline. The specific subgroups were identified for analysis during the study’s design phase in consultation with HUD and the study’s advisory team and expert panel. This subgroup analysis is motivated by interest in providing information to policymakers and practitioners about targeting and providing services.

We analyze the subgroup impacts by pooling all of the sample assigned to any treatment group with the subgroup characteristic and comparing their mean outcomes to those of the control group with the subgroup characteristic. All of the subgroups results are reported in appendix E, with some results reported in chapters 5, 6, and 7.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 18

2. EVALUATION DESIGN & ANALYSIS

Calculating Impact Two Ways:

Understanding Intent-to-Treat (ITT) versus Treatment-on-the-Treated (TOT)

This study reports two kinds of impact estimates: the “intent-to-treat” (ITT) impact and the “treatment-on-the-treated” (TOT) impact, each of which is relevant to a different question. The ITT provides an estimate of the impact of being offered homebuyer education and counseling services, regardless whether those treatment group members did or did not take up services. In contrast, the TOT provides an estimate of the impact of actually taking up services. Of those offered services, 55.1 percent of treatment group members took up that offer and used the study’s set of services; the other 44.9 percent did not take up the offered services (the no-shows).

Intent-to-Treat (ITT) Impacts

Because the treatment and control group members were assigned to their groups randomly, we can assume that the only systematic difference between the groups is that the treatment group members were offered the opportunity to participate in the study’s services. As is standard practice, the study confirmed analytical y that the treatment and control groups are otherwise similar; it also used baseline characteristics of the sample members in the multiple regressions that produce the outcome estimates, in order to increase the precision of the estimates.

Because of the random assignment to treatment and control groups, we can conclude that any difference between the outcomes of the treatment group and the control group that passes a test of statistical significance was caused by the offer of services. This impact estimate includes both the offer of service for al members of the treatment group and the actual use of services for those who took it up. We calculate the impact of making those services available as the mathematical difference between the treatment group’s mean outcome and the control group’s mean outcome. For example, any difference between the treatment group’s average rate of preparing and monitoring a budget and the control group’s average rate would be caused by the intervention.

This mathematical difference in mean outcomes between all the treatment group members (those who took up the services and those who did not) versus the control group is called the “intent-to-treat” (ITT) impact. ITT

analysis is meant to capture the combined impact of both the degree to which individuals decide to use services and the effectiveness of those services for those who took them up. In this study, we can interpret the ITT

estimate as the causal impact of making homebuyer education and counseling services available but not mandatory.

Treatment-on-the-Treated (TOT) Impacts

The “treatment-on-the-treated” (TOT) impact describes the impact the intervention had on only those individuals who took up services. Therefore, the TOT estimate could be especially relevant for programs interested in understanding their impacts on clients who use services. In addition, the TOT impact is relevant to understanding the impact of policies or programs that require prospective homebuyers to participate in homebuyer education and counseling services—for example, as a condition of mortgage loan approval—as opposed to simply making those services available.

To compute the TOT estimate, we divide the ITT estimate (that is, the difference between the average outcome for the treatment and control groups) by the treatment group’s take-up rate. To ascertain the TOT estimate’s statistical significance, the standard error is also divided by the take-up rate (Bloom, 1984). In practice, the computation uses multiple regression.a In carrying out this analysis, we assume that there is no impact on treatment group members who did not take up the intervention’s services (“no-shows”) and that there are no

“crossovers” (control group members who somehow receive any of the Demonstration’s homebuyer education and counseling services).b Given these assumptions, we can interpret the TOT estimate as the causal impact of taking up homebuyer education and counseling services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 19

2. EVALUATION DESIGN & ANALYSIS

Calculating Impact Two Ways (continued)

What This Means for Interpreting the Study’s Impact Findings

Importantly, because a TOT impact estimate is computed by dividing the ITT estimate and corresponding standard error by a constant (by the treatment group’s take-up rate), the ITT impact analysis and the TOT

impact analysis inevitably yield the same pattern of results. The sign of the ITT and TOT estimates (that is, whether the impact is positive or negative, or favorable or unfavorable) wil always be the same. The level of statistical significance of the ITT and TOT estimates also wil always be the same. Although the sign and statistical significance of the ITT and TOT impacts wil be the same, the magnitude of the TOT estimate wil always be greater. This is because ITT analysis distributes any difference in outcomes between the treatment and control groups over all treatment group members, whereas the TOT analysis distributes that difference over only the subset of treatment group members who receive services.

As shown by Bloom (1984), in moving from the ITT estimate to the TOT estimate, the standard error of the estimate is adjusted by the same factor as the impact estimate itself (that factor being the take-up rate); consequently, this means that the t-statistic (which determines statistical significance) of the estimate is unchanged by this transformation. Therefore, any observed ITT estimate that is statistically significant wil have a corresponding TOT estimate that also is statistically significant. Similarly, where an ITT impact is not statistically significant, the corresponding TOT impact also wil not be statistically significant, even though its magnitude is greater.

In this report, the discussion regarding the pattern of results for any finding—for example, whether a particular impact is favorable or unfavorable—is based on the sign and statistical significance level of that finding, which are always the same, whether the ITT or the TOT estimate. When reporting magnitude, we start with the ITT

estimate, because it is purely experimental. Because of successfully executed random assignment, ITT

estimates reflect causal impacts of being offered the intervention’s services. TOT results, by contrast, are not purely experimental, in that they require the assumption of no effect of the intervention on no-shows and no crossovers. When that assumption is credible (as is the case in this study), we also can have confidence in the quality of the TOT estimates to represent the impacts of the intervention on those who took up services. We report both ITT and TOT estimates in exhibits showing impact estimates, and we let readers decide which magnitude is of greater relevance to them, and also based on their comfort level with the no-show assumption’s plausibility.

Where to Find More Information

Interested readers can find additional information related to the computation and presentation of ITT and TOT

estimates in this report:

• Additional detail on how ITT estimates are computed appears in appendix A (section A.1).

• Additional detail on how TOT estimates are computed appears in appendix A (section A.2).

• Appendix C reports overal impacts, including ITT and alternative TOT estimates.

• Appendix D reports effects by mode (remote versus in-person education and counseling), including ITT and alternative TOT estimates.

______________

a See appendix A (section A.2) for a detailed description of the regression framework used to compute the study’s TOT

estimates.

b We believe it is plausible that the intervention had no effect on those who did not take up services. Additionally, there were multiple barriers to control group members accessing the same homebuyer education and counseling services offered to treatment group members, making crossovers unlikely, as elaborated in appendix A (section A.2). There might have been some control group members, however, who despite not having been offered them by the study, did receive some kind of homebuyer education and counseling services through other, non-study sources. The appendix to this report includes alternative TOT estimates based on alternative assumptions, including use of the information we have on control group members who reported receiving homebuyer education and counseling services. Importantly, these alternative TOT estimates retain the same sign and statistical significance as the ITT and TOT estimates reported in the body of the report, and therefore do not alter the study’s basic findings.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 20

2. EVALUATION DESIGN & ANALYSIS

2.4.3 Impacts by Delivery Mode

This report also considers the following research questions: what are the program’s impacts by service delivery mode? and to what extent do these impacts of service delivery mode differ from one another? As permitted by randomization into multiple treatment groups, we estimate the intervention’s impacts by service delivery mode:

• The impact of in-person homebuyer education and counseling services, computed as the difference in mean outcomes between treatment group members offered in-person services and their control group counterparts.

• The impact of remote homebuyer education and counseling services, computed as the difference in mean outcomes between treatment group members offered remote services and their control group counterparts.

• Comparison of the impact of in-person and remote services, computed as the difference between the impacts of in-person and remote services.

Because of the smaller sample sizes, it is more difficult to detect statistically significant impacts for each service delivery mode independently than to detect effects for the overall impact of services. The textbox Understanding Null Effects on page 52 explains why. Similarly, it is more difficult to detect statistically significant impacts for in-person services than for remote services because the sample size available for estimating the impact of in-person services is smaller still. The implication of this is that the impact of in-person services has to be relatively large to be detected. In other words, the in-person intervention has a “higher bar” for attaining statistical significance than does the remote intervention.

Chapters 5, 6, and 7 highlight these mode effect results. Appendix D reports them all, including results that explore the possible effect of providing a choice of service mode. 19

In order to address these evaluation questions, we compare the treatment and control groups’

outcomes, including overall, by subgroup, and by mode. We use the treatment and control groups in ways that respect and retain the experiment’s random assignment of individuals into their respective experimental groups, combining certain portions in order to maximize the sample size available to answer each question. Exhibit 2.5 shows how we used the groups to produce experimental estimates of overall impacts, subgroup impacts, and impacts by delivery mode.

Exhibit A.2 provides additional detail.

19 The redesign of the study, changing the in-person treatment group to a “choice” treatment group, provided an opportunity to estimate the impact of being offered a choice of service modes. The results reported in appendix D

address the question of whether choice matters by comparing the impact of being given a choice of service modes versus the impact of being offered remote services without a choice.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 21

2. EVALUATION DESIGN & ANALYSIS

The second and third panels of exhibit 2.5 show how we used the sample to estimate separately the impact of in-person services and remote services, respectively. Recall that the September 2014 study redesign changed the in-person treatment group to a “choice” treatment group, including the addition to the baseline survey of a question on service mode preference. Because all study participants who enrolled in the study after the redesign were asked about their preference, we are able to identify the control group counterparts to the choice treatment group members who stated that same baseline preference. Their control group counterparts are the control group members who enrolled in the study after the study design’s modification who had a stated baseline preference for in-person services or for remote services and therefore would have been offered that mode had they been randomly assigned to the choice treatment group.

Exhibit 2.5: Experimental Samples Used to Answer Each Impact Evaluation Question a Study participants whose stated baseline preference was for in-person services.

b Study participants whose stated baseline preference was for remote services.

Notes: This exhibit excludes study withdrawals; as such, it reflects the sample available for analysis. Purple shading indicates “no services offered”; teal shading indicates “in-person services offered”; and grey shading indicates “remote services offered.”

Source: Study’s random assignment and service tracking system

As detailed in the following, we include the choice treatment group members when estimating the impact of in-person and remote services—the choice treatment group members (and their control group counterparts) who, prior to random assignment, had expressed a preference for in-The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 22

2. EVALUATION DESIGN & ANALYSIS

person or remote services, respectively. This strategy retains the integrity of the experimental design and increases the sample size available for estimating mode effects, which provides a greater opportunity for the study to detect differential mode effects, if they exist. As shown in exhibit 2.5 and described in more detail in appendix section A.3, we combine various subsets of our experimental sample to produce experimental estimates of the impact of in-person services and remote services as follows:

• The impact of in-person services is computed as a weighted combination of the following two experimental contrasts: (1) comparison of in-person treatment group versus Initial Study Design control group; and (2) comparison of choice treatment group members with a stated baseline preference for in-person services versus Modified Study Design control group members with a stated baseline preference for in-person services.

• The impact of remote services is computed as a weighted combination of the following two experimental contrasts: (1) comparison of the remote treatment group versus the full control group; and (2) comparison of choice treatment group members with a stated baseline preference for remote services versus Modified Study Design control group members with a stated baseline preference for remote services.

2.5 The Analytic Process for Computing Impacts

As detailed in the textbox Calculating Impact Two Ways (beginning on page 19), this study conducts two types of impact analyses: ITT analysis, which measures the impact of being offered homebuyer education and counseling, and TOT analysis, which measures the impact of taking up the offer to participate in homebuyer education and counseling. Because the study uses an experimental evaluation design, the ITT impacts are estimated as the difference between the treatment group’s mean outcome and the corresponding control group’s mean outcome.

As is common practice in program evaluation, we use a multiple regression to estimate all reported impacts. 20 Details of the model specification appear in appendix A. The main reason for using multiple regression rather than simply taking the simple numeric difference in mean outcomes is that multipole regression permits taking into consideration small baseline differences (“noise”) between the treatment and control groups. In doing so, we can more precisely estimate the impact, decreasing the size of the impact that is detectably different from zero.

Conceptually, the impact of taking up services (the TOT estimate) is equal to the ITT estimate divided by the difference in treatment and control group take-up rates. In the exhibits where we 20 Exhibit B.3 in appendix B describes the baseline covariates included in the impact analysis model.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 23

2. EVALUATION DESIGN & ANALYSIS

report impact results, we refer to the ITT impacts as the “Impact of Being Offered Services,” and we refer to the TOT impacts as the “Impact of Taking Up Services.” 21

As elaborated in appendix A, estimating the TOT impacts involves making some assumptions and drawing on additional data sources regarding take-up of services. Because the study has two main sources of data on take-up—the service agencies’ administrative data and the Short-Term Follow-Up Survey—we compute two alternative TOT estimates:

• We prefer the administrative data as the source for who, within a treatment group, took up services because those data offer external validation of whether a treatment group member participated in services. The TOT impacts reported in the chapters are based on the administrative measure of take-up and reflect an adjustment for no-shows within the treatment group, assuming (1) that there are no impacts on no-shows, and (2) that there are no crossovers.

• In comparison, the followup survey data include measures of self-reported take-up for both treatment and control group members. However, that source embeds two important kinds of measurement error: recall error and differential understanding. Importantly, the survey data reflect different understanding between treatment and control group members about what it means to participate in homebuyer education and counseling services. The treatment group’s understanding is more likely to align with the specific set of services offered in the Demonstration, whereas the control group’s understanding is more ambiguous. This

asymmetry makes us wary of using the survey data to compute TOT estimates, although we do report those results in the various appendixes for interested readers.

Chapters 5, 6, and 7 report the study’s impacts on the three outcome domains of preparedness and search, financial capability, and sustainable homeownership, respectively. Within each of these chapters, we present overall impacts, impacts for selected subgroups, and impacts by service delivery mode. In reporting the study’s findings, we focus on the pattern of results, which is the same whether one prioritizes the ITT or TOT estimates. In order to provide context for the study results, we include information about the broader housing market where such data are available and relevant.

2.6 Data Sources

This evaluation uses a wide variety of primary and secondary data sources in its analyses:

• Two surveys of study participants, a baseline survey and the Short-Term Follow-Up Survey.

The baseline survey captures the characteristics of study participants at the time of study 21 For more detail on How to Read the Impact Exhibits in This Report, see that textbox on page 51.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 24

2. EVALUATION DESIGN & ANALYSIS

enrollment. The followup survey was initiated at 12 months after study enrollment and captures outcomes observed approximately 12 to 18 months after random assignment. 22

• Credit data on study participants from one of the three major credit bureaus. The study team collected credit bureau data every 2 months during the enrollment period to capture study participants’ baseline credit attributes 0 to 2 months prior to their enrollment in the study. We then routinely collected credit bureau data during the followup period to capture outcome measures for the impact analyses. For this report, these data capture study participants’

outcomes 12 to 18 months after random assignment. 23

• Loan origination and servicing data from participating lenders and the Federal Housing Administration (FHA). These data were collected on the same schedule as followup credit data, capturing study participants’ outcomes 12 to 18 months after random assignment.

• Data on services that treatment group members received from eHome America, ClearPoint, and the 63 local housing counseling agencies. These data capture whether study participants participated in homebuyer education and counseling services within 12 months of enrolling in the study.

• Focus groups with treatment group members in four locations during the final 5 months of the study enrollment period, which corresponds to September 2015 through February 2016.

These data are used to explore study participants’ experiences with homebuyer education and counseling services and the homebuying process.

2.7 Outcomes for the Evaluation’s Impact Analyses

The study categorizes outcomes as confirmatory, secondary, or exploratory as a means to focus the analyses and protect the integrity of the interpretation of statistical tests. With a large number of outcomes, there is a high likelihood that at least one of the outcomes will show a significant impact purely as a result of chance. 24 Categorizing outcomes as confirmatory, secondary, or 22 Among the 79 percent of the study sample who replied to the Short-Term Follow-Up Survey, 93 percent replied between 12 and 18 months after the month they were randomly assigned. The average was 13.2 months after random assignment, and the median was 13 months. Exhibit B.1 provides more detail on the timing of study participants’ responses to the survey.

23 Credit bureau data on housing outcomes and loan performance from these followup extracts cover 92.7 percent of the study sample and provide outcome measures for the impact analyses. We observed study participants’ credit bureau data an average of 14.3 months after random assignment, with a median of 14 months.

24 This problem pertains to the potential for drawing a false positive conclusion that increases as a result of making multiple hypothesis tests. If the statistical test for each impact estimate is based on a 95-percent confidence interval, for example, then 1 in every 20 random samples will show a statistically significant “impact” purely as a result of chance. As a result, an evaluation that tests for statistically significant impacts on a large number of outcomes faces a high likelihood that at least one of the outcomes will show a significant impact due to chance. By identifying the specific confirmatory outcome a priori, this approach designates a “central” outcome to provide stronger evidence on a main question and designates other outcomes as secondary or exploratory.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 25

2. EVALUATION DESIGN & ANALYSIS

exploratory helps to mitigate this problem by identifying a narrow set of outcomes that are most important to the study and treating other outcomes as less definitive. The study’s categories are as follows:

• The confirmatory outcome is the main indicator of the extent to which the program is effective in the long term. The evaluation set one key outcome— the 60-day mortgage delinquency rate—as the single confirmatory outcome for the study. The confirmatory outcome is included in all impact analyses. Estimating the impact of homebuyer education and counseling services on the 60-day delinquency rate provides evidence on whether homebuyer education and counseling services lead to improved loan performance. However, at this short-term followup time point, we would expect to see relatively low 60-day delinquency rates because relatively little time will have passed since purchase of a home.

Therefore, this outcome is more relevant to the study’s Long-Term Impact Report, which will report the longer-term impact of homebuyer education and counseling services on outcomes measured about 3½ to 5 years after study enrollment.

• Secondary outcomes are additional important indicators tied to the logic of the intervention in its efforts to influence outcomes in the three domains (preparedness and search, financial capability, and sustainable homeownership). Secondary outcomes are included in all impact analyses.

• Exploratory outcomes are of two types: (1) alternative specifications of secondary outcomes; and (2) additional outcomes of interest that are less directly tied (or are more ambiguously tied) to the logic of the intervention but that still might be influenced by the program.

Exploratory outcomes are included in the analyses related to the overall impact of homebuyer education and counseling and to the impacts of the two service delivery modes. Exploratory outcomes are selectively included in the study’s subgroup analyses based on whether a story emerged for these outcomes in the analyses of overall impacts or mode effects.

Exhibit 2.6 lists the outcomes in each of these categories. It also groups outcomes into the study’s three outcome domains; defines each outcome; identifies the source of the measure; and reports the mean outcome for the sample as a whole, the standard deviation, and the number of study participants with nonmissing data. Each outcome for the impact analysis is defined for the full study sample (as opposed to, say, the subsample of purchasers). Additional details on the operationalization of each outcome are included in appendix B.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 26

2. EVALUATION DESIGN & ANALYSIS

Exhibit 2.6: Outcomes for Impact Analysis: Confirmatory, Secondary, and Exploratory Full Sample Mean

(Standard Deviation)

Domain

Outcome Definition

Data Source(s)

[Sample Size]

Category: Confirmatory Outcome

Sustainable

Ever 60 days delinquent (%)

Short-Term Follow-Up Survey;

0.6

homeownership

credit bureau data; lender data;

(7.9)

FHA data

[5,708]

Category: Secondary Outcomes

Preparedness and Study participant purchased a

Short-Term Follow-Up Survey;

62.1

search

home (%)

credit bureau data; lender data;

(48.5)

FHA data

[5,708]

Study participant was confident that Short-Term Follow-Up Survey

72.3

he/she could find the information

(44.7)

he/she needed about the

[4,543]

homebuying process (%)

Financial

Number of cor ect answers (out of

Short-Term Follow-Up Survey

2.75

capability

four) to mortgage literacy quiza

(0.88)

[4,546]

Credit score (out of 850)

Credit bureau data

707.7

(77.7)

[5,308]

Study participant has a credit score Credit bureau data

87.6

above or equal to 620 (%)

(32.9)

[5,308]

Total nonhousing debt ($)

Credit bureau data

23,835

(30,185)

[5,346]

Study participant has a budget and Short-Term Follow-Up Survey

34.6

often compares it against actual

(47.6)

spending (%)

[4,524]

If in financial difficulty, the study

Short-Term Follow-Up Survey

39.1

participant would contact his/her

(48.8)

lender for assistance prior to

[4,504]

missing a mortgage payment (%)

If in financial difficulty, the study

Short-Term Follow-Up Survey

19.4

participant would contact his/her

(39.5)

counseling agency, consumer credit

[4,368]

counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage payment (%)

Sustainable

Ratio of monthly housing costs to

Short-Term Follow-Up Survey

24.7

homeownership

monthly income

(18.9)

[4,207]

Ever 30 days delinquent (%)

Short-Term Follow-Up Survey;

2.3

credit bureau data; lender data;

(14.9)

FHA data

[5,708]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 27

2. EVALUATION DESIGN & ANALYSIS

Full Sample Mean

(Standard Deviation)

Domain

Outcome Definition

Data Source(s)

[Sample Size]

Category: Exploratory Outcomes

Preparedness and Study participant was very satisfied Short-Term Follow-Up Survey 30.3

search

with the homebuying process (%)

(46.0)

[4,505]

Number of lenders from which the

Short-Term Follow-Up Survey

1.60

study participant received price

(1.44)

quotes

[4,428]

Financial

Total monthly debt-to-income ratio

Short-Term Follow-Up Survey;

26.1

capability

(back-end ratio)

credit bureau data; lender data;

(17.2)

FHA data

[5,304]

Total monthly debt-to-income ratio

Short-Term Follow-Up Survey;

16.5

(back-end ratio) greater than 0.43

credit bureau data; lender data;

(37.1)

(%)

FHA data

[5,304]

Study participant occasional y does Short-Term Follow-Up Survey

10.1

not have enough money to cover all

(30.1)

of the bil s at the end of the month

[4,538]

(%)

Student loan balance ($)

Credit bureau data

9,972

(23,787)

[5,346]

Credit card balance ($)

Credit bureau data

3,797

(5,313)

[5,346]

Study participant usually has

Short-Term Follow-Up Survey

72.2

enough savings set aside to cover 3

(44.8)

months of expenses (%)

[4,535]

Study participant has tried to figure Short-Term Follow-Up Survey

79.2

out how much he/she needs to

(40.6)

save for retirement (%)

[4,511]

Study participant never uses

Short-Term Follow-Up Survey

86.2

payday lenders (%)

(34.5)

[4,404]

Study participant usually pays credit Short-Term Follow-Up Survey

72.0

card balance in full to avoid interest

(44.9)

charges (%)

[4,510]

Study participant often saves

Short-Term Follow-Up Survey

63.3

money (%)

(48.2)

[4,539]

Total savings and investments ($)

Short-Term Follow-Up Survey

47,100

(77,211)

[4,295]

Study participant knows how to

Short-Term Follow-Up Survey

78.0

correct inaccurate information in

(41.4)

credit report (%)

[4,496]

Study participant has an electronic Short-Term Follow-Up Survey

55.1

or written budget (%)

(49.7)

[4,529]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 28

2. EVALUATION DESIGN & ANALYSIS

Full Sample Mean

(Standard Deviation)

Domain

Outcome Definition

Data Source(s)

[Sample Size]

Regularly required mortgage

Short-Term Follow-Up Survey

28.0

payment is automatically deducted

(44.9)

from a bank account (%)

[4,540]

If study participant started having

Short-Term Follow-Up Survey

82.3

financial problems and could not

(38.2)

pay all of the bil s, he/she would

[4,520]

pay the mortgage first (%)

Nonhousing debt bankruptcy,

Credit bureau data

4.7

foreclosure, or repossession

(21.2)

indicator (%)

[5,369]

Sustainable

Monthly housing costs exceed 30

Short-Term Follow-Up Survey

23.5

homeownership

percent of monthly income (%)

(42.4)

[4,207]

Ever 90 days delinquent (%)

Short-Term Follow-Up Survey;

0.3

credit bureau data; lender data;

(5.3)

FHA data

[5,708]

FHA = Federal Housing Administration

a For details, see the textbox Mortgage Literacy Quiz on page 633.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 29

3. The Intervention: In-Person and Remote Homebuyer Education

and Counseling

The intervention that is the focus of this evaluation consists of the offer of free homebuyer education and counseling services, an offer which some of the treatment group took up.

Homebuyer education provides general information about buying a home, either in a classroom workshop format or via an online program. Homebuyer counseling provides one-on-one guidance, either in person or by telephone, tailored to the particular needs of the individual homebuyer. The content of both the intervention’s homebuyer education and its homebuyer one-on-one counseling aligns with two sets of standards: the National Industry Standards for Homeownership Education and Counseling and standards set by HUD. Additionally, the services are provided to study participants by HUD-approved housing counselors who are specially trained and certified to deliver these services.

This chapter on the study’s intervention begins by describing the National Industry Standards that the study’s agencies followed in delivering the homebuyer education and counseling. Then it describes the key features of the two modes of homebuyer education and counseling (in-person versus remote) as delivered to study participants in this Demonstration. This is followed by a description of the service experience from the treatment group members’ perspective. The chapter ends with a discussion of factors about the intervention that are relevant for interpreting the impact results included in this report.

3.1 Content and Quality of Study’s Homebuyer Education and Counseling

The homebuyer education and the one-on-one counseling offered to treatment group members in the Demonstration both have the same curriculum content, regardless of service delivery mode.

All the study’s agencies followed the topics and guidelines of the National Industry Standards for Homeownership Education and Counseling, which aim to promote consistent and high-quality homebuyer education and counseling services across the country (Advisory Council, 2013).

• For homebuyer education, the Standards recommend core topic areas to be covered, including how to review a credit report and what to do if a mortgage payment is missed.

These core topic areas are summarized in exhibit 3.1. The Standards suggest that 8 hours of education are needed to adequately cover the content.

• For one-on-one counseling, the Standards recommend 30 to 60 minutes of individualized counseling involving the following activities: intake and needs assessment; review of income, expenses, debt, credit report, budget, and savings; housing affordability analysis; development of an action plan that includes the next steps in the homebuying process; referrals as needed; delinquency prevention counseling; and followup sessions as necessary.

In addition to following the National Industry Standards, all the agencies involved in this Demonstration were HUD-approved, meaning they applied to HUD to participate in HUD’s The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 30

3. THE INTERVENTION

Housing Counseling Program and, thereafter, receive training and active oversight and monitoring from HUD.

This attention to the content of services delivered—through both the Standards and HUD’s oversight—means that the homebuyer education and counseling offered to treatment group participants in this study met HUD’s standards for quality.

Exhibit 3.1: National Industry Standards, Homebuyer Education’s Core Content

Source: Advisory Council for the National Industry Standards for Homeownership Education and Counseling, 2013

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 31

3. THE INTERVENTION

3.2 In-Person Service Delivery

For the provision of in-person services, the study team partnered with 63 local HUD-approved housing counseling agencies across the study’s 28 metropolitan areas. Treatment group members taking up in-person services could attend homebuyer education workshops and one-on-one counseling sessions offered by a participating agency in their community. They had up to 1 year to complete services from the time of study enrollment. The average study participant who completed in-person education and counseling did so within about 3.5 months from random assignment.

3.2.1 In-Person Education

The format of in-person homebuyer education workshops offered to treatment group members in this study varied across the 28 sites, although the content of the curriculum was similar.

Generally, workshops were interactive, using a variety of tools, with guest speakers25 on specific topics such as the role and services of lenders, real estate agents, insurance agents, home inspectors, and closing attorneys. The in-person format allows some flexibility in the delivery of content by allowing the audience to influence the discussion through their interest and questions.

According to the service data received from our participating agencies, the in-person education curriculum for study participants averaged about 8 hours, either as a single full-day workshop or split over 2 half-days. Of those participants who took up in-person education, nearly nine-tenths (86 percent) completed the education in one session, and 80 percent spent between 7.0 and 8.5

hours completing the education.

Most of the study’s agencies held workshops in both English and Spanish, and most held workshops on Saturdays. Some agencies offered childcare to help parents attend workshops on the weekend. The average class size was 29 people, with classes ranging from 10 to 71 people and the larger classes falling during the “busy” homebuyer season (usually May through September). The number of homebuyer workshops held each month varied by agency, with most hosting one workshop per month.

3.2.2 In-Person One-on-One Counseling

The in-person one-on-one counseling sessions about homeownership provided participants with individualized, objective information and advice on developing a budget, saving, and the cost of homeownership. Counseling sessions for participants typically occurred in 1- to 2-hour time blocks, with the counselor reviewing the participant’s information in advance in order to address his/her particular situation.

25 Housing counseling agency staff and their volunteers are required to comply with the conflict of interest policies stated in 24 CFR Part 214, Housing Counseling Program Regulations. This includes the prohibition of marketing any specific services to clients participating in an education workshop.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 32

3. THE INTERVENTION

Tailored to the participants within the treatment group, these counseling sessions focused on the areas in which the participant needed or wanted the most assistance, considering where he/she was in the homebuying process. At the end of most counseling sessions, counselor and participant developed an action plan. The action plan was specific to the participant’s goals and outlined the steps involved to achieve those goals. The action items typically included (1) preparing a household budget, (2) reviewing his/her credit report, (3) beginning to save for amount needed to purchase a home, and (4) taking steps to repair credit. Participants were encouraged to follow up with the counseling agency as often as they felt necessary to reach their goal of homeownership, but few participated in more than one session.

3.2.3 Advantages/Disadvantages of In-Person Service Delivery

When asked about the advantages and disadvantages of the in-person service delivery, the study’s focus group attendees easily identified many benefits to completing the homebuyer education and counseling. They liked the ability to ask questions and learn from the questions others asked. They liked that many of the instructors were local professionals. They associated face-to-face meetings with familiarity and trust. The trust issue became more important during counseling, when there was a perceived risk to disclosing personal financial information.

The main disadvantages reported were in scheduling and traveling to the agency. Several focus group attendees described the difficulty in finding time to participate in a 6- to 8-hour session that was typically offered only once a month on a Saturday.

3.3 Remote Service Delivery

Remote services provided through the study included an online, Internet-based homebuyer education curriculum and one-on-one telephone counseling. The study team partnered with two national agencies to provide remote services to the study’s treatment group members. eHome America provided the online education curriculum for the study; and ClearPoint Credit Counseling Solutions provided the one-on-one telephone counseling. Treatment group members had up to 1 year to complete services from the time of study enrollment. The average study participant who completed remote education and counseling did so within about 2.5 months from random assignment.

3.3.1 Online Education Course

eHome America’s online education platform was interactive: in addition to providing informational text, the online course featured videos, worksheets, and quizzes to engage participants in learning. The curriculum had six modules, each of which included quizzes and a final test on the material covered. To proceed to the next module, participants had to complete the quiz with a grade of 80 percent or higher. Participants who took up the online education curriculum were able to save their progress and complete the online course at their own pace and convenience.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 33

3. THE INTERVENTION

According to the service tracking data received from eHome, participants who took up the remote education averaged 6.8 hours working through the online curriculum, but the time spent varied widely. One-half of all participants who took up online homebuyer education spent 5.1

hours or less working through the curriculum, and roughly 10 percent spent more than 15 hours.

Most users completed the course in 6 to 8 hours over a span of 3 to 5 days.

3.3.2 Telephone One-on-One Counseling

For the telephone one-on-one counseling component, counselors followed a three-stage protocol to cover the same core elements for each client: (1) budget and personal financial assessment, (2) loan qualification, and (3) closing. Discussions about finances, budgeting, and the mortgage process were individualized to the participant’s needs and questions.

Following the session, each participant received a counseling summary package by mail. The package contained a completed budget that incorporated the information collected and analyzed during the session, along with the housing counselor’s recommendations. A typical one-on-one counseling session lasted about 1 hour; and the vast majority of participants had only one session with a housing counselor. Although counselors encouraged them to call back with questions after the initial session, few did. The action items following a remote counseling session typically included (1) preparing a household budget, (2) reviewing his/her credit report, and (3) beginning to save for amount needed to purchase a home.

3.3.3 Advantages/Disadvantages of Remote Service Delivery

The study’s focus group attendees reported that the advantages of remotely completing homebuyer education and counseling services included the ability to complete them at the participants’ own convenience and at their own pace. The disadvantages included the long length of the course and finding the time and motivation to complete the homebuyer education and counseling.

3.4 Comparison of In-Person and Remote Services

The primary similarities of the remote and in-person homebuyer education and counseling are the content covered and the length of time to complete the services. Both modes followed the topics outlined in the National Industry Standards and both take about the same amount of time to complete (typically 8 hours for the education and 1 hour for the counseling).

As described in section 2.3, the rates at which treatment group members took up services and completed services differ meaningfully between the in-person and remote service modes, with both take-up and completion rates much higher for those offered remote services than for those offered in-person services. Further, study participants clearly preferred remote services (see exhibit 2.3 on page 16): among the 1,142 choice treatment group members, 291 (25.5 percent) expressed a preference for in-person services, whereas the remaining 851 (74.5 percent) expressed a preference for remote services. The preference for remote services is consistent with prior research, which has found that people prefer online education and telephone counseling The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 34

3. THE INTERVENTION

over in-person services (Barron and Staten, 2012). Even those choice treatment group members who expressed a preference for in-person services had a service take-up rate of 32.0 percent, notably lower than the 64.6 percent take-up rate for those who expressed a preference for remote services, despite the service mode offer being in line with their stated preference (see exhibit 2.4

on page 17).

As for who among the study participants chose to take up and complete homebuyer education and counseling services, Moulton et al. (forthcoming) show that, across both in-person and remote service modes, women and those with relatively higher educational attainment were more likely to take up and complete homebuyer education and counseling services. Those referred to in-person services were more likely to participate in services if they were at an early stage of the homebuying process at the time of study enrollment. They were also more likely to complete the education curriculum if they reported being “pretty good at math” or if they planned to purchase a home without a co-borrower. In comparison, those referred to remote services were more likely to participate in services if they planned to spend more years living in their purchased home; scored better on a baseline mortgage literacy quiz; or had a baseline credit score of 740 or higher.

Race/ethnicity, age, marital status, and household size were not associated with service participation levels.

The most important difference is in the delivery format. Although the study took measures to offer similar homebuyer education and counseling content across participating housing counseling agencies, the variation that arises from the delivery format can affect how participants experience and respond to the content. For example, in-person homebuyer education provides group instruction by an in-person trainer who, more than the online course, can be responsive to participants in the room, emphasizing one topic area over another in response to their apparent needs and interests. Additionally, group instruction allows participants to learn from one another and hear about others’ experiences in the homebuying process. In contrast, remote homebuyer education can be completed at the participant’s convenience and own pace. However, remote education requires the completion of pre-specified modules without the ability to expand on topics of interest or ask questions and learn from others. Remote education also requires passing a quiz before moving to the next module. Very few of the study agencies providing in-person services used quizzes.

Homebuyer one-on-one counseling is less affected by the mode of delivery, because face-to-face and telephone counseling sessions are both tailored to the participant’s circumstances, including readiness for home purchase and specific issues or challenges he/she faces. However, some focus group attendees reported that they associate the face-to-face format with trust and familiarity, which becomes especially important when working with a counselor and disclosing personal financial information.

The two modes of homebuyer education and counseling can have different transaction costs for participants. Most of the in-person agencies offered one day-long workshop a month, usually on a Saturday, which made scheduling a challenge for study participants. The remote mode was The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 35

3. THE INTERVENTION

likely to be more convenient in scheduling and did not have costs associated with travel.

However, some focus group participants assigned to complete remote homebuyer education and counseling said they were subject to more distractions while completing the remote services than if they had been in an in-person workshop or with a housing counselor. Exhibit 3.2 summarizes the differences between in-person and remote homebuyer education and counseling.

These differences between the in-person and remote service modes have implications for what we might expect regarding their differential impacts. Considering the outcome of mortgage literacy (which we measure by way of a four-question assessment; see the textbox Mortgage Literacy Quiz on page 63), the quiz aspect of the online education component might lead to those offered remote services having greater improvement in mortgage literacy. That is, remote participants had to “prove” that they mastered the content, whereas in-person participants did not. Conversely, the ability to ask questions, learn from others, and clarify and discuss information might lead to those offered access to the in-person workshops having greater relative gains in mortgage literacy.

Our observations about how these services differ in practice result in conflicting hypotheses, and this study examines the relative impacts of the two service modes without a clear a priori hypothesis about which would lead to more favorable impacts.

Exhibit 3.2: Differences Between In-Person and Remote Homebuyer Education and Counseling Feature

In-Person Services

Remote Services

Education

Format

Treatment group members attended one or

Treatment group members accessed the training

several in-person classroom sessions at a local

online from a home or public computer at their

HUD-approved homebuyer counseling agency

convenience. Participants could complete

training in as few or as many sessions as

desired

Hours

Typical y 7-8 hours

Typical y 7-8 hours

Variation in content Moderate variation. In-person providers generally Low variation. Al participants experienced the covered similar topics but could vary curricular

same online learning environment and had to

materials and emphasize different topics based

pass a quiz to be able to proceed to the next

on the attendees’ interest and questions

module

Participant

Higher transaction costs. Participants must

Lower transaction costs. Participants must invest

transaction costs

schedule in advance and travel to sessions that

time but can complete the process at their own

are most typically offered once a month on a

pace, when and where they want

Saturday

Counseling

Format

Treatment group members attend one or more

Treatment group members hold one or more

in-person, one-on-one sessions with a certified

telephone calls with a ClearPoint counselor from

housing counselor at a HUD-approved

their home or location of choice

homebuyer counseling agency

Hours

1+ hours

1+ hours

Variation in content Customized

Customized

Participant

Higher transaction costs. Participants must

Lower transaction costs. Participants may

transaction costs

schedule in advance and travel to sessions

complete counseling via telephone at a

scheduled time but at a location of their choice

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 36

3. THE INTERVENTION

3.5 Participant Experiences With Services

In general, treatment group members who attended the study’s focus groups reported their main reason for participating in homebuyer education and counseling services was that they expected the homebuying process would be complicated and stressful and that having education and counseling would help them through it. Participants who completed the services said they gained increased knowledge and confidence about the homebuying process, terminology used, and long-term costs of homeownership.

Both the focus group attendees and the Short-Term Follow-Up Survey respondents reported that their reasons for not completing services pertained mostly to logistics. For all treatment group members, finding the time to fit education and counseling into busy schedules was a challenge.

For those offered in-person services, agency location and scheduling were challenges; for those offered remote services, the length of the online course was a challenge.

That said, treatment group members who completed services reported being satisfied with their experience (exhibit 3.3):

• Among respondents to the Short-Term Follow-Up Survey who completed homebuyer

education, 93 percent reported being “very satisfied” or “somewhat satisfied” with the education curriculum. Both those who received the education in person and those who received education remotely rated the most helpful topics in the curriculum as (1) homeownership readiness, (2) budget and credit, and (3) financing a home.

• Among respondents to the Short-Term Follow-Up Survey who completed homebuyer one-on-one counseling, 88 percent reported being “very satisfied” or “somewhat satisfied” with it. For those who received the counseling in person, the two most helpful topic areas26 were (1) reviewing household income, expenses, debt, and savings and (2) reviewing credit reports. For those who received remote counseling, the two most helpful topic areas were (1) reviewing household income, expenses, debt, and savings and (2) calculating affordability based on one’s income and debt.

26 The followup survey offered a menu of topic choices: reviewing information about you and your household’s income, expenses, debt, and savings; reviewing your credit report(s) with you; identifying your credit challenges; developing a household budget; analyzing your budget and recommending modifications; conducting various calculations including affordability based on income and debt; developing a written action plan for you; following up with you after you completed counseling; making referrals for additional services; and providing you information on delinquency and foreclosure services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 37

3. THE INTERVENTION

Exhibit 3.3: Satisfaction With Services and Most “Helpful” Topics Covered, by Service and Delivery Mode Source: Short-Term Fol ow-Up Survey data

3.6 Key Take-Aways About the Intervention

The homebuyer education and counseling services offered to the study’s treatment group members have some major strengths. All local counseling agencies participating in the study adhered to the National Industry Standards and provided similar curriculum content. All study agencies were HUD-approved. This ensures that the intervention services provided through the study were reasonably consistent in structure and content.

Although the in-person and remote service delivery modes cover similar content and require similar amounts of time to complete services, they also differ in a few key ways. In-person services offer greater flexibility to customize the content and pacing of the session than do remote services. However, in-person services also have greater transaction costs than remote services; participants must plan in advance and then travel to a workshop that is offered usually only once a month.

The experiences of study participants and the experiences of the clients that our study agencies typically serve are similar in two important ways. First, both groups are exposed to the same curriculum. Second, both groups take about the same amount of time to complete that material.

However, there are important differences between the study participants and agencies’ typical clients. Agency clients typically are referred by an organization that is providing them with downpayment assistance or a mortgage that requires applicants to complete homebuyer education and counseling. Qualifying for that assistance or loan can be considered the incentive for participating. In contrast, study participants were referred to the study by the study’s The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 38

3. THE INTERVENTION

cooperating lenders after having inquired about a standard home loan. 27 In addition, treatment group members were offered the services free of charge, and all study participants were given monetary compensation for their time. This is in contrast to standard practice, where clients often pay for homebuyer education and counseling services and are not provided monetary compensation for their participation. 28 Although it is unclear what, if any, implications these differences may have for how individuals are affected by homebuyer education and counseling services, it is nonetheless important to note that these differences do exist.

27 The study design excluded anyone participating in a downpayment assistance program or in a loan program that required applicants to complete homebuyer education and counseling.

28 Study participants could receive up to $150 for completing the homebuyer education and counseling: $50 after completing the first online education module or the first in-person education workshop session, and then an additional $100 after completing education and one phone or in-person counseling session.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 39

4. Prospective Homebuyers’ Experiences

Key Findings: Prospective Homebuyers’ Experiences

This chapter provides a descriptive analysis of the study sample’s experiences with the home purchase process.

• At 12 to 18 months after study enrollment, three out of five study participants had purchased a home. These participants had, on average, stronger financial profiles than did study participants who had not purchased a home.

• Among study participants who had not purchased, about three-fifths were stil actively looking for a home, whereas two-fifths had postponed their home search. Among those study participants who reported postponing their search, the most frequently cited reasons for postponement related to affordability.

• Among study participants with a mortgage loan, mortgage delinquency rates were very low. Some 1 percent reported missing a mortgage payment, and only 0.3 percent reported experiencing a 60-day delinquency as of the short-term followup. These low delinquency rates imply that we are unlikely to detect impacts on these measures of loan performance during the short-term period examined by this report.

The purpose of the chapter is to provide descriptive information about the overall experience of all study participants homebuying experiences during the first 12 to 18 months after study enrollment. More specifically, we describe where in the home buying process study participants were, as well as their housing arrangements. For those who postponed their home search, we describe their reasons for this decision. For study participants who purchased a home, we describe their experiences searching for a home and the price and financing information associated with their purchased home. By providing information on the housing and homeownership status of all study participants, as well as the financial characteristics of and mortgage terms for those participants who purchased homes, this chapter provides insight into the particular environment and experience of our study sample at the time of the Short-Term Follow-Up Survey. 29,30, 31

The findings presented in this chapter are reported for the entire sample of participants—that is, the combined sample of treatment and control group members. In this chapter, we do not 29 Understanding the context for our findings and the degree to which they are generalizable is a critical issue. For more on this issue, see section 8.4.

30 The main data sources used for this chapter are the Short-Term Follow-Up Survey administered to study participants beginning at 12 months after random assignment, and credit bureau data from the same time. Study participants replied to the followup survey an average of 13.2 months after enrollment (the median time is 13

months). We observed study participants’ credit bureau data an average of 14.3 months after their enrolling in the study (the median time is 14 months).

31 For all analyses of followup survey data presented in this chapter, we applied sample weights that adjust for followup survey nonresponse to ensure that the estimates are generalizable to our full study sample.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 40

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

compare treatment and control group outcomes; consequently, this chapter’s discussion does not imply anything about the effectiveness of the homebuyer education and counseling intervention.

4.1 Homeowner Status and Housing Arrangements

As shown in exhibit 4.1, 12–18 months after study enrollment, approximately three out of five study members (59.3 percent) had purchased a home, a 46.1-percentage point increase compared with the proportion of study members who were homeowners at baseline (13.2 percent, exhibit A.7). 32,33 At followup, 23.3 percent of study participants were still actively looking for a home, whereas 17.3 percent had postponed their home search.

Exhibit 4.1 also shows the home search status of study participants at the time of the followup survey, given their homebuying stage at baseline. Not surprisingly, those study participants who were further along in the homebuying process at baseline were more likely to have purchased a home at the time of the followup survey and less likely to have postponed their home search than their counterparts who were earlier in the process at baseline. For example, of those study participants who had not yet started the search process at baseline, 22.4 percent had purchased a home at the time of the followup survey, compared with 61.7 percent of those who at baseline, had made an offer but had not signed a purchase agreement. 34 Meanwhile, 32.7 percent of those who at baseline, had not yet started their search, had postponed their search at the time of the followup survey, compared with 16.7 percent of those who, at baseline, had made an offer but not signed a purchase agreement.

32 For an in-depth discussion of characteristics of the study sample at baseline, see DeMarco et al. (2017).

33 Despite targeting customers early in the homebuying process, some 13 percent of study participants had already purchased a home at the time of enrollment in the study, and another 25 percent had already signed a purchase agreement (DeMarco et al., 2017). That participants enter the study through a referral by lenders likely explains why many participants were fairly far along at enrollment.

34 At first glance, some of the figures in exhibit 4.1 could seem nonsensical. For example, of those study members who at baseline, had signed a purchase agreement, 3.0 percent at the time of the followup survey, “made an offer, but no purchase agreement.” However, it is possible for participants to “move backwards” in the homebuying process; for example, if a purchase agreement falls through due to a financing or inspection contingency.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 41

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

Exhibit 4.1: Stage in the Homebuying Process at Short-Term Followup, by Baseline Stage in the Homebuying Process

Subsample of Study Participants Who Have:

Made an

Of er on a

Home, but

Not Yet

Started

No

Signed a

Started

Home

Visited

Purchase

Purchase

Home

Search, but Homes, but Agreement Agreement

Full

Search at No Visits at No Of er at

at

at

Samplea

Baselineb

Baselinec

Baselined

Baselinee

Baselinef

(%)

(%)

(%)

(%)

(%)

(%)

Not actively searching at short-term fol owup

Postponed home search

17.3

32.7

28.6

25.1

16.7

6.5

Actively searching at short-term fol owup

Started home search, but no visits

2.3

8.3

8.9

0.8

0.2

0.0

Visited homes, but no offer

8.9

20.6

18.9

14.9

1.9

0.8

Made an offer on a home, but no

purchase agreement

9.1

12.8

10.8

14.5

14.8

3.0

Signed a purchase agreement

3.0

3.2

2.7

4.1

4.9

2.6

Purchased a home at short-term fol owup

Purchased a home

59.3

22.4

30.2

40.4

61.7

87.1

a Sample: 4,431 study participants who responded to the Short-Term Fol ow-Up Survey.

b Sample: 464 study participants who responded to the Short-Term Fol ow-Up Survey and at baseline had not yet started search.

c Sample: 571 study participants who responded to the Short-Term Fol ow-Up Survey and at baseline had started home search, but no visits.

d Sample: 1,063 study participants who responded to the Short-Term Fol ow-Up Survey and at baseline had visited homes, but made no offers.

e Sample: 624 study participants who responded to the Short-Term Fol ow-Up Survey and at baseline had made an offer on a home, but no purchase agreement.

f Sample: 1,140 study participants who responded to the Short-Term Fol ow-Up Survey and at baseline had signed a purchase agreement.

Notes: Excludes study participants who withdrew from the study and those missing measure-specific data. At baseline, 11.2 percent of study participants had not yet started their home search; 13.2 percent had started their search but not made any visits, 23.8

percent had made visits but not made any offers, 13.2 percent had made an offer but not signed a purchase agreement; 25.3

percent had signed a purchase agreement; and 13.2 percent had purchased a home (exhibit A.7).

Sources: Baseline survey and Short-Term Follow-Up Survey

A number of baseline measures of financial and credit characteristics, housing arrangements, and demographic characteristics are statistically significant predictors of whether a study participant had purchased a home as of short-term followup:35

• Both employment and income are predictors of home purchase: those who were employed part-time at baseline and those with annual incomes of less than $25,000 were less likely to purchase a home. Additionally, those with a credit score below 620 were less likely to purchase a home.

35 These findings are based on a model that use baseline characteristics to predict whether a study participant purchased a home at followup. See Appendix G for more detail.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 42

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

• Those who reported that they set aside extra money for retirement, education, or to build a financial cushion were more likely to purchase a home. Similarly, those who searched for information about the mortgage loans available from multiple lenders were more likely to purchase a home.

• Not surprisingly, study participants who were at later stages in the homebuying process at baseline (that is, made an offer, signed a purchase agreement, or had already purchased a home) were more likely to have purchased a home as of followup. Additionally, those who planned to live in their purchased home for 6 or more years were more likely to have purchased a home relative to those who planned to live in their purchased home for less than 6 years.

• Certain demographic characteristics are also predictors of whether the study participant had purchased a home as of the study’s followup. Hispanics and African-Americans were less likely to purchase a home relative to non-Hispanic Whites. Those married at the time they entered the study were more likely to purchase than those single or never married. Those with a bachelors’ degree or higher at baseline were more likely to purchase than those with lower education attainment.

Agencies providing homebuyer education and counseling services might find these descriptive findings informative to their approach to providing services. For example, that certain characteristics are associated with home purchase could inform the way agencies decide to market and provide their services to meet potential clients’ interests, preferences, and needs.

Among those study participants who reported postponing their search, reasons for postponement varied. However, three of the five most frequently cited reasons related to affordability (exhibit 4.2): 30.6 percent of study participants reported not liking the homes they could afford, 28.8

percent said that they could not afford to buy, and 20.9 percent said they did not like the neighborhoods they could afford. 36 Other common reasons that participants cited for postponing their home search included changes to their employment situation (22.8 percent) and learning that they needed to repair their credit (22.1 percent).

36 Respondents could select multiple reasons for postponing their home search.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 43

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

Exhibit 4.2: Reasons for Postponing Purchase, Among Nonpurchasers Who Postponed Their Home Search Subsample of Nonpurchasers Who

Postponed Their Home Searcha

(%)

Among those who postponed their home search, reason for postponing

Did not like the houses I could afford

30.6

Learned couldn’t afford to buy

28.8

There was a change in my (or my co-purchaser’s) employment situation

22.8

Learned I needed to repair my credit first

22.1

Did not like the neighborhoods I could afford

20.9

Current economic climate has made it more difficult to get a mortgage

16.1

Information from a counseling agency workshop made me better aware of my

personal situation in the homebuying process

11.8

The person I was planning to purchase a home with is no longer interested in

purchasing a home

4.6

a Sample: 777 study participants who responded to the Short-Term Fol ow-Up Survey and had postponed their home search.

Note: Excludes study participants who withdrew from the study and those missing measure-specific data.

Source: Short-Term Fol ow-Up Survey

Almost all study participants who had purchased a home (97.2 percent) were living in their purchased home as of the followup period, as shown in exhibit 4.3. The vast majority of participants who had not purchased were renting (86.0 percent), 10.1 percent were living in someone else’s home without paying rent, and 3.9 percent had some other living arrangement.

Exhibit 4.3: Housing Arrangements at Short-Term Followup, Purchasers versus Nonpurchasers Subsample of

Subsample of

Purchasersa

Nonpurchasersb

(%)

(%)

Lives in purchased home

97.2

0.0

Rents a home or apartment

1.7

86.0

Lives in someone else’s house or apartment w/o paying rent

0.9

10.1

Other living arrangement (for example, military base or college

0.3

3.9

dorm)

a Sample: 2,715 study participants who responded to the Short-Term Fol ow-Up Survey and had purchased a home.

b Sample: 1,831 study participants who responded to the Short-Term Fol ow-Up Survey and had not purchased a home.

Note: Excludes study participants who withdrew from the study and those missing measure-specific data.

Source: Short-Term Fol ow-Up Survey

Overall, purchasers reported paying more per month on housing than did nonpurchasers. As presented in exhibit 4.4, purchasers were paying an average of $414 more per month in housing payments than their nonpurchasing counterparts ($1,343 for purchasers and $929 for nonpurchasers). However, purchasers and nonpurchasers were both paying roughly the same percentage of their monthly income on housing payments (on average, about one-quarter of their income was spent on housing). 37 This is explained by the higher average income of purchasers 37 Monthly income is the monthly income received by study participant and any co-borrowers in last 12 months based on responses to the Short-Term Follow-Up Survey.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 44

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

relative to nonpurchasers: the average income of purchasers and their co-borrowers in the 12

months prior to the followup survey was $73,463 compared with $57,847 for nonpurchasers.

Perhaps related to their lower incomes, nonpurchasers were more likely to have a housing cost burden at followup survey: 26.9 percent of nonpurchasers were paying at least 30 percent of their income in housing costs and 7.6 percent were paying at least 50 percent, compared with 22.7

percent and 3.6 percent for purchasers (exhibit 4.4).

Exhibit 4.4: Housing Costs at Short-Term Followup, Purchasers versus Nonpurchasers

Subsample of

Subsample of

Purchasersa

Nonpurchasersb

Difference

Monthly housing cost ($)c

10th percentile

635

0

-

25th percentile

868

650

-

50th percentile

1,200

900

-

75th percentile

1,698

1,200

-

90th percentile

2,300

1,600

-

Mean

1,343

929

414***

Ratio of monthly housing payment to monthly income

50th percentile

0.22

0.21

-

Mean

0.25

0.24

0.00

Ratio greater than or equal to 0.30 (%)

22.7

26.9

–4.2***

Ratio greater than or equal to 0.50 (%)

3.6

7.6

–4.0***

a Sample: 2,715 study participants who responded to the Short-Term Fol ow-Up Survey and had purchased a home.

b Sample: 1,831 study participants who responded to the Short-Term Fol ow-Up Survey and had not purchased a home.

c For homeowners, the monthly housing cost includes their monthly payment to lenders (including payments toward principal, interest, taxes, and insurance), but does not include utilities. For nonpurchasers, monthly housing cost includes rent, but does not include utilities. Nonpurchasers who live rent-free are coded as having $0 in monthly housing cost.

*** Difference is statistical y significant (two-sided t-test) at the p< 0.01 level.

** Difference is statistically significant (two-sided t-test) at the p< 0.05 level.

* Difference is statistically significant (two-sided t-test) at the p< 0.10 level.

Note: Excludes study participants who withdrew from the study and those missing measure-specific data.

Source: Short-Term Fol ow-Up Survey

4.2 Homebuying Experience of Purchasers

In this section, we look more closely at the homebuying experiences of the subset of study participants who purchased a home by the time of the followup survey. Since starting the home search process, study participants who purchased a home reported visiting an average of 15.7

homes, contacting an average of 2.4 lenders, and receiving price quotes from an average of 1.9

lenders (exhibit 4.5). Some 7 percent of those who eventually purchased a home had a loan application denied.

From the time of study enrollment, study participants who purchased took an average of 2.4

months to purchase a home (exhibit 4.5). Among study participants who purchased a home by the time of the followup survey, about one-fourth (25.6 percent) planned to have a co-borrower at baseline, and a similar share of purchasers actually ended up purchasing a home with a co-borrower at followup (22.5 percent). About three-fourths (77.3 percent) of purchasers reported that they were “confident” or “very confident” in their ability to find the information they needed The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 45

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

about the homebuying process. Overall, 87.5 percent of study purchasers were either “somewhat satisfied” or “very satisfied” with their homebuying process. Almost one-half (47.5 percent) of purchasers planned to spend more than a decade in their new home.

Exhibit 4.5: Home Purchase Experience at Short-Term Followup, Among Purchasers Subsample of

Purchasersa

Time from study enrollment to home purchase (months)

Median

1.0

Mean

2.4

Homes visited

Number of homes visited (mean)

15.7

Mortgage shopping

Number of lenders contacted (mean)

2.4

Number of lenders from which the study participant received price quotes (mean)

1.9

Had loan application denied (%)

7.2

Presence of co-borrower (%)

Planned to have a co-borrower at baseline

25.6

Co-borrower at followup

22.5

Confidence of study participant in finding needed information about the homebuying process (%)

Very confident

34.6

Confident

42.7

Somewhat confident

20.3

Not confident at all

2.4

Satisfaction of study participant with the homebuying process (%)

Very satisfied

41.8

Somewhat satisfied

45.7

Somewhat dissatisfied

9.4

Very dissatisfied

3.0

Years planned to own purchased home (%)

Less than 5 years

19.3

6 to 10 years

33.2

11 or more years

47.5

a Sample: 2,715 study participants who responded to the Short-Term Fol ow-Up Survey and had purchased a home.

Note: Excludes study participants who withdrew from the study and those missing measure-specific data.

Sources: Baseline Survey and Short-Term Follow-Up Survey

The median purchase price for study participants who purchased a home was $183,000, slightly below the national median sales price for 2014 ($200,000) and 2015 ($217,500) (Urban Institute, 2018: 19). However, the purchase prices for those study participants who bought homes ranged considerably, with 10th and 90th percentile prices of $85,000 and $400,000 (exhibit 4.6).

The vast majority of study participants who took out a mortgage loan—some 90 percent—

received 30-year fixed-rate mortgages, and another 6.0 percent received fixed-rate mortgages of another term. Among those with a 30-year fixed-rate mortgage, the average interest rate was 4.1

percent. These interest rates are comparable to average interest rates offered over the study The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 46

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

period. 38 Some 4 percent received adjustable-rate mortgages. Among those with adjustable-rate mortgages, the average initial interest rate was 3.4 percent. More than one-third (35.9 percent) of study participants with a mortgage had an FHA loan. Additionally, one-fourth (25.6 percent) reported having private mortgage insurance and 3.1 percent had a loan guaranteed by the Veterans Administration.

The average downpayment for purchasers was $27,443 (although the median downpayment was notably less, at $9,000); and 8.5 percent received downpayment assistance, whether formally or from friends or family. About one-third (34.7 percent) of those with a mortgage loan had a loan-to-value (LTV) ratio less than or equal to 0.80; about two-fifths (38.6 percent) had a LTV ratio between 0.80 and 0.85; and about one-fourth (26.7 percent) had an LTV ratio greater than 0.95. 39

All these findings are simply descriptive and compare experiences for purchasers and nonpurchasers or describe experiences for specific subsets of purchasers; none of these results compares treatment and control group outcomes (those are reported in chapters 5, 6, and 7).

Exhibit 4.6: Home Purchase Price and Financing, Among Purchasers

Subsample of

Purchasers

Purchase price a ($)

10th percentile

85,000

25th percentile

128,900

50th percentile (median)

183,000

75th percentile

279,000

90th percentile

400,000

Mean

222,996

Loan termsb (%)

Fixed-rate mortgage with 30-year term

89.9

Fixed-rate mortgage with 15-year term

4.3

Fixed-rate mortgage with term other than 15 or 30 years

1.7

Adjustable-rate mortgage, where time to adjust is 5 years or less

1.3

Adjustable-rate mortgage, where time to adjust is greater than 5 years

2.8

Mortgage loan interest rateb

Interest rate among those with fixed-rate mortgage with 30-year term (mean)

4.1

Interest rate among those with fixed-rate mortgage with 15-year term (mean)

3.3

Interest rate among those with adjustable rate mortgage (mean)

3.4

Loan-to-value (LTV) ratio categoriesb (%)

LTV less than or equal to 0.80

34.7

LTV 0.80 to 0.95

38.6

LTV greater than 0.95

26.7

38 Interest rates on 30-year fixed-rate mortgages between 2014 and 2016 ranged between 4.43 and 3.44 percent (Freddie Mac, 2018).

39 The loan-to-value ratio is computed as the reported mortgage loan amount divided by the purchase price.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 47

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

Subsample of

Purchasers

FHA loanc (%)

FHA loan

35.9

Mortgage loan insurance/guaranteeb (%)

Has mortgage loan insurance from private insurance company

25.6

VA loan

3.1

Has mortgage insurance from other source (for example, a state agency)

6.9

Downpayment assistanceb

Received downpayment assistance (%)

8.5

Amount of downpayment assistance among those receiving assistance ($)

1,009

Downpaymenta ($)

50th percentile

9,000

Mean

27,443

FHA = Federal Housing Administration; VA = Veterans Administration.

a Sample: 2,715 study participants who responded to the Short-Term Fol ow-Up Survey and had purchased a home.

b Sample: 2,545 study participants who responded to the Short-Term Fol ow-Up Survey and had taken out a mortgage loan.

c Sample: 3,467 study participants who had taken out a mortgage loan. Source: Short-Term Follow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data.

Note: Excludes study participants who withdrew from the study and those missing measure-specific data.

Source: Short-Term Fol ow-Up Survey except where noted otherwise.

Among study participants with a mortgage loan, 3.7 percent reported missing a mortgage payment (that is, had been at least 30 days delinquent) as of the short-term followup40 1.0

percent of those with a mortgage loan reported experiencing a 60-day delinquency, and 0.5

percent reported a 90-day delinquency (exhibit 4.7).

Exhibit 4.7: Loan Performance, Among Purchasers with a Mortgage Loan

Subsample of Purchasers

with a Mortgage Loana

Loan performance at short-term fol owup

(%)

Ever 30 days delinquent

3.7

Ever 60 days delinquent

1.0

Ever 90 days delinquent

0.5

a Sample: 3,467 study participants with administrative data or Short-Term Fol ow-Up Survey data that had taken out a mortgage loan.

Note: Excludes study participants who withdrew from the study and those missing measure-specific data.

Sources: Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data.

40 The delinquency rates presented in exhibit 4.7 differ from those presented in chapter 7 for a couple reasons. First, the sample in exhibit 4.7 includes study participants (both treatment and control group members) who had taken out a mortgage loan at short-term followup, whereas the sample used in chapter 7 includes all study participants, regardless of whether they took out a mortgage loan. Second, exhibit 4.7 reports the delinquency rates for the pooled sample of all treatment and control group members, whereas chapter 7 reports regression-adjusted delinquency rates separately for the treatment group and control group. For these reasons, the delinquency rates presented here are not directly comparable with those presented in chapter 7.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 48

4. PROSPECTIVE HOMEBUYERS’ EXPERIENCES

These low delinquency rates at the time of the short-term followup are not surprising given (1) that purchasers had been in their homes, on average, fewer than 12 months (the mean number of months that purchasers have been in their homes is 10.5 and the median is 11); and (2) the relatively strict underwriting environment during the time of the study. 41, 42 Because of these low rates, we would not expect to detect impacts on these measures of loan performance in the short term. Delinquency rates will instead be the focus of the Long-Term Impact Report.

41 CoreLogic’s Housing Credit Index®, which measures how easy it is to obtain mortgage financing based on six underwriting factors, indicates that the credit environment during the study period has been considerably tighter than in prior years (DeMarco et al., 2017). In other words, study participants who had purchased a home had to meet relatively high underwriting standards including higher credit scores and lower debt-to-income ratios.

42 It should be noted that the delinquency rates for our sample are low compared to the FHA portfolio. For 2015

FHA loans, 8.2 percent of loans were ever 30 days delinquent, 2.5 percent were ever 60 days delinquent, and 1.0

percent were ever 90 days delinquent after 10 months (Source: calculations by Kevin Park at HUD). However, the delinquency rate is high compared to GSE loans. For example, the Fannie Mae cumulative default rates for 2014, 2015, and 2016 loan originations are close to zero. For more information, see Fannie Mae’s 2017 Credit Supplement (Fannie Mae, 2018).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 49

5. Impacts on Preparedness and Search

Key Findings: Impacts on Preparedness and Search

• Homebuyer education and counseling services improved treatment group members’ ability to find needed information related to the home purchase process.

• Homebuyer education and counseling services did not affect the intensity of search as measured by the number of price quotes study participants received from lenders.

• Homebuyer education and counseling services did not affect rates of home purchase overall.

However, two subgroups experienced increased home purchase rates:

- For those age 29 or younger at baseline, the offer of services increased home purchase rates by 3.3 percentage points. In contrast, we find no detectable evidence that services increased home purchase rates for those age 30 or older.

- Homebuyer education and counseling services had a larger impact on home purchase rates for participants with more cash available for downpayment (>$15,000) relative to those with less downpayment cash available.

• Although levels of satisfaction with the homebuying process were relatively low (less than one-third of all study participants were “very satisfied”), homebuyer education and counseling services increased satisfaction with the homebuying process. This finding is driven by a 9-percentage point impact on those offered in-person services.

This first of three impact chapters reports the impacts of the HUD First-Time Homebuyer Education and Counseling Demonstration’s services on outcomes that fall within the domain of preparedness and search. 43 These are the outcomes hypothesized to show impacts the soonest following completion of homebuyer education and counseling. Homebuyer education and counseling services are hypothesized to increase awareness and knowledge of the pros and cons of homeownership, homeowners’ responsibilities, mortgages and terms, and underwriting criteria. Homebuyer education and counseling services should also give recipients the skills needed to determine whether homes are affordable and financing options are appropriate for their individual circumstances. From this perspective, the services might encourage some to postpone buying a home until conditions are more favorable. As a result, the anticipated impact on purchasing a home is ambiguous.

The discussion of results focuses on the pattern of results, which is always the same whether we consider ITT impact or TOT impact (as explained in the textbox Calculating Impact Two Ways beginning on page 19). We leave the readers to decide which specific result—the ITT estimate or TOT estimate—represents the magnitude of greater interest to them. We also provide guidance on how to read the impact exhibits in this report (See the textbox “How to Read the Impact 43 Chapter 6 reports on the financial capability domain, and chapter 7 on the sustainable homeownership domain.

Chapter 2 and appendix A provide detail on how the impacts reported in this chapter (as well as in chapters 6 and 7) are estimated. Expanded results for the overall impact are reported in appendix C, and expanded results for delivery mode effects are reported in appendix D. Appendix E presents subgroup impacts.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 50

5. IMPACTS: PREPAREDNESS & SEARCH

Exhibits in This Report,” which follows) and how to interpret impacts that are not statistically significantly different from zero (see the textbox “Understanding Null Effects” on page 52).

Unless noted otherwise, we discuss only impacts that are statistically significant.

How to Read the Impact Exhibits in This Report

The sample table below presents the impact of homebuyer education and counseling services on a measure of whether study participants were very satisfied with the home purchase process. The table reports the mean level of the outcome for both the treatment and control groups: 32 percent of the treatment group and 29

percent of the control group were very satisfied with the home purchase process.

The difference between the two mean outcomes is the impact of being offered homebuyer education and counseling services, estimated using multiple regression as described in appendix section A.1. The table’s

“Impact of Being Offered Services” column, reporting the ITT estimate, shows that the treatment group was 3.1

percentage points more likely than the control group to say they were very satisfied with the home purchase process. We also report the impact of taking up homebuyer education and counseling services. The table’s “Impact of Taking Up Services” column, reporting the TOT estimate, shows that those in the treatment group who took up homebuyer education and counseling services were 5.1 percentage points more likely than the control group to say they were very satisfied with the home purchase process.a Impacts marked with one or more asterisks are statistical y significant, indicating that it is unlikely that the impact is due to chance. The number of asterisks indicates whether the impact is statistically significant at the 10 percent (*), 5 percent (**), or 1 percent (***) level. The more asterisks, the less likely the finding is due to chance. In the sample table below, the impact is statistically significant at the 5-percent level.

Sample Table. Overal Impact of the Demonstration’s Homebuyer Education and Counseling on Satisfaction with the Home Purchase Process

Impact of

Impact of

Treatment

Control

Being Offered

Taking Up

Outcome

Group Mean

Group Mean

Services

Services

Study participant was very satisfied with the home

purchase process

31.8

28.7

3.1**

5.1**

*** Dif erence is statistically significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Dif erence is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Due to rounding, reported impacts (T-C dif erences) may dif er from dif erences between reported regression-adjusted means for the treatment and control groups. Sample includes study participants with nonmissing outcome data.

Appendix A provides technical details related to the analytic methods used to estimate the impacts reported in these tables in chapters 5, 6, and 7. Appendix B provides additional detail on the construction of measures, including outcomes, covariates, and subgroup identifiers.

______________

a As always, the magnitudes of the ITT and TOT estimates differ, with the TOT always being larger, but the two estimates are in the same direction and have the same level of statistical significance.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 51

5. IMPACTS: PREPAREDNESS & SEARCH

Understanding Null Effects: Minimum Detectable Effects and Why They Matter

There are two reasons why we might not detect an impact on a given outcome. First, the null hypothesis might be true—that is, there might simply not be an impact. Alternatively, there could be an impact that is smaller than we can detect given the study design and available data. Related to the latter, the minimum detectable effect (MDE) is helpful for understanding findings that are not statistically significant.

MDEs indicate how large an impact needs to be in order to be detected at a given level of confidence. For example, as shown in the exhibit sample below, the MDE that corresponds to the impact of being offered homebuyer education and counseling on the share of study participants who purchased a home is 2.5

percentage points. That is, the true impact of being offered homebuyer education and counseling on the home purchase rate needs to be at least 2.5 percentage points to be detected as statistically significantly different from zero (at the 10-percent significance level 80 percent of the time). However, this study’s estimate of the impact of being offered services on the home purchase rate is 0.5 percentage point and is not statistically significantly different from zero. It is possible that the lack of significance is due to their being no real impact; however, it is also possible that the lack of significance is due to a real impact that is smaller than 2.5

percentage points.

Excerpt from Exhibit C.1: Il ustration of Minimum Detectable Effects

Treatment

Control

Overall Impact of

Minimum Detectable

Outcome

Group Mean

Group Mean

Being Offered Services

Effect

Study participant purchased

62.2

61.7

0.5

2.5

a home (%)

(1.0)

Note: Standard errors are reported in parentheses. See full exhibit notes in appendix C (exhibit C.1).

MDEs are inversely related to sample size—that is, MDEs increase as sample size gets smaller. It is therefore more difficult to detect a statistically significant impact for each service delivery mode independently than it is to detect overal impacts of services using the full study sample, simply because of the smaller sample sizes available for estimating service mode effects. Similarly, it is more difficult to detect statistically significant impacts for in-person services than for remote services because the sample size available for estimating the impact of in-person services is comparatively smal .

For example, as shown in the exhibit below, the MDE for the impact of being offered in-person services on the home purchase rate is 5.3 percentage points, and the MDE for the impact of being offered remote services is 2.7 percentage points. That is, the impact of in-person services on the home purchase rate would have to be 5.3 percentage points to be detected with a high probability, compared to 2.7 percentage points for remote services. In other words, the in-person intervention has a “higher bar” in order to attain statistical significance than does the remote intervention.

Excerpt from Exhibits D.1 and D.2: Il ustration of Minimum Detectable Effects for Mode Effects Impact of Being

Minimum

Impact of Being

Minimum

Offered In-Person

Detectable

Offered Remote

Detectable

Outcome

Services

Effect

Services

Effect

Study participant purchased a home (%)

1.2

5.3

0.1

2.7

(2.1)

(1.1)

Note: Standard errors are reported in parentheses. See full exhibit notes in appendix D (exhibits D.1 and D.2).

Keeping these issues in mind, in this report we are careful to note that for outcomes where the impacts are not statistical y significantly different from zero, there are not necessarily “no impacts.” Rather, there are “no detectable impacts,” as there might be impacts that are smaller than this study is powered to detect.

Appendix C reports MDEs for the overall impacts of homebuyer education and counseling, and appendix D

reports MDEs for the in-person and remote service delivery mode effects. Further discussion related to the computation and interpretation of MDEs can be found in appendix A.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 52

5. IMPACTS: PREPAREDNESS & SEARCH

• Homebuyer education and counseling services improved the confidence of the treatment group in their ability to find needed information related to the homebuying process.

Based on responses to the Short-Term Follow-Up Survey, 74.7 percent of prospective first-time homebuyers offered homebuyer education and counseling services were confident they could find the information they needed about the homebuying process, 4.9 percentage points more than the control group were (exhibit 5.1). The TOT estimate—the impact for those treatment group members who actually took up homebuyer education and counseling services—was, as always, larger than the ITT estimate. Those who took up homebuyer education and counseling services were 8.1 percentage points more likely than their control group counterparts to be confident in their ability to find needed information related to the homebuying process (computed under the assumptions that there is no effect on no-shows and there were no crossovers). See appendix section A.2 for details. 44

Better access to information generally could help prospective homebuyers make better decisions about whether or not to purchase a home. Among purchasers, increased access to information might improve their ability to search for and choose an affordable home and to select and qualify for appropriate financing.

Exhibit 5.1: Overall Impact of the Demonstration’s Homebuyer Education and Counseling on Preparedness and Search

Impact of

Impact of

Treatment Control Group Being Offered Taking Up

Outcome

Group Mean

Mean

Services

Services

Study participant was confident that he/she could find the

74.7

69.8

4.9***

8.1***

information he/she needed about the homebuying processa

(%)

Number of lenders from which the study participant received

1.58

1.64

−0.05

−0.09

price quotesa

Study participant purchased a homeb (%)

62.2

61.7

0.5

0.9

Study participant was very satisfied with the homebuying

31.8

28.7

3.1**

5.1**

processa (%)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, 5,708 sample members.

*** Dif erence is statistically significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

44 The impact on improved confidence appears to be driven by the participants offered remote services, who experienced an impact of 5.6 percentage points (exhibit 5.2). That said, the difference between the in-person and remote impacts is not detectably different from zero.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 53

5. IMPACTS: PREPAREDNESS & SEARCH

• Homebuyer education and counseling services did not detectably affect the intensity of search for financing, as measured by the number of price quotes study participants received from lenders.

Treatment and control group members each received price quotes from an average of 1.6 lenders, indicating that homebuyer education and counseling services did not detectably lead to a more exhaustive search for the best financing.

• Homebuyer education and counseling services did not detectably affect rates of home purchase overal .

The control and treatment groups purchased homes at similar rates (62 percent). Homebuyer education and counseling services are designed to provide the skills needed to determine whether homes are affordable and financing options are appropriate for the homebuyer’s individual circumstances. This can help qualified clients purchase a home, meanwhile dissuading underqualified prospective homebuyers from purchasing or causing them to postpone their purchase until conditions are right. However, this analysis does not reveal whether the composition of treatment and control group purchasers differs: if a greater number of “better suited” treatment group members are buying homes and fewer “less suited” ones are, then the treatment group home purchase rate overall could remain the same as the control group rate, although the types of people would differ. 45

Although no overall impact was detected, the impact of homebuyer education and counseling services did differ across select subgroups of interest. 46 Among study participants age 29 or younger at baseline, 68.8 percent of those offered homebuyer education and counseling services purchased a home. This home purchase rate is 3.3 percentage points higher than the rate of their age 29 or younger control group counterparts. In contrast, we find no detectable evidence that homebuyer education and counseling services increased home purchase rates for those age 30 or older (exhibit E.2). Because of the historically low rate of homeownership for younger Americans, this finding is encouraging, as it suggests that the intervention is helping these 45 The number of quotes and the share of study participants that purchased a home does not necessarily reveal anything about the quality of the mortgage loan. To further explore whether homebuyer education and counseling influenced the quality of the mortgage loan, we constructed a variety of measures that capture (1) whether the study participant received downpayment assistance; (2) whether the study participant has an FHA loan; (3) loan-to-value ratio categories; (4) whether the study participant has a fixed- or adjustable-rate mortgage; and (5) mortgage loan interest rate categories. For each of these categorical outcomes measures, we coded study participants with no mortgage loan as the reference category. Using a multinomial logit model that included our standard set of covariates (as described in appendix section A.1), we found no evidence that homebuyer education and counseling affected these measures.

46 We find no evidence that homebuyer education and counseling services affected rates of home purchase for subgroups defined by race/ethnicity, including the subgroup of White non-Hispanics, African-American non-Hispanics, or a subgroup comprising all race/ethnicities besides White non-Hispanics. More broadly, as presented in exhibits E.4 and E.5, we find no evidence of systematic between-subgroup differences in impacts on preparedness and search, financial capability, or sustainable homeownership for subgroups defined by race/ethnicity.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 54

5. IMPACTS: PREPAREDNESS & SEARCH

individuals overcome barriers to purchasing homes (see section 8.3 for more discussion on this finding).

Homebuyer education and counseling services also had a larger impact on home purchase rates for participants with more cash available for downpayment (>$15,000), relative to those with less downpayment cash available (exhibit E.11). 47 Those with $15,000 or more on hand generally had better financial metrics than those with less than $15,000 on hand, as evidenced by their higher incomes and credit scores. 48 Homebuyer education and counseling services might have provided this financially healthier group with key additional information needed to find a home and secure a mortgage with favorable terms. At the same time, homebuyer education and counseling services might have dissuaded the low downpayment subgroup (with comparatively worse financial health) from purchasing a home in the short term, and this could be a sensible outcome for them. 49

• Homebuyer education and counseling services increased satisfaction with the homebuying process.

Among those in the treatment groups, 31.8 percent said they were “very satisfied” with the homebuying process compared with 28.7 percent of the control group. Although these levels are relatively low, with less than one-third of all study participants reporting levels of being very satisfied, they indicate that homebuyer education and counseling services increased satisfaction with the homebuying process. 50 Homebuyer education and counseling services might increase 47 Cash on hand for downpayment and closing costs is self-reported in the baseline survey. For those participants who had already purchased a home at the time of study enrollment, cash on hand for downpayment and closing costs refers to the funds that had been available at the time of purchase.

48 Those study participants with $15,000 or more for downpayment and closing costs (51 percent of the study sample) had comparatively high baseline incomes and credit scores relative to those with less than $15,000 for downpayment and closing costs (49 percent of the study sample). Those with $15,000 or more on hand had a mean income (with co-borrowers) of $70,000 and mean credit score of 733, whereas the participants with less than $15,000 on hand had a mean income (with co-borrowers) of $49,000 and mean credit score of 679.

49 Policymakers, scholars, and practitioners might wonder whether there are different—perhaps larger—impacts of homebuyer education and counseling on those who purchase a home versus those who do not purchase. Because the decision to purchase could be affected by the intervention, we cannot simply compare treatment and control group outcomes within the subgroup of purchasers (or subgroup of nonpurchasers) without introducing bias into the impact estimates. Appendix G reports an alternate analysis, which is our best attempt to provide evidence on the topic of differential impacts for those who purchase versus those who do not.

50 In addition, we estimated the impact of the intervention on an alternative satisfaction measure, where this measure was set equal to 1 if the study participant reported being either “satisfied” or “very satisfied” with the home purchase process and 0 otherwise. The regression-adjusted mean values of this outcome show much higher levels of (general) satisfaction: 79.2 percent of the treatment group and 75.5 percent of the control group were satisfied or very satisfied with the home purchase process. The 3.7 percentage point difference (impact) is statistically significant ( p<0.01) and represents a 6.1 percent relative improvement of the treatment group over the control group.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 55

5. IMPACTS: PREPAREDNESS & SEARCH

satisfaction by providing valuable information that helps treatment group members navigate the stressful and complex homebuying process.

As documented by DeMarco et al. (2017), the study’s focus group participants who completed all homebuyer education and counseling services reported that one of the most meaningful benefits of completing services was reducing the stress they felt from the homebuying process. 51

Focus group participants who completed all services also said that completing homebuyer education and counseling services helped them gain knowledge needed to navigate the homebuying process and better negotiate and manage their time and resources.

• The study’s overall impact (of 3.1 percentage points) on being very satisfied with the homebuying process is driven by the experience of those offered in-person services.

The study’s overall impact masks a sizable between-mode difference: those offered in-person services experienced a large (8.9 percentage point) impact on this outcome, whereas those offered remote services were no more or less likely to be very satisfied with the process than the control group (exhibit 5.2). The magnitude of the impact of in-person services on satisfaction with the homebuying process is notably larger if we consider that only about one-fourth of those offered in-person services actually took up services. Indeed, we find that those who took up in-person services were 28.5 percentage points more likely to report that they were very satisfied with the homebuying process than their control group counterparts (exhibit D.1).

This finding about differential delivery mode effects on satisfaction with the homebuying process can potentially be explained by considering how the content of homebuyer education and counseling services is delivered through the in-person and remote service delivery modes. The finding that those offered in-person services showed greater satisfaction with the homebuying process could relate to the personal touch associated with in-person service provision. For instance, the study’s focus group participants described the experience of receiving in-person homebuyer education in a classroom with people in the same community who are in a similar situation as one of the benefits of the in-person format. This experience helps to alleviate stress associated with the homebuying process. As study participants who take part in in-person education workshops hear other prospective homebuyers’ questions, talk with them during class breaks, and connect with professionals presenting the workshop, they could feel increased connection to people facing the same challenges and gain a greater sense that someone is “in their corner.”

51 Focus group interviews were separately conducted with study participants who did not complete services to explore their decisions about whether to complete homebuyer education and counseling and discuss obstacles they faced initiating or completing services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 56

5. IMPACTS: PREPAREDNESS & SEARCH

Exhibit 5.2: Impact by Service Delivery Mode on Preparedness and Search

Difference

Impact of Being

Between In-

Of ered In- Impact of Being Person and

Person

Of ered Remote

Remote

Outcome

Services

Services

Impacts

Study participant was confident that he/she could find the information

1.3

5.6***

−4.2

he/she needed about the homebuying processa (%)

Number of lenders from which the study participant received price quotesa

−0.11*

−0.04

−0.07

Study participant purchased a homeb (%)

1.2

0.1

1.1

Study participant was very satisfied with the homebuying processa (%)

8.9***

1.5

7.4***

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

*** Dif erence is statistically significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of in-person services, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806); and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n= 1,184). For the analysis of remote services, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

In summary, we find multiple favorable impacts of the Demonstration’s homebuyer education and counseling within the domain of preparedness and search. For instance, we find that homebuyer education and counseling services improved the treatment group members’ ability to find needed information related to the home purchase. Homebuyer education and counseling services also increased satisfaction with the homebuying process, a finding that is associated with in-person services rather than remote services. In addition, although no impact on home purchase rates was detected overall, we do observe some subgroup differences: both younger study participants and participants with more cash available for downpayment showed higher home purchase rates.

With about 60 percent of the sample having purchased a home as of this short-term followup time point, it will be important to examine the sample’s experiences and outcomes in the longer term, to see when—if at all—the remaining 40 percent of the sample buys a home, under what conditions, and the ways in which their financial health evolves.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 57

6. Impacts on Financial Capability

Key Findings: Impacts on Financial Capability

Within the domain of financial capability, we analyzed three categories of variables—financial knowledge, behaviors, and indicators.

Impact on Financial Knowledge:

• Homebuyer education and counseling services improved treatment group members’ knowledge of how to correct inaccurate information in their credit report.

• These services improved treatment group members’ knowledge that they should proactively contact counseling agencies or other nonprofits in times of financial distress.

• These services had no detectable impact on treatment group members’ mortgage literacy or knowledge that they should proactively contact lenders or pay their mortgage first in times of financial distress.

Impact on Financial Behavior:

• Homebuyer education and counseling services increased the likelihood that treatment group members have their mortgage payments automatical y deducted from their bank account.

• These services had no detectable impact on the likelihood that treatment group members use budgets or pay off credit cards in full each month.

Impact on Financial Indicators:

• Overall, homebuyer education and counseling services had no detectable impact on credit scores or savings.

• Homebuyer education and counseling services increased levels of nonhousing debt and monthly debt-to-income ratios for those in the treatment group relative to the control group. These findings appear to be driven by increased student loan debt for those study participants with credit scores below 680 who were at an early stage in the homebuying process at the time of study enrol ment. We do find evidence of an associated increase in saving for those with credit scores below 680, indicating that these participants with lower credit scores might be taking out additional student loans (and/or deferring payment on existing student loans) to increase the availability of liquid savings (perhaps for a downpayment).

• Those in the treatment groups reported a lower ability to cover all bil s at the end of the month than the control group. However, there was no detectable impact on bankruptcy, foreclosure, or repossession as of this short-term followup.

This second of three impact chapters reports the impact of the Demonstration’s homebuyer education and counseling services on outcomes that fall within the domain of financial capability. 52 This domain encompasses a wide range of financial knowledge, attitudes and 52 Chapter 5 reports on the preparedness and search domain; and chapter 7 on the sustainable homeownership domain. Chapter 3 and appendix A provide detail on how the impacts reported in this chapter (as well as in chapters 5 and 7) are estimated. See the textbox How to Read the Impact Exhibits in This Report on page 51. Expanded results for the overall impact are reported in appendix C, and expanded results for delivery mode effects are reported in appendix D. Appendix E presents subgroup impacts.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 58

6. IMPACTS: FINANCIAL CAPABILITY

behaviors, and other factors that can affect a prospective first-time homebuyer’s financial situation.

In theory, homebuyer education and counseling services should improve service recipients’

knowledge of financial terminology—in particular, mortgage terminology—and their

understanding of the financial practices necessary to qualify for and sustain homeownership.

This knowledge should translate into improved financial behaviors such as more responsible budget, credit, and money management practices. Over time, these improved financial behaviors should ultimately be reflected in improved measures of traditional financial indicators (for example, improved credit, lower debt, and increased savings). This chapter elaborates on whether and how homebuyer education and counseling influences a set of financial capability outcomes in these expected ways.

6.1 Impacts on Financial Knowledge

• Homebuyer education and counseling services improved treatment group members’ ability to correct inaccurate information in their credit report.

Inaccurate information on a credit report can have unfavorable effects on credit scores and, consequently, serve as a barrier to accessing credit. 53 Responding to the Short-Term Follow-Up Survey, 79.6 percent of participants in the treatment groups reported knowing how to correct inaccurate information in their credit report, 2.9 percentage points more than the control group, as shown in exhibit 6.1. When we adjust this estimate for no-shows, the estimated impact of taking up services is 4.8 percentage points. 54 These findings indicate that homebuyer education and counseling services had a positive impact on this intermediate step in helping prospective first-time homebuyers improve their credit score.

This overall favorable impact of homebuyer education and counseling services on participants’

self-reported knowledge of how to correct inaccurate information in a credit report is driven by the impact of remote services, as shown in exhibit 6.2 (on page 61). Those offered remote services were 4.1 percentage points more likely to report knowing how to correct inaccurate information in their credit report than were their control group counterparts. In contrast, the impact on those offered in-person services was not statistically different from zero.

53 Prospective employers and owners of rental housing also sometimes use credit scores as a screening device.

54 The impact of taking up services presented in this chapter is computed under the assumption that there is no effect of the intervention on no-shows and there were no crossovers. See appendix section A.2 for details.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 59

6. IMPACTS: FINANCIAL CAPABILITY

• Homebuyer education and counseling services had a favorable impact on treatment group members’ knowledge that they should proactively communicate with their counseling agency, consumer credit counseling agency, or another nonprofit organization in times of financial distress.

The longer borrowers wait to reach out for help when they face difficulties meeting their mortgage obligations, the less likely they are to recover from a delinquency (Cutts and Merrill, 2008). Treatment group members were 4.4 percentage points more likely than control group members to report that if in financial difficulty, they would contact their counseling agency, consumer credit counseling agency, or other nonprofit organization for assistance prior to missing a mortgage payment (exhibit 6.1). 55 When we adjust this estimate to account for no-shows, the estimated impact of taking up services is 7.2 percentage points.

Exhibit 6.1: Overall Impact of the Demonstration’s Homebuyer Education and Counseling on Financial Knowledge

Impact of

Impact of

Treatment

Control Being Offered Taking Up

Outcome

Group Mean Group Mean Services

Services

Study participant knows how to correct inaccurate information in

79.6

76.7

2.9***

4.8***

credit reporta (%)

If in financial difficulty, study participant would contact his/her

21.3

16.9

4.4***

7.2***

counseling agency, consumer credit counseling agency, or other

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

40.6

38.0

2.6

4.2

lender for assistance prior to missing a mortgage paymenta (%)

Number of correct answers (out of four) to mortgage literacy quiza

2.77

2.75

0.02

0.04

If having financial problems and could not pay all the bil s, study

82.1

82.7

−0.7

−1.1

participant would pay the mortgage firsta (%)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, 4,546

sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

55 This outcome was coded as 1 if the respondent had a mortgage and would contact his/her counseling agency, consumer credit counseling agency, or other nonprofit organization for assistance prior to missing a mortgage payment; it was coded as 0 if the respondent would not contact any of them for assistance prior to missing a mortgage payment or if the respondent did not have a mortgage loan. To determine whether this finding is driven by differential mortgage loan rates across treatment groups, we estimated the impact of the intervention on an indicator for whether the respondent had a mortgage loan. We found no detectable evidence that treatment group members are more likely to have a mortgage loan, alleviating concerns that this finding is driven by differential mortgage loan rates across treatment groups.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 60

6. IMPACTS: FINANCIAL CAPABILITY

Exhibit 6.2: Impact by Service Mode on Financial Knowledge

Difference

Impact of

Impact of

Between In-

Being Offered Being Offered Person and

In-Person

Remote

Remote

Outcome

Services

Services

Impacts

Study participant knows how to correct inaccurate information in credit reporta

−1.3

4.1***

−5.4**

(%)

If in financial difficulty, study participant would contact his/her counseling

0.7

5.2***

−4.5*

agency, consumer credit counseling agency, or other nonprofit organization for

assistance prior to missing a mortgage paymenta (%)

If in financial dif iculty, study participant would contact his/her lender for

1.3

3.0*

−1.7

assistance prior to missing a mortgage paymenta (%)

Number of correct answers (out of four) to mortgage literacy quiza

0.08*

0.01

0.08

If having financial problems and could not pay all the bil s, study participant

2.0

−1.0

3.1

would pay the mortgage firsta (%)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of in-person services, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806); and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n=1,184). For the analysis of remote services, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

The overall favorable impact of homebuyer education and counseling services on participants’

knowledge that they should proactively contact a counseling agency or other nonprofit prior to missing a mortgage payment is driven by the experience of those offered remote services. Those offered remote services were 5.2 percentage points more likely than the control group to report they would take this action, whereas there was no detectable effect among those who were offered in-person services (see exhibit 6.2).

• Overall, homebuyer education and counseling services had no detectable impact on treatment group members’ knowledge that they should proactively communicate with lenders in times of financial distress.

Not overall, but being offered remote homebuyer education and counseling services did have a positive impact on this outcome. Participants offered remote services were 3.0 percentage points more likely than their control group counterparts to say that they would contact their lender if in financial distress, prior to missing a payment (see exhibit 6.2). 56 When homeowners are in 56 DeMarco et al. (2016), who reported preliminary impacts of being offered homebuyer education and counseling using data on a subset of “early enrollees” who enrolled in this study before December 1, 2014, found evidence of The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 61

6. IMPACTS: FINANCIAL CAPABILITY

financial distress and at risk of missing a mortgage payment, proactive contact with a lender can mean that the borrower has more options for using forbearance, repayment plans, or other strategies (such as loan modifications) to avoid default, and this finding indicates that remote services encouraged such behavior.

• Overall, homebuyer education and counseling services had no detectable impact on treatment group members’ mortgage literacy.

On a four-point mortgage literacy quiz, the pooled treatment group and the control group scored, on average, 2.77 and 2.75 out of 4 points, respectively. Homebuyer education and counseling services did not have an overall impact on mortgage literacy; however, in-person homebuyer education and counseling services did have a small, statistically significant impact. Those offered in-person services scored an average of 0.08 point higher than the control group (see exhibit 6.2). 57 In contrast, those offered remote services showed no detectable improvements in mortgage literacy. In addition, we find that participants age 29 or younger at baseline improved their mortgage literacy score by 0.11 point, whereas those who were age 30 and older showed no detectable improvement (exhibit E.2). Although these impacts are small in magnitude, they indicate that in-person services might lead to improved mortgage literacy, which has been shown to be associated with better mortgage outcomes (An, Bostic, and Yao, 2015).

an overall positive impact on this outcome. Though the overall impact of 2.6 percentage points for the full sample reported in exhibit 6.1 is not statistically different from zero, it is also not statistically different from the overall impact of 3.9 percentage points for the early enrollee sample. Even so, there are at least a few reasons why the impact findings for DeMarco et al. (2016) could differ from the findings included in this report for the full sample.

First, study participants who enrolled before December 1, 2014, could differ (in either observable or unobservable characteristics) from study participants who enrolled after this date. This could lead to a different composition of study participants in the early enrollee sample and study participants in the full sample. A second difference between the early enrollee sample and the full study sample used for this report is that the early enrollee sample contained a comparatively large share of treatment group members who were offered in-person services (because the in-person treatment group was switched to a “choice” treatment group in September 2014, and three-fourths of choice treatment group members selected remote services). Third, it could be that homebuyer education and counseling services had a different impact in years 2014–2015 when outcomes were observed for the early enrollee sample than it did in the years 2016–2017 when outcomes were observed for study participants who enrolled after the early enrollee sample cutoff date.

57 DeMarco et al. (2016) reported preliminary impacts of being offered homebuyer education and counseling using data on a subset of “early enrollees” and found evidence of an overall positive impact on mortgage literacy. In contrast, this report uses data from the full study sample and finds no evidence of an overall impact on mortgage literacy, but it does find that in-person services improved mortgage literacy. One difference between the early enrollee sample and the full study sample used for this report is that the early enrollee sample comprised a comparatively large share of treatment group members who were offered in-person services (because the in-person treatment group was modified to a “choice” treatment group in September 2014, and three-fourths of choice treatment group members selected remote services instead of in-person services). Given this, one explanation for why DeMarco et al. (2016) found evidence of an overall impact on mortgage literacy yet this report does not is that the early enrollee sample comprised a relatively large share of treatment group members who were offered in-person services who, on average, experienced the positive impact on mortgage literacy.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 62

6. IMPACTS: FINANCIAL CAPABILITY

Mortgage Literacy Quiz

Answer These True/False Questions:

1. The interest rate on a mortgage loan is the same thing as the annual percentage rate (APR).

2. A home equity loan is secured by your house.

3. When you first get a mortgage loan, only a small portion of your monthly payment, if any, reduces the amount you owe. Most of your monthly payment is applied to interest.

4. The loan officer is legal y obligated to tell you if you qualify for a different loan product that has a lower cost.

Scoring: One point for each correct answer (1/False + 2/True + 3/True + 4/False)

• Overall, homebuyer education and counseling services had no detectable impact on treatment group members’ knowledge that they should pay their mortgage first in times of financial distress.

Sustainable homeownership depends, in part, on prioritizing mortgage payments over other financial obligations (such as credit card debt) in the event of financial distress. 58 We find no detectable impact of homebuyer education and counseling on participants’ knowing that they should prioritize mortgage payments in this way (see exhibit 6.1). Approximately four out of five members of both the treatment and the control groups responded that they should pay their mortgage first in times of financial distress (82.1 percent and 82.7 percent).

6.2 Impacts on Financial Behavior

• Homebuyer education and counseling services had no detectable impact on treatment group members’ use of a budget or on whether they usual y pay off their credit card in full.

Good money management practices—such as using a budget and not carrying credit card debt—

contribute to a financially responsible lifestyle, which supports attaining and sustaining homeownership. At the time of the Short-Term Follow-Up Survey, the proportions of the treatment group and control group members who had a budget and compared it against their actual spending were not statistically significantly different, nor were the proportions of the groups who usually pay off their credit card in full each month to avoid interest charges (exhibit 6.3).

• Homebuyer education and counseling services favorably affected the likelihood that treatment group members wil set up their mortgage payments to be automatical y deducted from their bank account.

When homeowners set up their mortgage payments to come out of their bank account

automatically, they decrease the likelihood that they will forget to make a payment and therefore 58 Homeowners should pay their mortgage first when in financial distress for several reasons. First, unlike credit card companies, mortgage lenders do not accept partial payments. Second, mortgages are “secured” loans, meaning that they are backed by an asset (the home) that the homeowner can lose. Third, missed mortgage payments have larger impacts on credit scores than do other missed debt payments.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 63

6. IMPACTS: FINANCIAL CAPABILITY

become delinquent on their mortgage. As shown in exhibit 6.3, treatment group members were 5.0 percentage points more likely to have their mortgage payments automatically deducted from a bank account than were the control group. Adjusting for take-up rates, the impact is 8.3

percentage points. As shown in exhibit 6.4, there are no statistically significant differences between the in-person and remote impacts on financial behaviors.

Exhibit 6.3: Overall Impact of the Demonstration’s Homebuyer Education and Counseling on Financial Behaviors

Impact of

Impact of

Treatment

Control Being Offered Taking Up

Outcome

Group Mean Group Mean Services

Services

Study participant has a budget and often compares it against

34.0

35.8

−1.8

−3.0

actual spendinga (%)

Study participant usually pays credit card balance in full to avoid

72.0

71.0

1.0

1.7

interest chargesa (%)

Regularly required mortgage payment is automatically deducted

30.4

25.4

5.0***

8.3***

from a bank accounta (%)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, 4,546

sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

Exhibit 6.4: Impact by Service Mode on Financial Behaviors

Difference

Impact of

Impact of

Between In-

Being Offered Being Offered Person and

In-Person

Remote

Remote

Outcome

Services

Services

Impacts

Study participant has a budget and often compares it against actual spendinga

−2.6

−1.8

−0.8

(%)

Study participant usually pays credit card balance in full to avoid interest

−0.3

1.6

−1.9

chargesa (%)

Regularly required mortgage payment is automatically deducted from a bank

2.5

5.5***

−3.0

accounta (%)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of in-person services, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806); and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n= 1,184). For the analysis of remote services, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 64

6. IMPACTS: FINANCIAL CAPABILITY

6.3 Impacts on Financial Indicators

• Homebuyer education and counseling services had no overall impact on credit scores; however, in-person homebuyer education and counseling services did help treatment group members increase their credit scores over a critical credit score threshold.

The average credit scores for the pooled treatment group and the control group were 706 and 707, respectively, and this one-point difference is not statistically different from zero (exhibit 6.5). The overall proportion of participants whose credit scores were above 620—an important underwriting threshold—was also not detectably different between the pooled treatment group and the control group.

Exhibit 6.5: Overall Impact of the Demonstration’s Homebuyer Education and Counseling on Financial Indicators

Impact of

Impact of

Treatment

Control Being Offered Taking Up

Outcome

Group Mean Group Mean Services

Services

Credit

Credit score (out of 850)c

706.0

707.1

−1.0

−1.9

Study participant has credit score above or equal to 620c (%)

87.2

87.3

−0.1

−0.2

Debt

Student loan balancec ($)

10,392

9,843

549*

999*

Credit card balancec ($)

3,850

3,791

58

106

Total nonhousing debtc ($)

24,506

23,715

791*

1,439*

Total monthly debt-to-income ratio (back-end ratio)b

26.6

25.6

1.0**

1.8**

Total monthly debt-to-income ratio (back-end ratio) greater than

17.0

16.1

0.9

1.6

0.43b (%)

Savings

Study participant usually has enough savings set aside to cover 3

72.0

72.1

−0.0

−0.1

months of expensesa (%)

Total savings and investmentsa ($)

50,216

49,220

996

1,628

Other financial indicators

Study participant occasional y does not have enough money to

10.9

9.2

1.7**

2.8**

cover all of the bil s at the end of the montha (%)

Nonhousing debt bankruptcy, foreclosure, or repossession

5.1

4.6

0.4

0.8

indicatorc (%)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, 4,546

sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, 5,708 sample members of the study sample.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. Appendix A details the analytic methods; and appendix B provides additional detail on the construction of measures.

However, in-person services did help participants exceed this credit threshold: among those offered in-person services, 87 percent had credit scores above 620, which was 2 percentage The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 65

6. IMPACTS: FINANCIAL CAPABILITY

points greater than the 85 percent of the control group who had credit scores above the threshold.

(Exhibit 6.6 shows the impact of in-person services; see exhibit D.1 for expanded results.)59 This difference could indicate that in-person services are helping participants repair damaged credit or correct inaccuracies in their credit reports, bringing their scores above this important underwriting threshold.

Exhibit 6.6: Impact by Service Mode on Financial Indicators

Difference

Impact of

Impact of

Between In-

Being Offered Being Offered Person and

In-Person

Remote

Remote

Outcome

Services

Services

Impacts

Credit

Credit score (out of 850)c

2.9

−2.3

5.2*

Study participant has a credit score above or equal to 620c (%)

2.2*

−0.9

3.1**

Debt

Student loan balancec ($)

−295

779**

−1,074*

Credit card balancec ($)

−71

100

−171

Total nonhousing debtc ($)

−28

971*

−999

Total monthly debt-to-income ratio (back-end ratio)b

1.3

0.9*

0.4

Total monthly debt-to-income ratio (back-end ratio) greater than 0.43b (%)

1.5

0.7

0.8

Savings

Study participant usually has enough savings set aside to cover 3 months of

−1.2

0.2

−1.3

expensesa (%)

Total savings and investmentsa ($)

3,335

322

3,012

Other financial indicators

Study participant occasional y does not have enough money to cover all of the

1.9

1.6

0.3

bil s at the end of the montha (%)

Nonhousing debt bankruptcy, foreclosure, or repossession indicatorc (%)

0.7

0.4

0.3

59 DeMarco et al. (2016) reported preliminary impacts of being offered homebuyer education and counseling using data on a subset of “early enrollees” and found evidence of an overall positive impact on the share of the sample with a credit score greater than or equal to 620. In contrast, this report uses data from the full study sample and finds no evidence of an overall impact on this outcome, but does find that in-person services led to a higher share of the sample with a credit score greater than or equal to 620. One difference between the early enrollee sample and the full study sample used for this report is that the early enrollee sample comprised a comparatively large share of treatment group members who were offered in-person services (because the in-person treatment group was modified to a “choice” treatment group in September 2014 and three-fourths of choice treatment group members selected remote services instead of in-person services). Given this, one explanation for why DeMarco et al. (2016) found evidence of an overall impact on the share of the sample with a credit score greater than or equal to 620 yet this report does not is that the early enrollee sample comprised a relatively large share of treatment group members who were offered in-person services who, on average, experienced a positive impact on this outcome.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 66

6. IMPACTS: FINANCIAL CAPABILITY

Difference

Impact of

Impact of

Between In-

Being Offered Being Offered Person and

In-Person

Remote

Remote

Outcome

Services

Services

Impacts

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of in-person services, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806); and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n= 1,184). For the analysis of remote services, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

• Homebuyer education and counseling services increased nonhousing debt through increased student loan debt.

Although the treatment and control groups had similar levels of nonhousing debt at baseline and purchased homes at similar rates as of followup, those offered homebuyer education and counseling services had $791 more in nonhousing debt at followup than their control group counterparts (see exhibit 6.6). 60 This impact is driven by a positive impact on student loan debt—one component of nonhousing debt—with treatment group members having an average of $549 more in student loan debt at followup than control group members. 61 This finding is driven by the experience of those offered remote services, who had $971 more in nonhousing debt (and $779 more in student loan debt) than did their control group counterparts at short-term followup. 62

Although it is possible that incurring additional student loan debt is reflecting that individuals are investing in their skills in a way that may enhance future financial stability, whether more 60 Nonhousing debt includes all nonhousing debts recorded by the credit bureau, including student loan debt, credit card debt, auto loan debt, and medical debt. To reduce the risk that outliers are driving the results, total nonhousing debt was top coded at its 99th percentile.

61 We find no detectable evidence that homebuyer education and counseling services are related to levels of credit card debt or auto loan debt, which comprise the other major categories of nonhousing debt.

62 Importantly, we find that both the control and treatment groups have a large proportion of participants who experienced increases in their student debt levels from baseline to followup. Specifically, 37 percent of treatment group participants experienced an increase in their student debt levels from baseline to followup, compared to 42

percent of control group members. The higher levels of average student debt across treatment group participants relative to control group participants therefore could represent a combination of higher increases in student debt for some participants and smaller decreases in student debt for others.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 67

6. IMPACTS: FINANCIAL CAPABILITY

student loan debt actually results in a favorable or unfavorable consequence is not yet known.

The study’s Long-Term Impact Report will revisit this finding by, for example, reporting whether the offer of homebuyer education and counseling led to mortgage loan or student loan delinquencies for the subgroup of study participants with higher levels of student loan debt.

In addition, we find that the higher levels of nonhousing debt and student loan debt are driven by select subgroups of the study sample, as follows:

• Among study participants with credit scores below 680 at baseline, treatment group members had followup levels of total nonhousing debt that were $1,585 higher and student loan debt that were $1,583 higher than control group members (exhibit E.10). In comparison, no significant differences existed among the subgroup of student participants with credit scores of 680 or above.

• Among the subgroup of study participants with income less than 80 percent of the area median, treatment group members had followup levels of total nonhousing debt and student loan debt that were higher by $1,078 and $1,037 than control group members (exhibit E.13).

In comparison, no significant differences existed among the subgroup of student participants with income higher than 80 percent of the area median income.

• Among study participants who were at an early stage in the homebuying process at the time of study enrollment, treatment group members had followup levels of total nonhousing debt and student loan debt that were $1,038 and $1,025 higher than control group members (exhibit E.9). In comparison, no significant differences existed among the subgroup of student participants who were at a late stage in the homebuying process.

• Among study participants living in relatively expensive areas, treatment group members had followup levels of total nonhousing debt and student loan debt that were $1,023 and $603

higher than control group members (exhibit E.3). 63 In comparison, no significant differences existed among the subgroup of student participants living in relatively inexpensive areas.

Collectively, these findings indicate that the treatment group’s higher incidence of nonhousing debt and student loan debt seems to be driven by study participants who have relatively low credit scores and incomes, as well as being at an early stage in the homebuying process and 63 The “low housing affordability” subgroup includes study participants living in metropolitan areas where the ratio of the median value of an owner-occupied unit to the median income is greater than or equal to the corresponding ratio for the United States: 68.1 percent of the study sample lived in areas where this was the case. Examples of metropolitan statistical areas in the low housing affordability subgroup include Boston-Quincy, MA; Washington-Arlington-Alexandria, DC-VA-MD-WV; and San Francisco-San Mateo-Redwood City, California. Similarly, the

“high housing affordability” subgroup (the remaining 31.9 percent of the study sample) includes study participants living in areas where the ratio of the median value of an owner-occupied unit to median income is less than the U.S.-

wide ratio. Examples of areas in the high housing affordability subgroup include St. Louis, MO; Atlanta-Sandy Springs-Marietta, GA; and Minneapolis-St. Paul-Bloomington, MN-WI.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 68

6. IMPACTS: FINANCIAL CAPABILITY

living in relatively expensive areas. 64 Borrowers with relatively low incomes and credit scores might not qualify for some of the lower downpayment mortgages that are available. Potentially, treatment group members are shifting resources toward downpayment savings and are incurring greater student debt in the process. 65

More research is needed into the exact causes of these findings, but it could be that study treatment group participants with these characteristics receive advice to defer payments on student loans so they can shift financial resources toward building up savings in anticipation of putting down a larger downpayment. This is consistent with another finding: within the subgroup of those with credit scores below 680, treatment group members had followup savings $4,378

higher than control group members (exhibit E.10). 66

• Homebuyer education and counseling services increase the monthly debt-to-income ratio In the treatment group, relative to the control group.

The monthly debt-to-income ratio is all monthly debt obligations—including mortgage debt—

divided by monthly income. This ratio is helpful because it provides context for the level of participants’ monthly debt obligations. We find that homebuyer education and counseling services results in a higher monthly debt-to-income ratio (relative to the control group) as a result of treatment group members having higher monthly student loan payments relative to control group members (see exhibit 6.5).

However, we find no detectable evidence that homebuyer education and counseling services affected the likelihood that service recipients have a monthly debt-to-income ratio greater than 64 We also find that (1) among study participants age 30 or older, the offer of homebuyer education and counseling increased total nonhousing debt and student loan debt by $919 and $879, respectively (exhibit E.2); and (2) among study participants with an English language preference, the offer of homebuyer education and counseling increased total nonhousing debt and student loan debt by $1,008 and $617, respectively (exhibit E.7).

65 An important context for the finding on student debt is that large portions of both the control and treatment groups experienced increases in their student debt levels between the time they entered the study and the Short-Term Follow-Up Survey. Thirty-seven (37) percent of the treatment group experienced an increase in their student debt levels, as did 42 percent of the control group. Overall, at baseline, the student loan debt levels were the same, at about $9,150; at followup, the raw control group mean was $9,930 (a roughly $780 increase from baseline), and the raw treatment group mean was $10,055 (a roughly $890 increase from baseline). These numbers correspond to the treatment group experiencing about a 9.8-percent increase, whereas the control group experienced about an 8.5-percent increase. (Note that these unadjusted followup treatment and control group levels of student debt differ slightly from the regression-adjusted treatment and control group levels of student debt reported in exhibit 6.5). The higher levels of average student debt across the treatment group relative to the control group could represent a combination of higher increases in student debt for some participants and smaller decreases in student debt for others.

66 Full subgroup results appear in appendix E. They include a subset of all possible outcomes, selected to exercise restraint on the number of analyses; that subset does not include total savings. Additional analyses (not shown) undertaken to explore the impact on student debt involved considering the impact of homebuyer education and counseling for the subgroup of study participants with credit scores below 680, as reported here.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 69

6. IMPACTS: FINANCIAL CAPABILITY

43 percent, the upper limit on this ratio as specified by FHA guidelines. 67 Collectively, these findings indicate that treatment group members have marginally larger monthly debt burdens than do control group members, but that the treatment and control groups were equally likely to have monthly debt burdens high enough to pose barriers to home purchase.

• Overall, homebuyer education and counseling services did not have a detectable impact on total savings.

There is no detectable impact on the proportions of the treatment and control groups that reported having enough savings set aside to cover at least 3 months of expenses (see exhibit 6.5).

These findings do not differ by service delivery mode (see exhibit 6.6). However, as discussed earlier, among participants with credit scores below 680 (the same group that experienced an increase in student debt as a result of the homebuyer education and counseling services), we observe an increase in savings and investments of $4,378 relative to the control group (exhibit E.10). 68

• Homebuyer education and counseling services had an adverse effect on the treatment group’s reported ability to cover al bil s at the end of the month; however, there are no adverse effects of the homebuyer education and counseling services on bankruptcy, foreclosure, or repossessions. 69

We find that treatment group members are 1.7 percentage points more likely than control group members to report they occasionally do not have enough money to cover all their bills at the end of the month (exhibit 6.5). However, we find no evidence that homebuyer education and counseling had an adverse effect on loan performance or on a measure of whether study participants experienced a bankruptcy, charge-off, foreclosure, or repossession on a non-67 When underwriting mortgages, lenders rely on two metrics to determine housing affordability: the front-end and the back-end debt-to-income (DTI) ratios. The front-end ratio is the monthly housing costs (that is, mortgage payments, taxes, and insurance) divided by the borrower’s monthly income. The back-end ratio is the total recurring monthly debt (housing costs plus car payments, credit card, student loans, and so on) divided by monthly income.

Placing limits on these DTI ratios helps ensure that borrowers can afford to pay back their mortgage while meeting other necessary living expenses. For example, in 2015, the Federal Housing Administration placed limits on the front-end and back-end ratios of 0.31 and 0.43, respectively. That is, for mortgages insured by FHA, the total monthly mortgage payment (including taxes and insurance) could not exceed 31 percent of the borrower’s monthly income, whereas the total mortgage payment plus any non-mortgage debt payments could not exceed 43 percent.

Fannie Mae and Freddie Mac prefer loans with a back-end ratio under 0.36 (although can go higher with compensating factors) (DeMarco et al., 2017).

68 For participants with credit scores below 680 and at least $1 of student debt at baseline, treatment group members experienced an increase in student debt of $1,987, compared to an increase of $1,326 for control group members.

During the same time, control group members experienced an average decrease in savings of $3,294, whereas the treatment group members actually increased savings by $166. These descriptive findings support the idea that borrowers with credit scores below 680 and with student debt who receive the intervention might be more inclined to take on additional student loans (or defer payment on existing loans) in order to increase savings available for a downpayment.

69 The outcome Nonhousing debt bankruptcy, foreclosure, or repossession indicator captures the share of the sample that experienced a bankruptcy, charge-off, foreclosure, or repossession on a non-mortgage debt account.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 70

6. IMPACTS: FINANCIAL CAPABILITY

mortgage debt account. These findings may indicate that homebuyer education and counseling are not causing treatment group members to actually miss payments. Instead, one possible explanation for this finding is that homebuyer education and counseling services lead treatment group members to try to increase the amount they save or amount of debt they pay down each month, which leads them to perceive that they are less able to pay all their bills at the end of the month.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 71

7. Impacts on Sustainable Homeownership

Key Findings: Impacts on Sustainable Homeownership

• Homebuyer education and counseling services had no detectable impact on mortgage performance measures. The 30-, 60-, and 90-day delinquency rates were quite low, reflecting the relatively short followup period under study.

• Homebuyer education and counseling services led to increased prevalence of “high” monthly housing costs, as measured by the share of study participants who spend more than 30 percent of their monthly income on housing. The increased prevalence of high housing costs is concentrated among study participants who were least likely to participate in services, which could indicate that homebuyer education and services played a limited role in the decision to take on high-cost housing.

• This sustainable homeownership domain of outcomes wil be more important at long-term followup, when study participants wil have had more time to experience the effects of their exposure to homebuyer education and counseling services. In addition, a larger share of the sample likely wil have experienced delinquency in the longer term, permitting the study to better detect treatment-control differences on these measures.

In the longer term, the central goal of homebuyer education and counseling is sustainable homeownership—that is, helping prospective homebuyers make good decisions about whether to purchase a home, and if they do purchase, helping prepare them to make timely mortgage payments, avoid foreclosure, and build wealth. 70 If homebuyer education and counseling services result in better preparedness for homeownership and improved financial literacy and behaviors, then the study hypothesizes recipients of these services to be more likely to meet their monthly payments and accrue home equity. Homebuyer education and counseling recipients will also be more likely to avoid mortgage delinquency.

At this short-term followup time point, at 12 to 18 months after study enrollment, we do not expect to detect impacts on many of these longer-term sustainable homeownership measures.

This chapter reports impacts on these measures, however, because we expect them to be important to the study’s future analyses and reports.

All outcomes analyzed in this chapter (as in chapters 5 and 6), including those related to monthly housing costs and mortgage loan delinquencies, are defined for the full study sample. 71

70 Chapter 5 reports on the preparedness and search domain and chapter 6 on the financial capability domain.

Chapter 3 and appendix A provide detail on how the impacts reported in this chapter (as well as in chapters 5 and 6) are estimated. See the textbox How to Read the Impact Exhibits in This Report on page 51. Expanded results for the overall impact are reported in appendix C, and expanded results for delivery mode effects are reported in appendix D. Appendix E presents subgroup impacts.

71 For example, monthly housing costs are set equal to monthly rent if the study participant is a renter, set equal to the monthly mortgage payment if the study participant owns the home he or she lives in, and set to zero if the study participant lives somewhere without paying rent. Exhibit B.5 provides more detail related to the operationalization of outcome measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 72

7. IMPACTS: SUSTAINABLE HOMEOWNERSHIP

• Homebuyer education and counseling services led to increased prevalence of high monthly housing costs relative to income.

Within the treatment group, 24.3 percent of participants reported that they spend more than 30

percent of their monthly income on housing, whereas 22.1 percent of the control group reported doing so, as shown in exhibit 7.1. We find descriptive evidence that this finding is not sensitive to housing tenure—that is, it was not concentrated either among those who were homeowners or among those who remained renters. 72 The impact on housing cost burdens is also not driven by variation in housing costs across the 28 metropolitan areas from which study participants were recruited. The impact model includes a set of site fixed effects, which control for both observable and unobservable differences across these metropolitan areas.

Exhibit 7.1: Overall Impact of the Demonstration’s Homebuyer Education and Counseling on Sustainable Homeownership

Impact of

Impact of

Treatment

Control Being Offered Taking Up

Outcome

Group Mean Group Mean Services

Services

Monthly-housing-costs-to-monthly-income ratioa

24.9

24.2

0.8

1.2

Monthly housing costs exceed 30 percent of monthly incomea (%)

24.3

22.1

2.1*

3.5*

Ever 30 days delinquentb (%)

2.4

2.0

0.3

0.6

Ever 60 days delinquentb (%)

0.7

0.5

0.2

0.3

Ever 90 days delinquentb (%)

0.3

0.2

0.1

0.2

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, 4,546

sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Due to rounding, reported impacts (T-C differences) may differ from differences between reported regression-adjusted means for the treatment and control groups. Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

Homebuyer education and counseling services focus on the affordability of housing for participants, so the higher housing cost burden for the treatment group is somewhat surprising.

However, as elaborated in section 8.3.2, a few complicating factors may affect the interpretation of this finding. For instance, it could be that the intervention is simply encouraging participants to consider their longer-term housing needs and the transaction costs related to a “trade-up”

purchase later on, which might lead some buyers to purchase a more expensive home that could meet their needs for a longer time period. This could be a rational decision with no adverse consequences for homeownership. Even so, although the impact is statistically significant, the average housing-cost-to-income ratios for both the treatment and the control groups are quite low 72 The impact is nearly identical for the full sample, for the non-experimental subsets of the sample who purchased homes, and the non-experimental subset of the sample who did not purchase homes.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 73

7. IMPACTS: SUSTAINABLE HOMEOWNERSHIP

compared to low-, moderate-, and middle-income renters and homeowners in general. Some 39.0

percent of homeowners and 58.8 percent of renters with household incomes at or below 120

percent of area median income have housing cost burdens that exceed 30 percent. 73

This impact is largest for in-person services: as shown in exhibit 7.2, those participants offered in-person services were 6.1 percentage points more likely to have housing costs in excess of 30

percent of income than were their control group counterparts (see exhibit D.1 for expanded results). In contrast, we find no detectable evidence that remote services affected the prevalence of high monthly housing costs relative to income. We plan to estimate the impact on this outcome at long-term followup to determine whether these findings persist.

Exhibit 7.2: Impact by Service Delivery Mode on Sustainable Homeownership

Difference

Impact of Being

Between

Of ered

Impact of Being In-Person and

In-Person Of ered Remote

Remote

Outcome

Services

Services

Impacts

Monthly housing costs to monthly income ratioa

1.4

0.4

1.0

Monthly housing costs exceed 30 percent of monthly incomea (%)

6.1***

0.7

5.4**

Ever 30 days delinquentb (%)

1.0

0.2

0.8

Ever 60 days delinquentb (%)

0.3

0.2

0.1

Ever 90 days delinquentb (%)

0.3

0.0

0.2

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of in-person services, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806); and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n= 1,184). For the analysis of remote services, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

• The increased prevalence of high housing costs is concentrated among study participants who were least likely to participate in services.

Appendix F reports the impact of the intervention on the subgroup of those most likely (and least likely) to participate in services. The findings presented in that appendix indicate that the overall effect on high housing cost prevalence is driven by those least likely to take up or complete services. Across four different measures of service receipt—took up any services, completed the education curriculum, completed one-on-one counseling, and completed all homebuyer 73 Author’s calculations based on Worst Case Housing Needs: 2017 Report to Congress (HUD, 2017).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 74

7. IMPACTS: SUSTAINABLE HOMEOWNERSHIP

education and counseling services—we find that those treatment group members who are least likely to complete services are more likely to have high monthly housing costs relative to their control group counterparts.

Reasons for these results are not obvious. One possibility is that homebuyer education and counseling services gave service noncompleters a false sense of confidence in their ability to gauge how much housing is affordable given their financial situation: that is, a little information can be dangerous. Relatedly, perhaps service availability encouraged treatment group members to trust professionals in the industry, even if they did not complete services; whereas their control group counterparts, who were not offered services, were more skeptical of industry professionals and were thereby motivated to conduct independent research to determine what they could afford.

• Homebuyer education and counseling services did not affect the monthly-housing-costs-to-monthly-income ratio or the prevalence of housing costs above 25 percent, 35 percent, 40

percent of income.

The impact of homebuyer education and counseling on housing costs is not robust to alternative measures of housing costs. Although we find evidence of increased incidence of treatment group participants paying more than 30 percent of their income in housing costs (relative to control group participants), we do not see a statistically significant impact on a continuous housing-cost-to-income ratio (see exhibit 7.1) or on other ratio thresholds such as 0.25, 0.35, or 0.40.

• Homebuyer education and counseling services had no detectable impact on short-term loan performance, as measured by the 30-, 60-, and 90-day delinquency rates.

Estimating the impact of homebuyer education and counseling services on delinquency rates can provide evidence on whether homebuyer education and counseling services lead to better loan performance. That said, study participants who purchased a home had been in their homes, on average, fewer than 12 months at followup (as described in chapter 4, the mean number of months that purchasers had been in their homes is 10.5 and the median is 11) and mortgage delinquencies are rarely observed within the first year of origination.

The rates of delinquency for participants in this study are all less than 1 percent: that is, less than 1 percent of the sample was ever 60 days delinquent on a mortgage payment at short-term followup. Given the rarity of delinquencies, analysis of the impact of homebuyer education and counseling on loan performance is more relevant to the study’s Long-Term Impact Report, which will report the impact of homebuyer education and counseling services on outcomes measured about 3½ to 5 years after study enrollment.

In this Short-Term Impact Report, these estimates should be interpreted relative to their minimum detectable effects (MDEs). As discussed in more detail in appendix section A.7, the MDE is the smallest true intervention impact that can be detected with a given level of confidence. For the outcome Ever 60 days delinquent, the MDE is 0.5 percentage point (see exhibit C.1). Given that the control group mean was 0.5 percent (see exhibit 7.1), an MDE of 0.5

percentage point means that homebuyer education and counseling would have had to reduce the The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 75

7. IMPACTS: SUSTAINABLE HOMEOWNERSHIP

share of the treatment group that was Ever 60 days delinquent to 0.0 percent (or increase the share of the treatment group Ever 60 days delinquent to at least 1.0 percent) in order to have a reasonable chance of detecting a statistically significant effect. An impact this large in magnitude seems unlikely, as it implies a relative impact of 100 percent. In order to detect an impact on an outcome that occurs so infrequently, either that impact needs to be relatively very large or a larger sample size is needed.

We will be better positioned to estimate an impact on Ever 60 days delinquent and other measures of loan performance in the Long-Term Impact Report because we expect a greater share of control group members to be delinquent in the longer term, which implies that a smaller relative impact will be needed to detect an effect. For example, if 3.0 percent of control group members are Ever 60 days delinquent at the time of the Long-Term Impact Report, an MDE of 0.5 implies that homebuyer education and counseling would need to reduce the share of the treatment group that was Ever 60 days delinquent to 2.5 percent for the effect to be statistically significant. This 0.5 percentage point impact represents a 16.7 percent relative impact, which is much more plausible than a relative impact of 100 percent.

Although the 60-day delinquency rate is this study’s confirmatory outcome, we should not expect to find impacts at this time. Instead, the Long-Term Impact Report will revisit impacts on loan performance, tracking the sample over time as sample members experience homeownership and changes in the economy and their lives.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 76

8. Conclusion and Implications

This report provides findings from HUD’s First-Time Homebuyer Education and Counseling Demonstration, a large-scale randomized experiment on the effectiveness of homebuyer education and counseling. The Demonstration’s primary research objective is to estimate the impacts of homebuyer education and counseling on low-, moderate-, and middle-income prospective first-time homebuyers. The study seeks to estimate the impacts of homeownership education and counseling on outcomes related to (1) preparedness and search, (2) financial capability, and (3) sustainable homeownership. The study also evaluates whether the impacts vary by the mode of service delivery or for various subgroups of the study sample.

Sustainable homeownership is determined by a host of factors. Financial regulation influences the extent to which potential borrowers have access to safe, affordable mortgage

products. Macroeconomic and monetary policies affect interest rates and labor market conditions, influencing prospective homebuyers’ ability both to qualify for mortgages and to sustain those mortgages over time. Housing policy influences the degree to which there are affordable homeownership options and the relative attractiveness of those options compared to rental housing. This Demonstration examines how homebuyer education and counseling may contribute to sustainable homeownership in the context of these other, contributing factors.

This Short-Term Impact Report (reporting findings at 12 to 18 months after study enrollment) provides well-needed evidence to the field on the varied impacts of offering homebuyer education and counseling to a large, diverse sample of potential first-time homebuyers who have low to middle income levels. The study will provide objective evidence on how homebuyer education and counseling services help first-time homebuyers develop knowledge and skills and improve their financial health and sustainable homeownership. Unlike prior research, this evaluation uses a rigorous experimental design, not only to assess impacts of homebuyer education and counseling in general but also to test whether the service delivery mode (in-person versus online) makes a difference.

This final chapter highlights the study’s key findings and their policy implications. It also describes the context for the study and offers insights on the extent to which one might use the study’s findings in other market contexts and for other populations. Finally, the chapter previews the study’s next stage—the long-term data collection and analysis at 3½ to 5 years after study enrollment to be summarized in a future Long-Term Impact Report.

8.1 Key Findings

Overall, this report reveals mixed evidence of the impact of homebuyer education and counseling on outcomes observed 12 to 18 months after study participants enrolled in the study. Despite the overall mixed nature of the findings, we find several impacts of homebuyer education and counseling on the treatment group that are encouraging.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 77

8. CONCLUSION

• Greater confidence in ability to find needed information. Members of the treatment group were more likely than the control group to be confident that they could find information they needed about the homebuying process.

• Higher rates of home purchase for young adults. The full treatment group and full control group had similar rates of home purchase, but for those age 29 or younger at the time they enrolled in the study, the treatment group had higher rates of homeownership than their control group counterparts.

• Greater satisfaction with the homebuying process. The treatment group was more likely to report being “very satisfied” with the homebuying process than was the control group.

• Greater likelihood of having their mortgage payments automatically deducted from their bank account. The treatment group was more likely than the control group to have arranged for their mortgage to be paid monthly through automatic deduction.

• Greater likelihood of seeking assistance in times of financial distress. The treatment group was more likely than the control group to say they would contact a counseling agency or another nonprofit for assistance prior to missing a mortgage payment.

On the other hand, we find some outcomes on which homebuyer education and counseling had no detectable impact or even unexpected impacts:

• No detectable impact on loan performance measures. The 30-, 60-, and 90-day delinquency rates were quite low, reflecting the relatively short followup period of 12 to 18

months. These outcomes will be more important at long-term followup, when study

participants will have had more time to experience the effects of their exposure to homebuyer education and counseling services.

• Higher levels of debt. The treatment group experienced higher levels of nonhousing debt—

primarily student loan debt—than their control group counterparts. In addition, the treatment group had a higher monthly debt-to-income ratio than the control group, although the treatment and control groups were equally likely to have debt-to-income ratios that exceeded 0.43, the upper limit specified by FHA guidelines.

• Greater prevalence of high monthly housing costs relative to income. Overall, treatment group members more often had housing costs that exceeded 30 percent of their household income than did the control group. However, the impact of homebuyer education and counseling on housing costs is not robust to alternative measures of housing costs. We do not see a statistically significant impact on a continuous housing-cost-to-income ratio or on other ratio thresholds such as 25, 35, or 40 percent of income.

• Less reported ability to cover all bills. Overall, the treatment group reported a higher incidence than did the control group of occasionally not having enough money to cover bills.

Because we detected no impacts on measures of actual payment behavior, one possible explanation for this finding is that homebuyer education and counseling services led The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 78

8. CONCLUSION

treatment group members to try to save or pay down more debt each month, creating a perception that they were less able to pay all their monthly bills.

Exhibit 8.1 presents a summary of the overall, in-person, and remote impacts of homebuyer education and counseling for outcomes by each of the three outcome domains. For each outcome, we indicate whether the sign of the impact is positive (as indicated by a “+”), negative (as indicated by a “–”), or if there is no detectable impact (as indicated by a blank cell). The positive or negative sign of the impact does not directly imply a “favorable” or “unfavorable”

impact. Whether a specific impact is favorable or unfavorable is a matter of interpretation. For example, a decrease (negative sign) in delinquency rates would be a favorable result.

Exhibit 8.1: Summary of Selected Impacts

Overall

In-Person

Remote

Outcome

Impact

Impact

Impact

Chapter 5: Impact on Preparedness and Search

Is confident that he/she could find the information he/she needed about the

+

+

homebuying process (%)

Number of lenders from which the study participant received price quotes

–

Study participant purchased a home (%)

Study participant was very satisfied with the homebuying process (%)

+

+

Chapter 6: Impact on Financial Capability

Financial Knowledge

Knows how to correct inaccurate credit report information (%)

+

+

If in financial difficulty, would contact counseling agency, consumer credit

+

+

counseling agency, or other nonprofit organization for assistance prior to

missing a mortgage payment (%)

If in financial difficulty, would contact lender for assistance prior to missing a

+

mortgage payment (%)

Number of correct answers (out of 4) to mortgage literacy quiz

+

If having financial problems and could not pay all of the bil s, would pay

mortgage first (%)

Financial Behaviors

Has a budget and often compares it against actual spending (%)

Usually pays credit card balance in full to avoid interest charges (%)

Regularly required mortgage payment is automatically deducted from a bank

+

+

account (%)

Financial Indicators

Credit score (out of 850)

Has a credit score above or equal to 620 (%)

+

Student loan balance ($)

+

+

Credit card balance ($)

Total nonhousing debt ($)

+

+

Total monthly debt-to-income ratio (back-end ratio)

+

+

Total monthly debt-to-income ratio (back-end ratio) greater than 0.43 (%)

Usually has enough savings set aside to cover 3 months of expenses (%)

Total savings and investments ($)

Occasionally does not have enough money to cover all of the bil s at the end of

+

the month (%)

Nonhousing debt bankruptcy, foreclosure, or repossession indicator (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 79

8. CONCLUSION

Overall

In-Person

Remote

Outcome

Impact

Impact

Impact

Chapter 7: Impact on Sustainable Homeownership

Monthly-housing-costs to monthly-income ratio

Monthly housing costs exceed 30 percent of monthly income (%)

+

+

Ever 30 days delinquent (%)

Ever 60 days delinquent (%)

Ever 90 days delinquent (%)

Notes: This exhibit reports the direction of statistically significant impacts; the magnitudes of the impacts can be found in the exhibits presented in chapters 5, 6, and 7. In addition, exhibits C.1, D.1, and D.2 present expanded results for overall impacts, in-person impacts, and remote impacts, respectively.

+ Positively signed impact. – Negatively signed impact. Blank cell no detectable impact.

8.2 Implications of Findings

What do these short-term findings tell us about how well homebuyer education and counseling achieve their primary goal of improving homeownership-related outcomes for low- to middle-income prospective first-time homebuyers? Homebuyer education and counseling could increase successful homeownership for members of this population in two ways that relate to various sets of outcomes measured by the study. The first way would be to increase homeownership access—

that is, the share of prospective homebuyers who are able to become homeowners. The second would be to increase the sustainability of homeownership among those who did become homeowners—by bolstering their loan performance or by encouraging them to delay their purchase until they were in a better position to succeed. We consider the implications of our key findings on each set of outcomes separately.

8.2.1 Impact on Access to Homeownership

What do our key findings tell us about the impact of homebuyer education and counseling on helping people access homeownership? In the short term, we do not find evidence of an overall impact of homebuyer education and counseling on individuals purchasing homes. However, we do find an impact for select subgroups: in particular, study participants age 29 or younger at the time of study enrollment experienced increased home purchase rates as a result of being offered homebuyer education and counseling services. This is important because homeownership rates among young people are at historic lows (Spader, McCue, and Herbert, 2016).

Two theories are commonly cited for what has been driving the decline in homeownership rates for young people. One theory points out that younger adults are simply choosing to delay homeownership. More mobile and career-focused than previous generations, they are delaying the age at which they marry and start families, the primary triggers for purchasing a home (Joint Center, 2015). The declining homeownership rate for this group reflects a preference to not purchase. Another theory posits that the decline in homeownership among younger adults results from their financial circumstances precluding it. In particular, a lack of affordable homes for purchase and an inability to meet credit and underwriting standards present barriers for those young adults who do wish to purchase homes (Spader and Herbert, 2017).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 80

8. CONCLUSION

Although we do not know to what degree the younger adults in our sample are representative of those experiencing barriers to purchasing homes, we do know that the preference of our specific study participants is to purchase a home—after all, they were recruited into the study after inquiring about a mortgage. As a result, our short-term finding that the offer of homebuyer education and counseling increases home purchase among study participants age 29 and younger is encouraging, as it suggests that homebuyer education and counseling helps this group overcome homeownership barriers.

8.2.2 Impact on Sustainable Homeownership

How might our findings affect the sustainability of homeownership for those people who do become homeowners? Ultimately, loan performance indicators (in particular, the 60-day delinquency rate) over the longer term will be our best measure of “sustainable” homeownership.

Meanwhile, however, some findings in the short term are promising; in particular, the favorable impact of homebuyer education and counseling on treatment group members’ having their mortgage payments automatically deducted from their bank account and understanding that they should proactively contact counseling agencies or other nonprofits in times of financial distress.

These factors could translate into fewer delinquencies and fewer defaults and foreclosures down the road.

We do not detect an overall difference in the home purchase rate between treatment and control group members. However, our finding of increased confidence in the ability to find needed information and higher satisfaction with the homebuying process as a result of the intervention could mean that those in the treatment group who become homeowners are better equipped for it and more likely to find homes and/or loans that meet their needs.

The higher debt level finding is driven by higher student debt for borrowers with lower credit scores, who might be taking on additional student debt or shifting resources from paying down student debt in order to build savings—for example, for a downpayment. If that is what they are doing, then these individuals might be making rational financial decisions to increase their likelihood of becoming homeowners. Still, the higher levels of nonhousing debt for the treatment group are not without concern. The higher level of debt burden is on average only $791, but additional debt could adversely affect loan performance or erode some of the benefits of asset accumulation associated with homeownership.

The role homebuyer education and counseling might be playing in the decisions consumers make around debt and savings is not clear. The future Long-Term Impact Report will offer an opportunity to explore whether study participants who purchased a home made changes to their student loans or repayment plans in preparation to buy a home or qualify for a mortgage, based on their responses to the Long-Term Follow-Up Survey. In addition, the Long-Term Impact Report could examine credit bureau data to consider whether homebuyer education and counseling had an impact on student loan delinquencies. It may also be worth exploring how the rise in student debt levels have affected home purchase decisions.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 81

8. CONCLUSION

Multiple possible explanations exist for the finding that the offer of homebuyer education and counseling increases the likelihood of treatment group members’ paying more than 30 percent of their income on housing. If the intervention is somehow encouraging prospective first-time homebuyers to spend more of their income on housing than they can afford (by the 30-percent metric), then this might have a negative effect on sustainable homeownership. However, homebuyer education and counseling could simply be encouraging participants to consider their longer-term housing needs and the transaction costs related to a “trade-up” purchase later on.

Some buyers may be purchasing a larger, more expensive home that would be able to meet their needs for a longer time period. This could be a rational decision with no adverse consequences for sustainable homeownership. Furthermore, the impact of homebuyer education and counseling on housing costs is not robust to alternative measures of housing costs: we found no impact on a continuous housing-cost-to-income ratio or on other ratio thresholds such as 25, 35, or 40 percent of income. This may indicate that the effect of spending more than 30 percent of income on housing may be due to chance and should not be of major concern.

8.3 Insights into Service Delivery Modes

In addition to evaluating the overall impacts of homebuyer education and counseling on outcomes for low-, moderate-, and middle-income prospective homebuyers, this Demonstration allowed us to evaluate whether the impacts vary by service delivery mode.

Although we observe a large number of outcomes where only one service mode had a

statistically significant impact, we did not find that one delivery mode consistently outperformed the other. Rather, it appears that each of the two service modes has its own strengths and limitations. For example, compared to their control group counterparts, treatment group members offered in-person services—

• experienced greater satisfaction with the homebuying process.

• achieved higher scores on a mortgage literacy quiz.

• were more likely to attain a credit score of 620 or higher.

However, these treatment group members were also more likely to have relatively high monthly housing costs.

Meanwhile, treatment group members offered remote services experienced a positive impact on—

• their confidence finding information about the homebuying process.

• their self-reported ability to correct credit report inaccuracies.

• whether they would reach out to counseling agencies (or other organizations) or their lenders for help if they were in financial distress.

However, these treatment group members also experienced higher levels of nonhousing debt (including higher student loan balances) than the control group.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 82

8. CONCLUSION

The Demonstration’s qualitative research in this study documented implementation and operations of services, including both modes, as detailed in this report’s chapter 3 and further detailed in the study’s Baseline Report (DeMarco et al., 2017). That said, the process study did not document all the possible reasons why one mode or the other would result in all of the outcomes planned for analysis. As a result, this chapter’s summary of findings identifies delivery mode effects with only suggestive evidence about why they arose.

Some of these mode-specific effects might be understood by considering how the content of homebuyer education and counseling is delivered through each service mode. For instance, prospective homebuyers who gain knowledge of the homebuying process through the remote education curriculum might, as a result, become more comfortable navigating information related to homeownership online. That comfort may, in turn, translate more generally into increased confidence about seeking out needed information.

On the other hand, the greater satisfaction of those offered in-person services could be the result of the personal touch associated with such services. For instance, the experience of receiving homebuyer education in a classroom with people in their community who are in a similar situation might alleviate stress associated with the homebuying process. As study participants who take part in in-person education workshops hear other prospective homebuyers’ questions, talk with them during class breaks, and connect with professionals presenting the workshop, they could feel increased connection to a community facing the same challenges, and thereby gain a greater sense that someone is “in their corner.”

In addition, participants offered in-person services were statistically more likely to achieve a credit score of 620 or higher—an important underwriting threshold—than were their counterparts in the control group. This finding is particularly interesting because it was the remote treatment group—not the in-person treatment group—that experienced higher levels of confidence in their ability to correct credit inaccuracies. Perhaps the differences are, again, the result of the nature of the delivery mode. If, as we have hypothesized, the remote mode increases knowledge and skill at finding information, increased confidence would be a natural byproduct.

However, it could be that individuals are more compelled to actually take action to fix their credit scores by the delivery of in-person counseling, because they are face-to-face with a counselor.

Other differences between the modes are more difficult to explain. For example, why the offer of in-person services results in a population more likely to pay more than 30 percent of their income for housing, whereas the offer of remote services does not, is unclear. Likewise, it is not clear why the offer of remote services leads to higher levels of student debt for treatment group members relative to control group members, but the offer of in-person services does not. These outcomes will be tracked in the study’s longer-term followup. Further research might be needed to understand these findings, especially if they persist over time.

This Demonstration lays the groundwork for other researchers, practitioners, and policymakers to learn more about the relative effectiveness of each mode and, most importantly, if learning The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 83

8. CONCLUSION

opportunities for how to improve the efficacy of both modes exist. For example, as described, it could be that in-person services have the advantage of fostering a sense of support and community, whereas remote services help participants become comfortable conducting online research on their own. If these attributes are contributing to the respective successes in each mode, there might be ways to inject such attributes into other modes. For example, remote services could incorporate “online communities” that help connect members to one another into their online learning platform. In-person services could incorporate self-directed, online modules as part of their classroom curricula.

We conclude this discussion of differential mode effects with the observation that the sample size of each group (in-person, remote) contributes to our ability to detect effects: fewer observations are available for analysis from either mode, with the in-person mode being a smaller group still. Its smaller sample size creates a relatively “higher bar” to finding significant impacts for the in-person mode. We therefore encourage policymakers and researchers to keep this issue on our collective research agenda. Doing so will permit further exploration of the specific reasons for differential mode-specific impacts and to identify ways that programs across modes can adapt and innovate to improve outcomes across the field.

8.4 Generalizability of Findings

Given the experimental design of this study, we are confident that the impacts presented in this report have strong internal validity—that is, they are not biased by variation in the characteristics of those study participants offered homebuyer education and counseling from those assigned to the control group. However, the programmatic and policy implications of the findings also are influenced by the degree to which they are generalizable beyond the population and setting of this study—that is, their external validity. The study’s Baseline Report (DeMarco et al., 2017) considered the extent to which our findings would hold (1) during housing and credit market conditions different from those prevailing at the time of the study; and (2) for low-, moderate-, and middle-income first-time prospective homebuyers in general (not only for the study’s specific sample).

8.4.1 Housing and Credit Environment

DeMarco et al. (2017) provided an overview of market conditions during the study’s sample enrollment period of September 2013 to February 2016. The market conditions for the study were greatly influenced by the preceding, roughly 5-year housing market downturn of 2007–

2012. During that period, housing prices fell by an average of 40 percent across the nation, accompanied by an equally precipitous decline in home sales. Meanwhile, during 2007–2009, the unemployment rate had more than doubled to more than 10 percent, triggering rates of mortgage default and foreclosure not seen since the advent of the modern mortgage market. In contrast, the study’s enrollment period was a period of robust housing and labor market growth.

From 2013 to 2016, housing prices rose an average of 6 percent per year, and unemployment fell from 7.2 percent in September 2013 to 4.9 percent in February 2016 (FHFA, 2018; DOL, n.d.).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 84

8. CONCLUSION

Highlighting the longer period of time is important because this study’s 2013–2016 period of pilot and enrollment occurred in the shadow of the market downturn, when home purchase and mortgage decisions might reflect prospective homebuyers’ altered attitudes toward homeownership or the market and regulatory responses to the downturn. For instance, during the study enrollment period, mortgages were relatively inexpensive but also difficult to obtain by historical comparison. Such conditions could be an incentive for some prospective homebuyers to increase their financial capability in order to qualify for a mortgage and purchase a home.

However, some prospective homebuyers could become discouraged by the tight credit market and be less interested in homebuyer education and counseling.

Some of the key outcomes considered in this study—such as home purchase decisions and delinquency rates—are sensitive to prevailing economic, housing, and credit market conditions.

The strong labor market and strict underwriting requirements that characterized the time period of our study resulted in less frequent loan delinquencies than we might expect during periods of higher unemployment or looser credit. The impact of the homebuyer education and counseling offered by this study could be different under alternative environments. For example, if the impact of homebuyer education and counseling is largest for marginal borrowers who, during the study period, were being shut out of the market by strict underwriting criteria, we might not be capturing that benefit. In addition, if homebuyer education and counseling are particularly effective at helping marginal borrowers avoid or cure delinquencies, we might not capture this effect, both because these marginal borrowers did not purchase homes and because a major trigger of delinquency—unemployment—was so low.

Regardless of when effects are measured, the impacts of homebuyer education and counseling will inevitably be mediated by current market conditions, regardless of when the effects are measured. Readers should keep this in mind when applying the study’s findings to periods with markedly different market settings. Still, the study’s findings—because they are based on a rigorous, experimental evaluation design—provide valuable evidence for the field on the impacts of homebuyer education and counseling.

8.4.2 Study Sample

The study’s sample recruitment and enrollment design did not allow for strictly representative sampling from a well-defined population of low- to middle-income prospective first-time homebuyers across the nation. Instead, the sample was recruited from the customers of three major lenders and in the 28 large metropolitan areas within which recruitment took place.

Two additional factors could have resulted in some idiosyncrasy of the study’s sample. First, the sample comprises people who entered the study via their contact with the partner lenders and who agreed to participate in the study voluntarily. Second, a number of eligibility screens were necessary to facilitate successful data collection (see section 2.1). For example, we screened out prospective study participants for whom homeownership education and counseling were mandated under the requirements of a mortgage or downpayment assistance program. As a result, this study’s participants could differ from clients typically seen by housing counseling The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 85

8. CONCLUSION

agencies. For example, the Baseline Report (DeMarco et al., 2017, exhibit 5.24) found that this study’s sample is more educated, has a higher income, and contains a comparatively larger share of men relative to the typical clients of housing counseling agencies. Moreover—and unlike agencies’ typical clients—this study’s sample had relatively little knowledge of homebuyer education and counseling services before entering the study. Instead, they can be characterized as a relatively general population, whose access to these services comes through their contact with lenders.

This study’s findings are most relevant to populations that are similar to the study sample at baseline: low-, moderate-, and middle-income households (that is, households with incomes at or below 120 percent of their area median income) that have contacted a mortgage lender about acquiring a mortgage for a first-time home purchase. Differences between the demographic makeup of the sample and other groups, such as all renters, established homeowners, or homeowners in default, mean that this study’s eventual findings on the effectiveness of homebuyer education and counseling might not fully carry over to the other groups. Still, the study sample includes a large number of participants, across 28 large metropolitan areas, who vary in their sociodemographic composition and have characteristics that reflect a sizable share of the population of low-, moderate-, and middle-income prospective first-time homebuyers.

In sum, the study findings provide evidence on the effectiveness of homebuyer education and counseling for a robust sample of low- to moderate-income prospective first-time homebuyers.

8.5 Next Steps

This Short-Term Impact Report is the first step in providing experimental evidence of the impacts of homebuyer education and counseling. It was our hypothesis that homebuyer education and counseling would affect outcomes related to preparedness and search and to financial capability in the short term (the time period of the impacts presented in this report) and would affect sustainable homeownership in the longer term. Although some of the short-term findings appear favorable, others raise concerns. It will be important to follow these outcomes over a longer period of time to see how they evolve. Because outcomes related to sustainable homeownership are more important over the longer term—indeed, adverse mortgage events are quite rare within the first year—we defer here to the study’s Long-Term Impact Report, currently scheduled for 2020, to draw conclusions on loan performance.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 86

APPENDIX

Appendix A: Analytic Methods

This appendix provides additional detail related to the analytic methods used to produce the impact estimates presented throughout this report. Section A.1 presents the model used to estimate the impact of being offered homebuyer education and counseling services. Section A.2

presents the models used to estimate the impact of taking up homebuyer education and counseling services. Section A.3 provides additional detail related to the samples used to answer each evaluation question. Section A.4 describes the methods used to address missing data.

Section A.5 presents baseline balance tests for the subset of study participants who responded to the Short-Term Follow-Up Survey. Section A.6 describes the method used to produce Followup survey nonresponse weights. Section A.7 describes the study’s ability to detect impacts and how minimum detectable effects can help with the interpretation of null findings.

A.1 Estimating the Impact of Being Offered Services

The unbiased estimated impact of being in a treatment group (that is, of being offered homebuyer education and counseling services) is the difference between treatment group and control group mean outcomes. That is, if we find that study participants in the treatment group, for example, have a higher average homeownership rate than do those in the control group, then the difference in these two homeownership rates represents the causal impact of the intervention. To operationalize this concept, the study follows common practice and uses multiple regression, which uses baseline variables as covariates to increase the precision with which the intervention’s impact is estimated.

In addition to controlling for a set of baseline characteristics, the impact model includes a single binary variable for whether the study participant was randomly assigned before or after the September 2014 study redesign. This variable serves to control for any possible differences across the time periods that could influence the outcome. The impact model also includes a set of site fixed effects, which control for both observable and unobservable differences across the 28

metropolitan areas from which study participants were recruited.

The regression model for estimating the impact of being offered homebuyer education and counseling services is:

𝑌𝑌𝑖𝑖𝑖𝑖 = 𝛼𝛼 + 𝛿𝛿𝑇𝑇𝑖𝑖𝑖𝑖 + 𝛽𝛽𝑋𝑋𝑖𝑖𝑖𝑖 + 𝜃𝜃𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑖𝑖𝑖𝑖 + 𝜇𝜇𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖

(eq. A.1)

where:

Yis is the outcome of interest for study participant i in site s; Tis is a dummy variable that equals 1 if study participant i in site s was assigned to a treatment group, and equals 0 if the study participant was assigned to the control group; The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 87

APPENDIX

Xis is a vector of individual background characteristics for study participant i in site s; 74

Timeis is a dummy variable that equals 1 if study participant i in site s was randomly assigned prior to September 16, 2014, and equals 0 if the study participant was randomly assigned on or after this date;

μs is a set of site fixed effects; 75 and

εis is a random error term.

The coefficient, δ, provides an “intent-to-treat” (ITT) estimate of the impact of being offered free homebuyer education and counseling services and is the parameter of central interest. This coefficient provides a regression-adjusted estimate of the difference in mean outcomes between treatment group members, including both those who took up intervention services and those who did not, and control group members. We estimated the equation (A.1) model using weighted least squares regression for both continuous and binary outcomes, so that the coefficient, δ, has the same interpretation for different types of outcomes. To account for the possibility that study participants from the same site (that is, MSA) may have correlated error terms, following standard practice, we cluster standard errors at the site level. 76

Equation (A.1) depicts the model specification used to estimate the overall ITT impact of homebuyer education and counseling services and mode-specific effects. To produce subgroup impacts, we add an “interaction term” to the model: the treatment indicator is interacted with a subgroup identifier (as defined by baseline traits), and the coefficient on this interaction term provides an estimate of the difference in impacts between subgroups. Exhibit B.4 describes the operationalization of the specific subgroup identifiers, and the impacts on 13 of subgroups—

selected in the study’s design phase—as are presented in appendix E.

A.2 Estimating the Impact of Taking Up Services

The ITT estimate provides the impact of being assigned to a treatment group regardless of whether services are actually received, but we are also interested in estimating the impact of actually taking up the services that were offered, which is referred to as the “treatment-on-the-treated” (or TOT) impact.

The textbox Calculating Impact Two Ways beginning on page 19 offers a general introduction to the concept of the TOT impact. For this study, we estimate the impact of taking up services by 74 Exhibit B.3 describes the baseline covariates included in the impact analysis model.

75 Here “site” refers to the 28 large metropolitan areas where study participants enrolled.

76 As described by Cameron and Miller (2015), failure to control for within-site error correlation can lead to misleading standard errors, confidence intervals, and p-values.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 88

APPENDIX

two-stage least squares (for example, Angrist and Imbens, 1995; Angrist, Imbens, and Rubin, 1996), which is functionally the same as using the division-based Bloom (1984) approach. 77 In the first stage, we estimate a linear regression model that predicts the probability that a given study participant takes up homebuyer education and counseling services. As depicted by equation (A.2), the dependent variable in the model, P, is an indicator for whether the study participant took up any homebuyer education and counseling services. The model includes the same set of regressors included in the equation (A.1) model used to estimate the impact of being offered services: a treatment group indicator, a vector of baseline characteristics, a binary time variable, and a set of site fixed effects.

𝑃𝑃𝑖𝑖𝑖𝑖 = 𝜋𝜋1 + 𝜋𝜋2𝑇𝑇𝑖𝑖𝑖𝑖 + 𝜋𝜋3𝑋𝑋𝑖𝑖𝑖𝑖 + 𝜋𝜋4𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑖𝑖𝑖𝑖 + 𝜇𝜇𝑖𝑖 + 𝜀𝜀1𝑖𝑖𝑖𝑖

(eq. A.2)

In the second stage, we model the outcome of interest as a function of the predicted probability of taking up services from the first stage regression and a similar set of observed covariates. The second stage model embeds the predicted take-up indicator within it, as follows:

𝑌𝑌𝑖𝑖𝑖𝑖 = 𝛾𝛾1 + 𝛾𝛾2𝑃𝑃𝑖𝑖𝑖𝑖 + 𝛾𝛾3𝑋𝑋𝑖𝑖𝑖𝑖 + 𝛾𝛾4𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑇𝑖𝑖𝑖𝑖 + 𝜇𝜇𝑖𝑖 + 𝜀𝜀2𝑖𝑖𝑖𝑖

(eq. A.3)

In equation (A.3), Pis is the predicted probability that study participant i in site s takes up any of the offered homebuyer education and counseling services, as estimated from equation (A.2), and the other terms remain the same, as defined in section A.1. In this model, the coefficient 𝛾𝛾2 is the estimate of the impact of taking up homebuyer education and counseling services and is the parameter of central interest.78

77 The conventional Bloom adjustment, which computes the impact of taking up services by dividing the ITT

estimate (and corresponding standard error) by the take-up rate, assumes that the take-up rate has no sampling variability (that is, that the take-up rate would be constant across different samples from the universe of potential study participants). This assumption could lead to a biased TOT variance estimate. In contrast, the two-stage least squares model used to compute the TOT estimate accounts for the sampling variability of the take-up rate when computing the TOT variance estimate, allowing us to produce an asymptotically unbiased estimate of the TOT

variance (Schochet and Chiang, 2009; Litwok and Peck, 2018). That said, the magnitude of the TOT impact estimate is the same whether one uses the Bloom (simple division) approach or the regression-based (instrumental variables) approach. Because the Bloom approach is more intuitive and easier to describe, we use it as a means to explain the analysis, and then execute the analysis with the regression-based approach.

78 Following the work of Angrist, Imbens, and Rubin (1996), Schochet and Chiang (2009) provide a detailed description of the assumptions required to identify the Complier Average Causal Effect (CACE), which is defined as the average impact of intervention services on those who comply with their treatment assignments. Following their terminology, compliers are those who would take up services only if they were assigned to the treatment group; never-takers are those who would never take up services; always-takers are those who would always take up services; and defiers are those who would take up services only if assigned to the control group. Under standard assumptions, we can identify the average causal effect of the treatment for compliers if there are no defiers and the intervention has no impact on never-takers and always-takers (Schochet and Chiang, 2009).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 89

APPENDIX

Capturing Service Take-Up: Administrative Data Versus Short-Term Follow-Up Survey Data

The study collected two sources of data about whether participants took up services—

administrative data from service providers and survey data from the Short-Term Follow-Up Survey. For the estimates of the impact of taking up services presented in chapters 5, 6, and 7, the indicator for whether the study participant took up services, P, is defined using administrative data from eHome America, ClearPoint, and local housing counseling agencies. 79 This subsection explains why we chose to use administrative data rather than survey data to measure take-up when estimating the impact of taking up services.

This administrative measure of take-up is available only for the treatment group. Because the administrative data are not available for the control group, the analysis assumes no control group crossovers. Crossovers are those members within the control group who receive any of the demonstration’s homebuyer education and counseling services offered to the treatment group. It is possible that some members of the control group found a way to participate in some form of homebuyer education and counseling from some outside source. Control group members were not referred to services through the study, but they were not prohibited from participating in services on their own, and they certainly could have accessed any other services available in their communities or online. However, it is conventional in applied program evaluation to adjust only for pure crossovers in computing the effect of taking up services, under the assumption that whatever outside services the control receives represents the counterfactual conditions, even if those conditions include participation in some comparable services.

In this study, we expect crossovers—control group members who received the same homebuyer education and counseling services offered to treatment group members—are either unlikely or relatively small in number for a few reasons. First, individuals were ineligible to participate in the study at all, whether as treatment or control group members, if they previously had used homebuyer education or counseling services or if they applied for a mortgage or downpayment assistance program that required them to complete a homebuyer education course. Second, many focus group participants said they either did not even know that homebuyer education and counseling services existed and, unlike the study’s treatment group members, control group members were not referred to the study’s HUD-approved homebuyer education and counseling services and did not receive any study incentive payments for participating in homebuyer education and counseling services. This implies little knowledge or incentive that would compel control group members to seek out services on their own. Finally, in the situation that control group members did find the same homebuyer education services that the study offered to treatment group members, then the control group members would typically be required to pay for 79 Administrative data from eHome America, ClearPoint, and local housing counseling agencies were also used to compute the estimates of homebuyer education and counseling services initiation and completion described in chapter 2.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 90

APPENDIX

those services, where this fee might be an additional deterrent to their participation. It seems more likely that control group members who sought services would end up participating in freely available services, such as a through a buyer’s agent, friend or relative, or local community center or library.

However, according to the Short-Term Follow-Up Survey, we know that some control group members reported accessing some kind of homebuyer education and counseling services from some source. Further, it is possible that some control group members pursued a mortgage that required participation in homebuyer education and counseling services through a nonstudy lender. Among the 18.0 percent of control group members who reported receiving some form of homebuyer education and counseling services, 40.3 percent reported that completion of homebuyer education and counseling services were required by their lender. This implies that 7.3

percent of control group lenders took up services to meet a lender requirement, where these services are likely to be similar to those services offered to treatment group members.

Given the possibility of some control group crossover, we created and tested an alternative measure of take-up using data from the Short-Term Follow-Up Survey. The survey includes self-reported service receipt for both treatment and control group members. As shown in exhibit A.1, 18.0 percent of control group members, 41.7 percent of treatment group members offered in-person services, and 52.0 percent of treatment group members offered remote services reported receiving some form of homebuyer education and counseling services after enrolling in the study.

Differences between the survey-based, self-reported take-up rates and the agency-reported take-up rates have important implications for the alternative TOT impacts that we estimate using these alternative data sources.

• Administrative Measure of Take-Up. The study’s participating counseling agencies reported who from the treatment group took up services at their agencies, and we believe that these data accurately represent the participation experience of treatment group members. 80 We have no agency-reported service take-up measure for control group members, so the administrative data source requires that we assume that no control group members received the specific services offered to treatment group members. If it is the case that control group members did not access this study’s services, then the administrative-data-based measure of

take-up accurately represents the share of control group members who took up services (at zero). If some control group members received services, then the administrative-data-based

measure of take-up would underestimate the share of control group members who took up

80 This measure equals 1 if the study participant is a treatment group member who took up any (meaning the study participant initiated, participated at some level, or completed) services; it equals 0 if the study participant is a treatment group member who did not take up any services; and it equals 0 if the study participant is a control group member.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 91

APPENDIX

services. If no control group members received the same services offered to treatment group members, then the administrative data measure of take-up is accurate, and the associated TOT estimate is as well. If some control group members were indeed crossovers into this study’s services, then the administrative measure underestimates control group take-up; and the associated TOT estimate would understate the actual TOT impact (as discussed in more detail later).

• Survey-Based Measure of Take-Up. Although 18 percent of the control group self-reported that they accessed some sort of homebuyer education and counseling services, we expect that, given their different experiences in the study, the treatment and control group members hold different definitions of “homebuyer education and counseling services” and what it means to participate in them and that those definitions influenced their survey responses. 81

Treatment group members are likely to have provided answers to the survey questions that reflected whether they took up the services offered through the study.

In contrast, control group members might have had a broader view of what it meant to participate in services related to homebuying. Some control group members who responded to the survey reported they received homebuyer education through “someone at work” or

“through a bank,” or they reported they completed counseling “by mail,” “online,” or “at a bank.” 82 These responses imply a less formal definition of service receipt, calling into question the comparability of the treatment and control responses on the Short-Term FollowUp Survey on this topic. This implies that the survey-based measure of take-up likely 81 This survey-based take-up measure equals 1 for study participants who indicated on the Short-Term Follow-Up Survey that they initiated or completed (took up) any services, and it equals 0 for study participants who indicated that they did not initiate or complete services. More specifically, this measure is set equal to 1 if the study participant replied “yes” to either of the following two questions from the Short-Term Follow-Up Survey:

• G1. Since enrolling in the study, have you completed any one-on-one homebuyer counseling? Homebuyer counseling usually involves a one-on-one session with a certified housing counselor to discuss your specific circumstances either in-person at a local agency or over the telephone.

• G2. Now I’d like to ask you about any homebuyer education programs you may have participated in. This includes educational instruction provided in a group workshop or through an online course and can take from one to ten hours. Since enrolling in the study about a year ago have you participated in any homebuyer education?

82 If study participants indicated on the Short-Term Follow-Up Survey that they took up homebuyer counseling or homebuyer education, then they were then asked about the mode of that respective service through the following survey questions:

• G1a. Did you complete the homebuyer counseling over the telephone, in-person at an agency, or through some other means?

• G2a. Did you complete the homebuyer education online, in-person at a housing agency, or through some other means?

If the study participant responded “through some other means” to either of these questions, then they were asked to specify where they received services. The responses listed here (“by mail,” “online,” “at a bank,” and so on) are a selection of the responses to these open-ended questions.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 92

APPENDIX

overestimates the share of control group members who received homebuyer education and counseling services similar to those offered to the treatment group. As such, the TOT

estimate based on these data (and assumptions) overestimates the actual TOT impact (as discussed in more detail later).

Exhibit A.1: Take-Up of Services Based on Short-Term Follow-Up Survey Responses Treatment Group

Treatment Group

Of ered In-Person

Of ered Remote

Control Groupa

Servicesb

Servicesc

Completed any one-on-one counseling (%)

10.7

28.5

32.5

Participated in any homebuyer education (%)

12.3

33.6

39.1

Participated in any homebuyer education and

counseling services (%)

18.0

41.7

52.0

a Sample: 2,007 control group members who responded to the Short-Term Fol ow-Up Survey.

b Sample: 642 treatment group members who were offered in-person services and responded to the Short-Term Fol ow-Up Survey.

c Sample: 1,897 treatment group members who were offered remote services and responded to the Short-Term Fol ow-Up Survey.

Notes: Excludes study participants who withdrew from the study and those missing measure-specific data. We also expect that the treatment and control groups likely have different definitions in mind when reporting that they accessed services.

Source: Short-Term Fol ow-Up Survey

Implications of Choice of Data Source for Interpretation of the Impact of Taking Up Services

As initially established by Bloom (1984) and also described by Alasuutari, Bickman, and Brennan (2008), conceptually, the impact of taking up services is equal to the difference in mean outcomes for the treatment and control groups (in this application, this is the ITT estimate produced by equation (A.1)) divided by the difference in treatment and control group take-up rates:

Impact of Taking Up Services ≈

ITT / ( Treatment Group Take-Up Rate – Control Group Take-Up Rate) (eq. A.4)

As described earlier, the administrative measure of take-up by the control group is either accurate (if take-up really is 0 percent) or might underestimate the control group take-up rate. If we underestimate the control group take-up rate in equation (A.4), then we overestimate the treatment-control difference in take-up rates (the denominator in equation (A.4)), and underestimate the impact of taking up services. Therefore, using the administrative measure of take-up (in equations (A.2) and (A.3)) will either accurately represent or underestimate the impact of taking up services.

Conversely, the survey-based measure of the control group take-up rate (18.1 percent) is likely an overestimate. If we overestimate the control group take-up rate in equation (A.4), then we underestimate the treatment-control difference in take-up rates (the denominator in equation (A.4)), and overestimate the impact of taking up services. Therefore, using the survey-based measure of take-up (in equations (A.2) and (A.3)) will likely overestimate the impact of taking up services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 93

APPENDIX

Therefore, we view the impact of taking up services that is computed with the administrative measure of take-up as either an accurate representation or a “lower bound” on the true impact of taking up services. Conversely, we view the impact of taking up services that is computed using this survey-based measure of take-up as an “upper bound” on the true impact of taking up services.

Appendices C and D present these two separate estimates of taking up services: (1) a lower-bound impact of taking up services that uses data from eHome America, ClearPoint, and local housing counseling agencies to capture service receipt, 83 and (2) an upper-bound impact of taking up service that uses self-report data from the Short-Term Follow-Up Survey to capture service receipt. 84 In the main text, we focus on the impact of taking up services based on the administrative measure of take-up because we believe it provides a more reliable measure of whether treatment group members took up services and—whether accurate or an

underestimate—is the more conservative estimate.

A.3 Samples Used to Answer Evaluation Questions

This subsection provides additional detail on the samples used to estimate the impact of homebuyer education and counseling services presented throughout this report. We assigned study participants to mutually exclusive subsamples based on whether they enrolled in the study before or after the September 2014 study redesign (the study redesign is described in section 2.2); their stated preference for in-person or remote services; and their randomly assigned treatment group.

Appendix exhibit A.2 defines each of these subsamples based on these characteristics and provides a sample identifier that we use to refer to subsamples throughout this section. (Exhibit 2.3 in chapter 2 provides a visual depiction of these subsamples.) Exhibit A.3 provides additional detail related to the samples used to produce experimental estimates of the overall impact, subgroup impacts, and mode effects.

The samples used for the impact analysis and the covariates included in the impact model are chosen to ensure that all of the impact estimates described in exhibit A.3 are based on experimental comparisons. To ensure that each of these comparisons maintains the integrity of 83 We refer to the TOT estimate computed by using the administrative data as a “lower-bound impact” although, as the preceding discussion highlights, it may also be an accurate representation of the true TOT impact, if indeed there are no crossovers.

84 We take steps to ensure that we use the same sample to produce the upper-bound and lower-bound estimates of taking up services presented in appendix C and appendix D: After estimating equation (A.2), we predict Pis—the predicted probability of taking up services—for al study participants (even those with missing data on the survey-based measure of take-up). Then estimating equation (A.3) al ows us to produce an upper-bound estimate of taking up services for all study participants with nonmissing outcome data (which is the same sample used to estimate the lower-bound impact of taking up services based on the administrative measure of take-up).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 94

APPENDIX

the experimental design, the impact model includes controls for whether study participants enrolled in the study before or after the September 2014 study redesign, their stated preference for in-person or remote services, and their randomly assigned treatment group.

For example, when estimating the impact of remote services, we compare outcomes for remote treatment group and choice treatment group members who selected remote services with outcomes for the control group. For this comparison, these controls ensure that our estimated impact of remote services is computed as a weighted average of the following experimental comparisons: (1) remote treatment group members are compared to the full set of control groups members and (2) choice treatment group members with stated baseline preference for remote services are compared to their control group counterparts, which are Modified Study Design control group members with stated baseline preference for remote services.

Exhibit A.2: Defining Samples Used to Answer Evaluation Questions

Random

Sample

Period of the

Assignment

Mode of

Identifiers

Study

Treatment Group Mode Preference Services Offered

Sample Size

C

Before redesign

Control

Not observed

None

789

TI

Before redesign

In-Person

Not observed

In-Person

515

TR

Before redesign

Remote

Not observed

Remote

548

CPI

After redesign

Control

In-Person

None

395

CPR

After redesign

Control

Remote

None

1,249

CPM

After redesign

Control

Missing Data

None

15

TCPI

After redesign

Choice

In-Person

In-Person

291

TCPR

After redesign

Choice

Remote

Remote

851

TRPI

After redesign

Remote

In-Person

Remote

295

TRPR

After redesign

Remote

Remote

Remote

814

TRPM

After redesign

Remote

Missing Data

Remote

8

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 95

APPENDIX

Exhibit A.3: Detail on Samples Used to Answer Evaluation Questions

Evaluation

Treatment

Question

Control Group

Treatment Group

Control Group Group Sample

Total

Addressed

Contrast

Samples Included

Samples Included

Sample Size

Size

Sample Size

Overall Impact

Compares full treatment group versus full control group C; CPI; CPR; CPM

TI; TR; TCPI; TCPR;

2,448

3,322

5,770

TRPI; TRPR; TRPM

Impact of in-person Weighted combination of the following two contrasts: C; CPI TI; TCPI

1,184

806

1,990

services

• Comparison of in-person treatment group versus

Initial Study Design control group

• Comparison of choice treatment group members

with stated baseline preference for in-person

services versus Modified Study Design control

group members with stated baseline preference

for in-person services

Impact of remote

Weighted combination of the following two contrasts: C; CPI; CPR; CPM

TR; TCPR; TRPI; TRPR;

2,448

2,516

4,964

services

• Comparison of remote treatment group versus full

TRPM

control group

• Comparison of choice treatment group members

with stated baseline preference for remote

services versus Modified Study Design control

group members with stated baseline preference

for remote services

Impact of choice of Compares choice treatment group versus Modified

CPI; CPR; CPM

TCPI; TCPR

1,659

1,142

2,801

service modes

Study Design control group

Subgroup impacts Compares full treatment group versus full control group C; CPI; CPR; CPM

TI; TR; TCPI; TCPR;

2,448

3,322

5,770

within each subgroup of interesta

TRPI; TRPR; TRPM

a Each set of subgroup impacts is estimated in a separate “interaction model” using the full study sample. See exhibit B.4 for operationalization of subgroup identifiers.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 96

APPENDIX

A.4 Methods for Handling Missing Baseline and Outcome Data

Baseline covariates are included in the model used to produce impact estimates to control for any observed, chance differences in these baseline measures. To avoid dropping observations from the impact analysis due to missing baseline data, we use the “dummy variable adjustment” approach.

This strategy sets missing cases to a constant and includes a set of “missing data flags” in the impact model. As detailed by Puma et al. (2009), this method is appropriate for handling missing baseline data from experimental y designed evaluations, and it is straightforward to implement and is easily replicated. The method involves the fol owing three steps for each baseline covariate, X, with missing data:

• Step 1: Create a new variable, Z, that is set equal to X for al cases where X is nonmissing, and is set to a constant value, C=the mean of X, for those cases when X is missing.

• Step 2: Create a new variable, D, that is set equal to 1 for cases where X is missing, and is set equal to 0 for cases when X is not missing.

• Step 3: Replace the baseline covariate, X, in the impact analysis model with Z and D. This will al ow the impact model to estimate the relationship between Y and X when X is not missing, and to estimate the relationship between Y and D when X is missing.

When outcome data are missing for a given sample member, we fol ow the recommendation of Puma et al. (2009) to conduct a “full-case” analysis. When estimating the impact on a given outcome, this method excludes (or “case-deletes”) any observations with missing data for that outcome rather than imputing the value of the missing observation. This method has the benefit of ease of implementation and interpretation. Similarly, we conduct a ful -case subgroup analysis, excluding observations with missing subgroup identifiers.

A.5 Balance Testing on Baseline Characteristics of the Analytic Sample

We know that the study sample was successfully randomized and that no systematic differences in treatment-control characteristics exist. 85 This is the case for the full sample, but it is important to consider this question for the sample of treatment and control group members who responded to the Short-Term Follow-Up Survey, because this is the analytic sample used to estimate impacts on outcomes constructed from the survey data. Treatment and control group members responded to the survey at different rates: 82.0 percent of control group members and 76.4

percent of treatment group members. The concern is that this difference in response rates between the groups could imply that the sample of control group responders is somehow different from the sample of treatment group responders. As a result, we test whether systematic 85 DeMarco et. al (2017), section 5.2, presents baseline balance testing for the full study sample.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 97

APPENDIX

differences in baseline characteristics exist between treatment and control groups after taking into account the attrition of the sample that resulted from survey nonresponse.

We compared the baseline characteristics of the three groups to which study participants were assigned and that comprise the study’s analytic sample: the control group, the remote treatment group, and the combined in-person treatment and choice treatment groups. The in-person and choice treatment groups could be pooled for this analysis because these two groups were part of random assignment during different, mutual y exclusive phases of the study. (The in-person treatment group was part of the early, Initial Study Design period, whereas the choice treatment group was part of the post-redesign Modified Study Design period.) Therefore, the pooled sample of in-person and choice treatment group participants is expected to be balanced with both the full control group and the ful remote treatment group.

Exhibits A.4 and A.5 present baseline characteristics for those study participants who responded to the Short-Term Follow-Up Survey within the remote treatment group, the in-person and choice treatment groups, and the control group. These exhibits also report results of a statistical test of whether baseline characteristics differ across each pairwise combination of these groups: remote versus control; in-person and choice versus control; and remote versus in-person and choice. Each row of exhibits A.4 and A.5 shows three separate tests of baseline balance:

• A test of whether the remote treatment group is statistically different from the control group, where an asterisk (*) in the last column indicates a difference at the 5-percent significance level.

• A test of whether the in-person/choice treatment group is statistically different from the control group, where a hashtag (#) in the last column indicates a difference at the 5-percent significance level.

• A test of whether the remote treatment group is statistically different from the in-person/choice treatment group, where a tilde (~) in the last column indicates a difference at the 5-percent significance level.

Statistically significant differences in individual characteristics do not indicate systematic imbalance.

Differences between groups at the 5-percent significance level would be expected in about 5 percent of the variables due to random chance. Only where differences appear as statistically significant for more than 5 percent of the variables would we be concerned about sample imbalance.

Exhibit A.4 presents tests of baseline balance on demographic measures for the fol owing sets of characteristics: race/ethnicity, gender, age, marital status, household size, education, employment, and income. Exhibit A.5 presents tests of baseline balance on measures related to stage in the homebuying process, financial capability, and creditworthiness. These measures reflect key demographic measures and potential determinants of study participant outcomes. The set of baseline characteristics for which the balance besting was conducted aligns with the set of baseline characteristics used as covariates in the study’s impact analyses. This ensures that the analysis controls for any observed, chance differences in these baseline measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 98

APPENDIX

Exhibit A.4: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents, Demographic Characteristics of Study Participants

Full

Sample of

Short-Term

Pooled

Follow-Up

In-Person

Survey

Remote

and Choice

Respond-

Treatment Treatment

Control

Statistically

ents

Group

Group

Group

Significant

Baseline Variable

(%)

(%)

(%)

(%)

Difference

Race/Ethnicity of Study Participant

Hispanic

23.3

22.6

24.5

22.8

White non-Hispanic

39.6

38.0

39.7

40.5

African-American non-Hispanic

21.2

21.7

21.1

21.0

Asian non-Hispanic

12.0

12.9

10.8

12.1

Other

4.0

4.9

3.9

3.6

Male

59.3

61.1

56.2

60.3

# ~

Age greater than or equal to 30

69.2

69.9

67.0

70.2

Marital Status of Study Participant

Married

38.2

38.6

36.0

39.5

#

Divorced, widowed, or separated

15.2

15.8

15.1

14.8

Single and never married

46.6

45.6

48.9

45.7

Plans to purchase the home with a co-borrower

26.0

25.4

24.6

27.3

Household Size

One

23.1

22.6

24.6

22.4

Two

32.9

32.7

31.7

33.8

Three

19.7

20.1

20.4

19.1

Four

14.6

14.9

13.8

14.9

Five

6.3

6.2

5.8

6.8

Six or more

3.4

3.6

3.7

3.1

Education of Study Participant

Bachelor’s degree or higher

55.5

55.7

57.1

54.4

Associate’s degree

12.9

12.5

13.2

12.9

Some college, but no degree

15.8

16.0

16.0

15.5

High school diploma or less

15.8

15.8

13.6

17.2

#

Employment

Full-time employment (30+ hours per week)

89.2

90.4

88.3

88.9

Part-time employment (1-29 hours per week)

4.3

3.6

4.6

4.6

Unemployed and looking for work

0.5

0.6

0.5

0.3

Not working, homemaker, retired, student, or

6.1

5.4

6.5

6.1

other

Income Received by Study Participant and Any

Co-Borrowers in Last 12 Months

$24,999 or less

8.0

7.9

9.0

7.4

$25,000 to $49,999

33.9

34.0

33.8

34.0

$50,000 to $74,999

32.4

31.5

32.1

33.2

$75,000 to $99,999

14.8

16.0

14.7

14.2

$100,000 or more

10.9

10.6

10.5

11.3

* Remote treatment group is statistically significantly different from control group at the p<0.05 level.

# Pooled in-person/choice treatment group is statistically significantly different from control group at the p<0.05 level.

~ Remote treatment group and pooled in-person/choice treatment group are statistically significantly different from each other at the p<0.05 level.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 99

APPENDIX

Notes: The sample comprises study participants who responded to the Short-Term Follow-Up Survey, 78.8 percent of the study sample, which is 4,546 sample members. The sample size for the Remote Treatment Group column is 1,253; the sample size for the Pooled In-Person/Choice Treatment Group column is 1,286; and the sample size for the Control Group column is 2,007.

Measure-specific sample sizes may vary because of item nonresponse. Appendix B provides additional detail on the construction of measures.

Source: Baseline survey of study participants

Exhibit A.5: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents, Measures of Homebuying Stage, Financial Capability, and Creditworthiness of Study Participants Full Sample

Pooled

of Short-Term

In-Person

Follow-Up

Remote

and Choice

Statistically

Survey

Treatment

Treatment

Control

Significant

Baseline Variable

Respondents

Group

Group

Group

Difference

Stage in the Homebuying Process (%)

Not yet started home search

10.6

10.7

11.4

9.9

Started home search, but no visits

12.9

14.3

11.2

13.1

~

Visited homes, but no offer

24.1

23.4

24.9

23.9

Made an offer on a home, but no purchase

14.1

14.4

14.1

13.9

Signed a purchase agreement

25.6

25.5

26.3

25.2

Purchased a home

12.8

11.8

12.2

13.9

Uses a written budget (%)

74.9

73.6

74.2

76.3

Usually pays credit card balance in full to avoid

77.5

77.9

78.4

76.7

interest charges (%)

Over the past year, was short on money

16.5

16.1

15.8

17.2

sometimes or often (%)

Sets aside extra money for retirement, education,

91.1

92.3

91.7

89.9

*

or to build a financial cushion sometimes or often

(%)

Everyone in household has health insurance (%)

88.4

87.9

89.2

88.1

Level of total savings and investments ($)

$54,256

$48,920

$53,665

$57,944

*

Credit Score (%)

Less than 580

3.7

3.0

3.8

3.9

580 to 619

6.5

6.8

6.7

6.2

620 to 659

15.0

14.7

14.5

15.5

660 to 699

17.2

17.5

16.3

17.5

700 to 739

19.9

20.9

18.9

19.9

740 or more

37.8

37.1

39.8

37.0

Cash on hand for downpayment and closing

$32,997

$31,600

$31,659

$34,721

costs ($)

Total nonhousing debt ($)

$24,165

$24,370

$23,742

$24,308

Monthly payment nonhousing debt ($)

$452

$457

$446

$453

* Remote treatment group is statistically significantly different from control group at the p<0.05 level.

# Pooled in-person/choice treatment group is statistically significantly different from control group at the p<0.05 level.

~ Remote treatment group and pooled in-person/choice treatment group are statistically significantly different from each other at the p<0.05 level.

Notes: The sample is comprised of study participants who responded to the Short-Term Fol ow-Up Survey, 78.8 percent of the study sample, 4,546 sample members. The sample size for the Remote Treatment Group column is 1,253; the sample size for the Pooled In-Person/Choice Treatment Group column is 1,286; and the sample size for the Control Group column is 2,007. Measure-specific sample sizes may vary because of item nonresponse. Appendix B provides additional detail on the construction of measures.

Source: Baseline survey of study participants and credit bureau data

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 100

APPENDIX

The results of the statistical tests presented in exhibits A.4 and A.5 are summarized as follows.

Of the 51 baseline variables, differences were observed (at the 5-percent level):

• For the remote versus control comparison, for two variables.

• For the in-person/choice versus control comparison, for three variables.

• For the remote versus in-person/choice comparison, for two variables.

These totals are in line with the number of differences to be expected due to random chance, and they provide evidence that the observable baseline characteristics of survey respondents are balanced across the experimental groups. Although we cannot test whether unobservable characteristics are different across the treatment groups (because they are, by definition, unobservable), we are comforted that this is not a serious concern by the lack of systematic differences in a wide range of baseline characteristics that are observable.

When analyzing the impact of homebuyer education and counseling services on outcomes defined using only the Short-Term Follow-up Survey (which necessitates limiting the sample to survey respondents), this observed balance across treatment groups provides reassurance that the reported impact estimates isolate the unbiased experimental impact of homebuyer education and counseling. Beyond this assurance, and as detailed in section A.1, we also include baseline covariates in our impact analysis to control for any random variation in these baseline measures across the groups.

A.6 Survey Nonresponse Weighting Methods

The study had a strong Short-Term Follow-Up Survey response rate of 79 percent, which means that 21 percent of the study sample did not respond to the survey. Although we find no evidence of systematic difference between the experimental groups on measurable characteristics (appendix section A.5, exhibits A.4 and A.5), we do find systematic differences between those study participants who responded to the survey and those who did not (exhibits A.6 and A.7). As noted by Hsueh et al. (2012), if survey respondents and nonrespondents differ, then the impact results for the sample of respondents might not be generalizable to the full sample.

Exhibit A.6: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents and Nonrespondents, Demographic Characteristics of Study Participants

Follow-Up

Follow-Up

Survey

Survey

Statistically

Full Sample

Respondents

Nonrespondents

Significant

Baseline Variable

(%)

(%)

(%)

Difference

Race/Ethnicity of Study Participant

Hispanic

25.1

23.3

31.9

*

White non-Hispanic

38.5

39.6

34.4

*

African-American non-Hispanic

20.5

21.2

17.7

*

Asian non-Hispanic

12.1

12.0

12.6

Other

3.9

4.0

3.4

Male

60.2

59.3

63.5

*

Age greater than or equal to 30

68.3

69.2

64.7

*

Marital Status of Study Participant

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 101

APPENDIX

Follow-Up

Follow-Up

Survey

Survey

Statistically

Full Sample

Respondents

Nonrespondents

Significant

Baseline Variable

(%)

(%)

(%)

Difference

Married

38.2

38.2

37.9

Divorced, widowed, or separated

14.8

15.2

13.3

Single and never married

47.0

46.6

48.8

Plans to purchase the home with a co-

26.2

26.0

27.1

borrower

Household Size

One

22.7

23.1

21.5

Two

32.0

32.9

28.5

*

Three

19.8

19.7

19.9

Four

15.2

14.6

17.7

*

Five

6.7

6.3

8.2

*

Six or more

3.6

3.4

4.3

Education of Study Participant

Bachelor’s degree or higher

53.4

55.5

45.7

*

Associate’s degree

12.9

12.9

13.1

Some college, but no degree

16.1

15.8

17.2

High school diploma or less

17.6

15.8

24.0

*

Employment

Full-time employment (30+ hours per

89.9

89.2

92.6

*

week)

Part-time employment (1-29 hours per

4.1

4.3

3.3

week)

Unemployed and looking for work

0.5

0.5

0.5

Not working, homemaker, retired,

5.5

6.1

3.6

*

student, or other

Income Received by Study Participant and Any Co-Borrowers in Last 12 Months

$24,999 or less

8.1

8.0

8.7

$25,000 to $49,999

34.0

33.9

34.3

$50,000 to $74,999

32.7

32.4

33.6

$75,000 to $99,999

14.6

14.8

13.7

$100,000 or more

10.6

10.9

9.7

* Survey respondents are statistically significantly different from survey nonrespondents at the p<0.05 level.

Notes: Survey respondents are the 4,546 study participants who responded to the Short-Term Fol ow-Up Survey. Survey nonrespondents are the 1,224 study participants who did not respond to the Short-Term Fol ow-Up Survey. Appendix B

provides additional detail on the construction of measures.

Source: Baseline survey of study participants

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 102

APPENDIX

Exhibit A.7: Baseline Balance Testing for Short-Term Follow-Up Survey Respondents and Nonrespondents, Measures of Homebuying Stage, Financial Capability, and Creditworthiness of Study Participants Follow-Up

Follow-Up

Statistically

Survey

Survey

Significant

Baseline Variable

Full Sample

Respondents Nonrespondents

Difference

Stage in the Homebuying Process (%)

Not yet started home search

11.2

10.6

13.7

*

Started home search, but no visits

13.2

12.9

14.3

Visited homes, but no offer

23.8

24.1

23.0

Made an offer on a home, but no purchase

13.2

14.1

9.8

*

Signed a purchase agreement

25.3

25.6

24.4

Purchased a home

13.2

12.8

14.7

Uses a written budget (%)

74.7

74.9

73.8

Usually pays credit card balance in full to avoid

77.4

77.5

76.9

interest charges (%)

Over the past year, was short on money

16.4

16.5

15.7

sometimes or often (%)

Sets aside extra money for retirement,

91.1

91.1

91.3

education, or to build a financial cushion

sometimes or often (%)

Everyone in household has health insurance

88.3

88.4

88.1

(%)

Level of total savings and investments ($)

$51,962

$54,256

$43,417

*

Credit Score (%)

Less than 580

4.0

3.7

5.2

*

580 to 619

6.8

6.5

7.8

620 to 659

15.4

15.0

16.7

660 to 699

17.6

17.2

19.2

700 to 739

19.9

19.9

19.7

740 or more

36.4

37.8

31.4

*

Cash on hand for downpayment and closing

$32,207

$32,997

$29,262

*

costs ($)

Total nonhousing debt ($)

$23,835

$24,165

$22,605

Monthly payment nonhousing debt ($)

$460

$452

$489

*

* Survey respondents are statistically significantly different from survey nonrespondents at the p<0.05 level.

Notes: Survey respondents are the 4,546 study participants who responded to the Short-Term Fol ow-Up Survey. Survey nonrespondents are the 1,224 study participants who did not respond to the Short-Term Fol ow-Up Survey. Appendix B provides additional detail on the construction of measures.

Source: Baseline survey of study participants.

Recall from section A.4 that we exclude observations with missing data for a given outcome. For outcomes defined using only Short-Term Follow-Up Survey data, this implies that all study participants who did not respond to the survey are dropped from the analysis. To ensure that our impact results are generalizable to the full study sample, we apply sample weights that adjust for Short-Term Follow-Up Survey nonresponse for analyses of outcomes collected from the survey.

We generated nonresponse weights as follows. 86

86 This method is commonly used in applied evaluation research and is described by Hsueh et al. (2012), for example.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 103

APPENDIX

• Step 1: We calculated the predicted probability that a sample member responded to the Short-Term Follow-Up Survey by modeling response to the survey as a function of a treatment group indicator and the same set of covariates included in the impact analysis model described above. For missing baseline covariate data, we again used the dummy variable adjustment approach.

• Step 2: To guard against reduced precision that could result from inclusion of small or large weights, in this step we conduct weight trimming, which bounds the calculated predicted probabilities of Follow-Up Survey response. 87 In practice, predicted probabilities less than 0.5 were set equal to 0.5, resulting in predicted probabilities of survey response bounded between 0.5 and 1.0.

• Step 3: We calculated each survey respondent’s weight by dividing the overall survey response rate by the predicted probability of response as calculated in Steps 1 and 2. This method ensures that higher weights are assigned to individuals with characteristics that were underrepresented (relative to the baseline survey sample) among the survey respondent sample.

• Step 4: Finally, we calculated adjusted nonresponse weights by dividing each sample member’s nonresponse weight (as calculated in Step 3) by the overall mean value of the nonresponse weights. Scaling the nonresponse weights by the mean ensures that the mean of the adjusted nonresponse weights is equal to 1, and that the sum of all adjusted nonresponse weights is equal to the sample size.

A.7 Ability to Detect Impacts: Minimum Detectable Effects

The minimum detectable effect (MDE) is the smallest true intervention impact that can be detected with a given level of confidence. MDEs are helpful for understanding findings that are not statistically significant because MDEs indicate how large the impact would have needed to be to be detected at a given level of confidence.

MDEs are a function of a variety of factors, including:

• Statistical Significance Level: The statistical significance level is the probability of identifying a “false positive” result (also referred to as “type I error”). The MDE becomes larger as the statistical significance level decreases. In this application, we have set the statistical significance level to 10 percent.

87 As described by Izrael, Battaglia, and Frankel (2009), weight trimming refers to increasing the value of low weights and decreasing the value of high weights to reduce their impact on the variance of the estimates. By trimming low- and high-weight values, one generally lowers sampling variability, but could incur some bias. The mean squared error will be lower if the reduction in variance is large relative to the increase in bias arising from weight trimming.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 104

APPENDIX

• Statistical Power: The statistical power is equal to the probability of rejecting the null hypothesis if the alternative hypothesis is true (or, 1 minus the probability of a “false negative” result; “type II error”). The MDE becomes larger as statistical power increases. In this application, we set statistical power to 80 percent.

• Variance of the Impact Estimate: Variance is essentially a measure of the “noisiness” of the impact estimate. The MDE becomes larger as the variance of the impact estimate increases.

Because the variance of the impact estimate is inversely related to sample size, the MDE is also inversely related to sample size. 88 The sample size varies depending on the specific subset of the full study sample used for a given analysis. For instance, the sample size for estimating the overall impact of homebuyer education and counseling is 5,770; the sample size is 1,990 for estimating the impact of in-person services; and the sample size is 4,964 for estimating the impact of remote services (ignoring missing outcome data).

The excerpt from exhibit C.1 shows that the MDE for our confirmatory outcome— Ever 60 days delinquent—is 0.5. That is, the true impact of homebuyer education on the share of the sample who were ever 60 days delinquent would need to be 0.5 percentage points in order to be detected as statistically significantly different from 0 with a high level of confidence (that is, at the 10-percent significance level 80 percent of the time). Indeed, the estimated impact on this outcome is 0.2 percentage point, 0.3 point less than the corresponding MDE and the intervention was not determined to have a statistically significantly impact on this outcome.

Excerpt from Exhibit C.1: Minimum Detectable Effects for Confirmatory Outcome

Overall Impact of

Treatment

Control

Being Offered

Minimum

Outcome

Group Mean

Group Mean

Services

Detectable Ef ect

Ever 60 days delinquent (%)

0.7

0.5

0.2

0.5

(0.2)

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. See full exhibit notes in exhibit C.1.

Implications for the Study’s Ability to Detect an Impact on the Confirmatory Outcome The magnitude of the MDE for the confirmatory outcome suggests that we are underpowered to detect impacts on this outcome in this Short-Term Impact Report. Given that the control group mean outcome is 0.5 percent—that is, only 0.5 percent of the control group had ever been 60

days delinquent at the time of short-term followup—the MDE of 0.5 percentage point implies that homebuyer education and counseling would have needed to reduce the share of the treatment 88 The variance of the impact estimate (and, therefore, the MDE) is also influenced by the amount of variability in the outcome that can be explained by baseline covariates; the distribution of observations between treatment and control groups; and adjustments to the standard error of the impact estimate for clustering at the site level.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 105

APPENDIX

group who were ever 60 days delinquent to 0.0 percent in order for us to have a reasonable chance of detecting a statistically significant effect. An impact this large in magnitude seems unlikely, because it implies a relative percentage impact (computed as the impact divided by the control group mean) of 100 percent. 89 That the MDE of 0.5 is greater than the largest overall impact on the outcome Ever 60 days delinquent that we might reasonably expect homebuyer education and counseling to have indicates that we are underpowered to detect impacts on this outcome in this Short-Term Impact Report.

As time goes on, we expect a greater share of control group members to become delinquent. As a result, although the absolute value of the MDE will remain practically unchanged, it will reflect a smaller relative percentage impact. For example, suppose 3.0 percent of control group members are ever 60 days delinquent at the time of long-term followup period of the Long-Term Impact Report. Then, with the same MDE of 0.5 percentage point, homebuyer education and counseling would need to reduce the share of the treatment group who were ever 60 days delinquent to 2.5

percent in order for the study to have a high probability of detecting an effect. This 0.5

percentage point impact represents a 16.7 percent percentage impact—notably smaller (and therefore more plausible to arise) than the 100 percent relative impact needed to detect an impact as of this Short-Term Impact Report’s time point.

MDEs and Sample Size

Given that MDEs increase as sample size decreases, it is more difficult to detect statistically significant impacts for each service delivery mode (as distinct from the overall impact of services) simply by virtue of the smaller sample sizes available for estimating those mode effects. Similarly, it is more difficult to detect statistically significant impacts for in-person services than for remote services because the sample size available for estimating the impact of in-person services is comparatively small.

For instance, as shown in the excerpt from exhibits D.1 and D.2, the MDE for the impact of being offered in-person services on the outcome Ever 60 days delinquent is 1.2 and the MDE for the impact of being offered remote services is 0.5. Therefore, the impact of in-person services has to be relatively much larger to be detected with confidence. In other words, the in-person intervention has a “higher bar” to surmount to attain statistical significance than does the remote intervention.

89 The percentage impact helps normalize the magnitude of impacts and thereby provides context for interpreting the magnitude of the treatment-control difference.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 106

APPENDIX

Excerpt from Exhibit D.1 and D.2: Minimum Detectable Effects for Mode Effects

Impact of Being

Impact of Being

Of ered In-Person

Minimum

Of ered Remote

Minimum

Outcome

Services

Detectable Ef ect

Services

Detectable Ef ect

Ever 60 days delinquent (%)

0.3

1.2

0.2

0.5

(0.5)

(0.2)

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. See full exhibit notes in exhibits D.1 and D.2.

Similarly, it is more difficult to detect statistically significant impacts on subgroups of the study sample (as distinct from the overall impact on the full study sample), given that lower sample sizes are available for estimating impacts on a given subgroup. 90 Further, it is more difficult to detect impacts on subgroups with smaller sample sizes than for those with larger sample sizes.

In sum, this study reports on impacts that are statistically significantly different from 0. We are careful to note that, for outcomes where the impacts are not statistically significantly different from 0, this does not necessarily mean that there are no impacts. There are instead, more accurately, “no detectable impacts.” There may be impacts that are smaller than this study is powered to detect for a variety of reasons, as discussed in this subsection.

90 Impacts on select subgroups of the study sample are presented in Appendices E, F, and G.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 107

APPENDIX

Appendix B: Data Sources and Measures

This appendix details the study’s data sources and measures. Section B.1 describes the data sources used throughout this report. Section B.2 provides additional details on the construction of baseline covariates, subgroup identifiers, and outcome measures used for the study’s impact analyses.

B.1 Data Sources

This report uses data from a variety of sources. We discuss each below.

Baseline Survey

All mortgage customers referred by the three lenders who agreed to participate in the study completed a baseline survey after they were determined to be eligible for the study and signed the study’s consent agreement. The baseline survey was conducted over the telephone using computer-assisted telephone interviewing software. The baseline survey topics included questions on the study participant’s stage in the homebuying process, preparation for home purchase, financial capability, mortgage knowledge, current housing status, budgeting and debts, and demographic characteristics. We used the baseline data to describe the study participants for this report, to construct covariates for the impact analysis, and to define subgroups.

Short-Term Follow-Up Survey

The Short-Term Follow-Up Survey was administered on a rolling basis, beginning at 12 months after random assignment, based on when the study participant was randomly assigned.

Interviewers attempted to complete all cases within a 3- to 4-month window of sample releases to ensure that the data collected reflected a consistent followup period of 12 to 15 months. The cases first were worked by telephone interviewers who attempted to contact the respondents using all available contact information. If they could not reach a sample member by telephone, the case was sent to field interviewers, who then attempted to contact the member in person.

The Short-Term Follow-Up Survey provides information on a variety of outcomes related to preparedness and search, financial capability, and sustainable homeownership.

Study participants replied to the Short-Term Follow-Up Survey an average of 13.2 months after enrolling the study and being randomly assigned (the median time is 13 months). The overall response rate to the Short-Term Follow-Up Survey was 78.8 percent—82.0 percent of the control group responded to the Short-Term Follow-Up Survey, and 76.4 percent of the treatment group responded.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 108

APPENDIX

Exhibit B.1: Timing of Short-Term Follow-Up Survey Response

Cumulative

Number of Survey

Percentage of Survey Percentage of Survey

Follow-Up Montha

Respondents

Respondents

Respondents

9

1

0.0

0.0

11

101

2.2

2.2

12

1,656

36.4

38.7

13

1,560

34.3

73.0

14

415

9.1

82.1

15

215

4.7

86.8

16

207

4.6

91.4

17

110

2.4

93.8

18

83

1.8

95.6

19

36

0.8

96.4

20

22

0.5

96.9

21

15

0.3

97.3

22-24

13

0.3

97.5

Unknown

112

2.5

100.0

Summary Information

Median followup month

13

Mean followup month

13.2

Number of study participants who responded

4,546

to Short-Term Follow-Up Survey

Number of study participants who did not

1,224

respond to Short-Term Follow-Up Survey

Number of withdrawals

84

a Follow-up month is calculated by subtracting the month that the study participant completed the Short-Term Follow-Up Survey from the month of random assignment.

Credit Bureau Data

Data provided to the study team by one of the national credit bureaus included each study participant’s credit score, measures of mortgage loan and nonhousing debt, and mortgage delinquency. During the enrollment period, we collected credit bureau data every 2 months, for a total of 13 credit bureau data extracts, to capture baseline credit attributes of the study participants. Collecting credit bureau data every 2 months enabled us to match each study participant to credit attributes observed 0 to 2 months prior to his or her enrollment in the study.

Then, during the followup period, we requested followup credit bureau data every 6 months, allowing us to match each study participant to credit attributes captured between 12 and 18

months after random assignment. Credit bureau data on housing outcomes and loan performance from these followup extracts cover 92.7 percent of the study sample and provide outcome The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 109

APPENDIX

measures for the impact analyses. We observe study participants’ credit bureau data an average of 14.3 months after they enrolled in the study (the median time is 14 months). 91

Exhibit B.2: Timing of Followup Credit Data

Cumulative Percentage

Number of Study

Percentage of Study

of Study Participants

Participants who have Participants who have

who have Followup

Follow-Up Montha

Followup Credit Data

Followup Credit Data

Credit Data

11

6

0.1

0.1

12

917

17.2

17.3

13

1,099

20.6

37.8

14

987

18.5

56.3

15

877

16.4

72.7

16

824

15.4

88.1

17

636

11.9

100.0

Summary Information

Median followup month

14

Mean followup month

14.3

Number of study participants who have

followup credit data

5,346

Number of study participants who do not

have followup credit data

424

Number of withdrawals

84

a Follow-up month is calculated by subtracting the month that credit data is observed for the study participant from the month of random assignment.

Loan Origination and Servicing Data from Participating Lenders and FHA

We also received loan origination and servicing data from the three participating lenders and from the Federal Housing Administration (FHA). We requested this data every 6 months during the followup period, allowing us to match each study participant to loan origination and servicing data captured 12 to 18 months after random assignment.

The primary use of the origination and servicing data is to construct outcomes related to mortgage characteristics and payment history.

Counseling Agency Service Tracking Data

The agencies’ service tracking data provides a detailed record of the homebuyer education and counseling services provided to study participants. This is the primary source of information on treatment take-up, intensity, and completion.

91 All credit bureau data extracts are “soft” inquiries, meaning they are not recorded as a credit inquiry and do not otherwise affect a study participant’s credit record/score.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 110

APPENDIX

Study Participant Focus Groups

During the final 5 months of the enrollment period, the study team conducted 14 focus groups across four study sites, with a total of 64 treatment group members. The focus groups explored treatment group members’ progress toward home purchase and their interaction with the participating housing counseling agencies or remote education and counseling providers. Focus group participants included treatment group members who had and who had not completed the offered homebuyer education and counseling services. This diversity of participation experience permitted us to explore focus group participants’ reasons for completing or not completing services. This study’s Baseline Report (DeMarco et al., 2017) provides a detailed description of the focus group findings; this Short-Term Impact Report provides a summary of the findings.

B.2 Combining Data Sources for Impact Analysis

The study combines data from multiple sources to construct some of the key outcomes used in the impact analysis. This strategy helps to address the fact that each individual data source has incomplete coverage, as follows:

• Despite a high response rate (78.8 percent), Short-Term Follow-Up Survey data are not available for all study participants.

• Credit bureau data are not available for all loan origination and servicing measures, and the data that are available do not cover the full sample (coverage rate is 92.7 percent). Study participants could be missing credit bureau data if they had not previously borrowed money.

• FHA has origination and servicing information for FHA loans (comprising 21.5 percent of all study participants). This data excludes study participants who chose non-FHA loans or who did not become homeowners during the period covered by this report.

• Participating lenders have origination data for study participants who originate a loan with that particular lender. Because study participants are referred early in the homebuying process, some study participants might have purchased using a lender other than the one that referred them to the study. For their part, lenders have servicing data for study participants only when the lender services the loan internally. If servicing was transferred, then the lender’s servicing data are truncated.

The implication of using multiple data sources to construct our outcome measures is that we have a high coverage rate for outcomes constructed using data available from all of these sources. Outcomes constructed using data available from all of these sources cover 5,707 study participants (98.9 percent of the study sample). The next section details which data sources we used to construct each outcome included in the impact analysis. Some outcomes we constructed using data available from all data sources, whereas some were constructed using data available from only a subset of data sources, depending on the availability of the data.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 111

APPENDIX

B.3 Measure Construction

This section provides additional details on the construction of baseline covariates (exhibit B.3), subgroup identifiers (exhibit B.4), and outcome measures used for the study’s impact analyses (exhibit B.5).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 112

APPENDIX

Exhibit B.3: Operationalization of Baseline Covariates

Domain

Variable Description Operationalization

Data Source(s)

Demographic

Race/ethnicity of

Series of mutually exclusive binary variables:

Baseline survey

Characteristics study participant

• Hispanic

• White non-Hispanic

• African-American non-Hispanic

• Asian non-Hispanic

• Other race

Demographic

Gender of study

Binary variable that takes on value:

Baseline survey

Characteristics participant

• 0 if female

• 1 if male

Demographic

Age 30 or older at

Binary variable that takes on value:

Credit bureau data

Characteristics baseline

• 0 if age 29 or younger at baseline

• 1 if age 30 or older at baseline

Demographic

Marital status of

Series of mutually exclusive binary variables:

Baseline survey

Characteristics study participant

• Married

• Divorced, widowed, or separated

• Single and never married

Demographic

Plans to purchase the Binary variable that takes on value:

Baseline survey

Characteristics home with a co-

• 0 if does not plan to purchase the home with a co-

borrower

borrower

• 1 if plans to purchase the home with a co-borrower

Demographic

Household size

Series of mutually exclusive binary variables:

Baseline survey

Characteristics

• One

• Two

• Three

• Four

• Five

• Six or more

Demographic

Education of study

Series of mutually exclusive binary variables that capture Baseline survey

Characteristics participant

educational attainment:

• Bachelor’s degree or higher

• Associate’s degree

• Some college, but no degree

• High school diploma or less

Stage in the

Stage in the

Series of mutually exclusive binary variables that capture Baseline survey

Homebuying

homebuying process the stage in the homebuying process:

Process

• Not yet started home search

• Started home search, but no visits

• Visited homes, but no offer

• Made an offer on a home, but no purchase

• Signed a purchase agreement

• Purchased a home

Employment and Employment

Series of mutually exclusive binary variables for

Baseline survey

Income

employment status of the study participant:

• Full-time employment (30+ hours per week)

• Part-time employment (1-29 hours per week)

• Unemployed and looking for work

• Not working, homemaker, retired, student, or other

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 113

APPENDIX

Domain

Variable Description Operationalization

Data Source(s)

Employment and Income received by Series of mutually exclusive binary variables defined Baseline survey

Income

study participant and based on the reported income received by study

any co-borrowers in participant and any co-borrowers in last 12 months:

last 12 months

• $24,999 or less

• $25,000 to $49,999

• $50,000 to $74,999

• $75,000 to $99,999

• $100,000 or more

Note: If the study participant reported an income range

rather than a specific value (for example, $40,000 to less

than $55,000), we used the midpoint of the reported range

to determine the appropriate income category. If neither a

value nor a range was reported, then the variable was set

to missing

Financial

Uses a written budget Binary variable that takes on value:

Baseline survey

Responsibility

• 0 if does not have a budget of monthly household

expenses

• 1 if has a budget of monthly household expenses

Financial

Usually pays credit Binary variable that takes on value:

Baseline survey

Responsibility

card balance in full to • 0 if does not usually pay credit card balance in full to avoid interest

avoid interest charges

charges

• 1 if does usually pay credit card balance in full to

avoid interest charges

Financial

Over the past year, Binary variable that takes on value:

Baseline survey

Responsibility

was short on money • 0 if reports being short on money rarely or never

sometimes or often • 1 if reports being short on money sometimes or often

Financial

Sets aside extra

Binary variable that takes on value:

Baseline survey

Responsibility

money for retirement, • 0 if reports setting aside extra money rarely or never

education, or to build • 1 if reports setting aside extra money sometimes or

a financial cushion

often

sometimes or often

Financial

Everyone in

Binary variable that takes on value:

Baseline survey

Responsibility

household has health • 0 if at least one household member does not have

insurance

health insurance

• 1 if everyone in the household has health insurance

Financial

Level of total savings Sum of reported values for checking accounts, savings

Baseline survey

Responsibility

and investments

accounts, retirement accounts, and other savings and

investment accounts (continuous variable)

Note: The responses were capped at $999,999 for each

of these separate categories, and the level of total savings

and investments was top coded at its 99th percentile

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 114

APPENDIX

Domain

Variable Description Operationalization

Data Source(s)

Creditworthiness Credit score

Series of mutually exclusive binary variables defined

Credit bureau data

based on credit score of the study participant at baseline:

• Less than 580

• 580 to 619

• 620 to 659

• 660 to 699

• 700 to 739

• 740 or more

Note: For each study participant, we capture the person’s

baseline credit bureau data within 2 months prior to

his/her enrollment

Creditworthiness Cash on hand for

Total cash on hand for downpayment and closing costs Baseline survey

downpayment and

(continuous variable)

closing costs

Note: If study participant did not provide an exact amount

but reported a range, we used the midpoint of the range

and included this value in the continuous measure. Cash

on hand for downpayment and closing costs was top

coded at its 99th percentile

Creditworthiness Total nonhousing

Total nonhousing debt equals the total balance on open Credit bureau data

debt

installment accounts plus open revolving accounts minus

the balance on open mortgage accounts. This measure is

set equal to 0 if the study participant was included in the

credit file but there were no open debt accounts on file

Note: Total nonhousing debt was top coded at its 99th

percentile

Creditworthiness Monthly payment

Monthly scheduled payments for nonhousing debt equals Credit bureau data

nonhousing debt

the total scheduled monthly payments for all open

accounts besides mortgage accounts. This measure is set

equal to 0 if the study member was included in the credit

file but there were no open debt accounts on file

Note: Monthly payment nonhousing debt was top coded at

its 99th percentile

Service Mode

Baseline preference Binary variable that takes on value:

Baseline eligibility

Preference

for remote services • 0 if enrolled prior to study redesign or if baseline

assessment

preference for in-person services

• 1 if enrolled after study redesign and baseline

preference for remote services

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 115

APPENDIX

Exhibit B.4: Operationalization of Subgroup Identifiers

Domain

Subgroup Comparison Operationalization

Data Source(s)

Demographic

White non-Hispanic

Binary variable that takes on value:

Baseline survey

Characteristics compared with all other

• 0 if Hispanic; African-American non-Hispanic;

races/ethnicities

Asian non-Hispanic; or other race/ethnicity

• 1 if White non-Hispanic

Demographic

Bachelor’s degree or

Binary variable that takes on value:

Baseline survey

Characteristics higher compared with less • 0 if associate’s degree; some college, but no than a bachelor’s degree

degree; or high school diploma or less

• 1 if bachelor’s degree or higher

Demographic

Married compared with not Binary variable that takes on value:

Baseline survey

Characteristics currently married

• 0 if divorced, widowed, separated, or single and

never married

• 1 if married

Demographic

Age 30 or older at

Binary variable that takes on value:

Credit bureau data

Characteristics baseline compared with • 0 if age 29 or younger at baseline age 29 or younger at

• 1 if age 30 or older at baseline

baseline

Demographic

English language

Binary variable that takes on value:

Baseline eligibility

Characteristics preference compared with • 0 if Spanish language preference assessment

Spanish language

• 1 if English language preference

preference

Stage in the

Made an offer on a home, Binary variable that takes on value:

Baseline survey

Homebuying

signed a purchase

• 0 if not yet started home search; started home

Process

agreement, or purchased

search, but no visits; or visited homes, but no

a home compared with

offer

early stage in the

• 1 if made an offer on a home but no purchase;

homebuying process

signed a purchase agreement; or purchased a

home

Financial

Credit score 680 or above Binary variable that takes on value:

Credit bureau data

Characteristics compared with credit

• 0 if baseline credit score below 680

score below 680

• 1 if baseline credit score above or equal to 680

Financial

Savings $20,000 or higher Binary variable that takes on value:

Baseline survey

Characteristics compared with savings

• 0 if baseline savings less than $20,000

less than $20,000

• 1 if baseline savings greater than or equal to

$20,000

Financial

Cash on hand for

Binary variable that takes on value:

Baseline survey

Characteristics downpayment and closing • 0 if baseline cash on hand for downpayment costs $15,000 or higher

and closing costs less than $15,000

compared with less than • 1 if baseline cash on hand for downpayment

$15,000

and closing costs greater than or equal to

$20,000

Financial

Borrower income 80

Binary variable that takes on value:

a

Characteristics percent of area median

• 0 if borrower income less than 80 percent of

income or higher

area median family income

compared with borrower • 1 if borrower income 80 percent of area median

income less than 80

family income or higher

percent of area median

income

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 116

APPENDIX

Domain

Subgroup Comparison Operationalization

Data Source(s)

Financial

Nonhousing debt $10,000 Binary variable that takes on value:

Credit bureau data

Characteristics or higher compared with • 0 if baseline nonhousing debt less than $10,000

nonhousing debt less than • 1 if baseline nonhousing debt greater than or

$10,000

equal to $10,000

Housing Market Ratio of area median

Binary variable that takes on value:

b

value of owner-occupied • 0 if ratio of area median value of owner-

unit to area median

occupied unit to area median family income is

income is above national

less than ratio of national median value of

median compared with

owner-occupied unit to national median family

ratio of area median value

income

of owner-occupied unit to • 1 if ratio of area median value of owner-

area median income is

occupied unit to area median family income is

below national median

greater than or equal to ratio of national median

value of owner occupied unit to national median

family income

a The area median family incomes are from the Federal Financial Institutions Examination Council’s FFIEC Median Family Income Report (FFIEC, 2013) . The addresses used to determine which area median income is matched to each study participant are from the baseline survey.

b The area median values of an owner-occupied unit are from 2013 American Community Survey data. The area median family incomes are from the Federal Financial Institutions Examination Council’s FFIEC Median Family Income Report (FFIEC, 2013) . The national median value of an owner-occupied unit and the national median family income are from the 2013 American Community Survey, accessed via the Census Bureau’s American FactFinder (U.S. Census Bureau, 2013).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 117

APPENDIX

Exhibit B.5: Operationalization of Outcome Measures

Domain

Variable Description Operationalization

Data Source(s)

Confirmatory

Sustainable

Ever 60 days

Binary variable that takes on value:

Short-Term Follow-Up

Homeownership delinquent

• 1 if ever 60 days delinquent on mortgage loan

Survey; credit bureau

• 0 otherwise

data; lender data; FHA

data

Secondary

Preparedness

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

and Search

purchased a home

• 1 if observed that study participant purchased a

Survey; credit bureau

home in Short-Term Follow-Up Survey, credit

data; lender data; FHA

bureau data, lender data, or FHA data

data

• 0 if did not observe that study participant

purchased a home in Short-Term Follow-Up

Survey, credit bureau data, lender data, or FHA

data, but study participant has data from at least

one of these data sources

Preparedness

Study participant was Binary variable that takes on value:

Short-Term Follow-Up

and Search

confident that he/she • 1 if study participant reports being confident or

Survey

could find the

very confident that he/she could find information

information he/she

about the homebuying process

needed about the

• 0 if study participant reports being somewhat

homebuying process

confident or not confident at all that he/she could

find information about the homebuying process

Financial

Number of correct

Sum of correct answers to mortgage literacy quiz

Short-Term Follow-Up

Capability

answers (out of four) (ranges from 0 to 4). Below, we list the four mortgage Survey to mortgage literacy literacy quiz questions from Short-Term Follow-Up

quiz

Survey. The correct answer appears in brackets. If a

study participant indicated “don’t know” or refused to

answer a given question, it was marked as incorrect

B2. The interest rate on a mortgage loan is the same

thing as the annual percentage rate (APR). [False]

B3. A home equity loan is secured by your house.

[True]

B4. When you first get a mortgage loan, only a small

portion of your monthly payment, if any, reduces the

amount you owe. Most of your monthly payment is

applied to interest. [True]

B5. The loan officer is legally obligated to tell you if you

qualify for a different loan product that has a lower cost.

[False]

Financial

Credit score

Credit score (continuous variable)

Credit bureau data

Capability

Financial

Study participant has Binary variable that takes on value:

Credit bureau data

Capability

a credit score above • 1 if credit score is above or equal to 620

or equal to 620

• 0 if credit score is below 620

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 118

APPENDIX

Domain

Variable Description Operationalization

Data Source(s)

Financial

Total nonhousing

Total nonhousing debt equals the total balance on open Credit bureau data

Capability

debt

installment accounts plus open revolving accounts

minus the balance on open mortgage accounts. This

measure is set equal to 0 if the study participant was

included in the credit file but there were no open debt

accounts on file

Note: Total nonhousing debt was top coded at its 99th

percentile

Financial

Study participant has Binary variable that takes on value:

Short-Term Follow-Up

Capability

a budget and often • 1 if has a budget and often compares it with actual Survey compares it against

spending

actual spending

• 0 if has a budget but sometimes, rarely, or never

compares it with actual spending, or study

participant does not have a budget

Financial

If in financial difficulty, Binary variable that takes on value:

Short-Term Follow-Up

Capability

the study participant • 1 if would contact his/her lender for assistance

Survey

would contact his/her

prior to missing a mortgage payment

lender for assistance • 0 if would not contact his/her lender for assistance

prior to missing a

regarding missed payments or would wait to

mortgage payment

contact lender until after missed payment

Note: The Short-Term Follow-Up Survey questions

used to construct this outcome were asked only of

study participants who have a mortgage loan.

Therefore, to ensure that this outcome is defined for all

enrollees (thereby maintaining the integrity of the

experimental design), this outcome was set equal to 0 if

the study participant does not have a mortgage loan

Sustainable

Monthly-housing-

Monthly-housing-costs to monthly-income ratio, where Short-Term Follow-Up

Homeownership costs to monthly-

monthly income is monthly income reported at 12-

Survey

income ratio

month followup and monthly housing costs are set

equal to:

• Monthly rent if study participant rents a house or

apartment

• Monthly mortgage payment if study participant

owns the home he/she lives in

• 0 if study participant lives in someone else’s house

or apartment without paying rent

• Missing if responded “don’t know,” refused to

answer, or indicated an alternative housing

arrangement (for example, military setting, col ege

dorm, correctional facility, hotel/motel, homeless,

etc.)

Sustainable

Ever 30 days

Binary variable that takes on value:

Short-Term Follow-Up

Homeownership delinquent

• 1 if ever 30 days delinquent on mortgage loan

Survey; credit bureau

• 0 otherwise

data; lender data; FHA

data

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 119

APPENDIX

Domain

Variable Description Operationalization

Data Source(s)

Exploratory

Preparedness

Study participant was Binary variable that takes on value:

Short-Term Follow-Up

and Search

very satisfied with the • 1 if reports being very satisfied with the

Survey

homebuying process

homebuying process

• 0 if reports being somewhat satisfied, somewhat

dissatisfied, or very dissatisfied with the

homebuying process

Preparedness

Number of lenders

Total number of lenders from which study participant

Short-Term Follow-Up

and Search

from which the study received price quotes (continuous variable). Variable

Survey

participant received equals 0 if study participant did not contact any lenders

price quotes

or if study participant contacted lenders but did not

receive any quotes

Note: Top coded at 10

Financial

Total monthly debt-to- When constructing total debt-to-income ratio (back-end Short-Term Follow-Up Capability

income ratio (back-

ratio), preference was given to back-end ratio values

Survey; credit bureau

end ratio)

provided directly by lenders and FHA. If the back-end data; lender data; FHA

ratio was not available from these sources, we used

data

data on total monthly debt expense from the credit

bureau and income from the Short-Term Follow-Up

Survey to construct this measure

Note: Total monthly debt expense from the credit

bureau data was top coded at its 99th percentile.

Follow-up income used to construct this measure was

top coded at its 99th percentile and bottom coded at its

1st percentile (to eliminate incomes of $0 appearing in

the denominator). Additionally, the resulting back-end

ratio was top coded at its 99th percentile

Financial

Total monthly debt-to- Binary variable that takes on value:

Short-Term Follow-Up

Capability

income ratio (back-

• 1 if total monthly debt-to-income ratio (back-end

Survey; credit bureau

end ratio) greater

ratio) is greater than 0.43

data; lender data; FHA

than 0.43

• 0 if total monthly debt-to-income ratio is less than data

or equal to 0.43

Financial

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

occasional y does not • 1 if agrees or strongly agrees

Survey

have enough money • 0 if disagrees or strongly disagrees

to cover all of the bil s

at the end of the

month

Financial

Student loan balance Student loan balance (top coded at 99th percentile)

Credit bureau data

Capability

Financial

Credit card balance Credit card balance (top coded at 99th percentile)

Credit bureau data

Capability

Financial

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

usually has enough • 1 if agrees or strongly agrees

Survey

savings set aside to • 0 if disagrees or strongly disagrees

cover 3 months of

expenses

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 120

APPENDIX

Domain

Variable Description Operationalization

Data Source(s)

Financial

Study participant has Binary variable that takes on value:

Short-Term Follow-Up

Capability

tried to figure out how • 1 if agrees or strongly agrees

Survey

much he/she needs • 0 if disagrees or strongly disagrees

to save for retirement

Financial

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

never uses payday • 1 if agrees or strongly agrees

Survey

lenders

• 0 if disagrees or strongly disagrees

Financial

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

usually pays credit

• 1 if agrees or strongly agrees

Survey

card balance in full to • 0 if disagrees or strongly disagrees

avoid interest

charges

Financial

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

often saves money • 1 if often saves money

Survey

• 0 otherwise

Financial

Total savings and

Sum of reported values for checking accounts, savings Short-Term Follow-Up

Capability

investments

accounts, retirement accounts, and other savings and Survey

investment accounts (continuous variable)

Note: The responses were capped at $999,999 for each

of these separate categories, and the level of total

savings and investments was top coded at its 99th

percentile

Financial

Study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

knows how to correct • 1 if agrees or strongly agrees

Survey

inaccurate

• 0 if disagrees or strongly disagrees

information in credit

report

Financial

Study participant has Binary variable that takes on value:

Short-Term Follow-Up

Capability

an electronic or

• 1 if has an electronic or written budget

Survey

written budget

• 0 if does not have an electronic or written budget

Financial

Regularly required

Binary variable that takes on value:

Short-Term Follow-Up

Capability

mortgage payment is • 1 if regularly required mortgage payment is

Survey

automatically

automatically deducted from a bank account

deducted from a bank • 0 if regularly required mortgage payment is not

account

automatically deducted from a bank account or no

mortgage loan

Financial

If study participant

Binary variable that takes on value:

Short-Term Follow-Up

Capability

started having

• 1 if would pay mortgage first if started having

Survey

financial problems

financial problems and could not pay all of bil s

and could not pay all • 0 otherwise

of the bil s, he/she

would pay the

mortgage first

Financial

Nonhousing debt

Binary variable that takes on value:

Credit bureau data

Capability

bankruptcy,

• 1 if bankruptcy, foreclosure, or repossession due

foreclosure, or

to nonhousing debt

repossession

• 0 otherwise

indicator

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 121

APPENDIX

Domain

Variable Description Operationalization

Data Source(s)

Financial

If in financial difficulty, Binary variable that takes on value:

Short-Term Follow-Up

Capability

study participant

• 1 if would contact his/her housing counseling

Survey

would contact his/her

agency, consumer credit counseling agency, or

housing counseling

other nonprofit organization for assistance prior to

agency, consumer

missing a mortgage payment

credit counseling

• 0 if would not contact his/her housing counseling

agency, or other

agency, consumer credit counseling agency, or

nonprofit organization

other nonprofit organization for assistance

for assistance prior to

regarding missed payments or would wait to

missing a mortgage

contact lender until after missed payment

payment

Note: The Short-Term Follow-Up Survey questions

used to construct this outcome were asked only of

study participants who have a mortgage loan.

Therefore, to ensure that this outcome is defined for all

enrollees (thereby maintaining the integrity of the

experimental design), this outcome was set equal to 0 if

the study participant does not have a mortgage loan

Sustainable

Monthly housing

Binary variable that takes on value:

Short-Term Follow-Up

Homeownership costs exceed 30

• 1 if monthly housing costs exceed 30 percent of

Survey

percent of monthly

monthly income

income

• 0 if monthly housing costs are less than or equal to

30 percent of monthly income

Sustainable

Ever 90 days

Binary variable that takes on value:

Short-Term Follow-Up

Homeownership delinquent

• 1 if ever 90 days delinquent on mortgage loan

Survey; Credit bureau

• 0 otherwise

data; lender data; FHA

data

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 122

APPENDIX

Appendix C: Expanded Results for the Overall Impact of Services

This appendix presents expanded results for

the overall impact of the Demonstration’s

Key Findings: Expanded Results for

homebuyer education and counseling,

the Overall Impact of Services

including additional information related to the

This appendix presents expanded results for the

overal impact of the Demonstration’s homebuyer

overall impacts presented in the main text and

education and counseling

impacts on additional exploratory outcomes

• We find no evidence that the overal impact of

not presented in the main text (exhibit C.1).

services is sensitive to the inclusion of the in-

The reason for this additional information is

person treatment group, which had a

comparatively low take-up rate, in the

that we expect that some readers will be

analysis.

interested not just in the main results (mean

outcome levels for treatment and control

groups and impact estimates) but also in some of the finer details such as sample sizes, standard errors, and post hoc minimum detectable effects, the added details of which we explain in section C.1. This appendix also presents alternative estimates of the overall impact of homebuyer education and counseling services when the in-person treatment group is excluded from the analysis (exhibit C.2).

C.1 Overall Impact (and How to Read the Impact Exhibits in This Appendix)

We begin by reviewing how to interpret the contents of exhibit C.1, as a model for how to interpret the elements of the impact tables provided across the rest of the appendixes, which contain parallel content organized in a similar way. Considering each column of exhibit C.1, from left to right:

• The Treatment Sample Size and Control Sample Size columns report the sample sizes for the treatment and control groups, respectively.

• The Treatment Group Mean and Control Group Mean columns report the regression-adjusted mean level of the outcome for the treatment and control groups, respectively.

• The difference between the treatment and control group means is the Impact of Being Offered Services, and is estimated using multiple regression, as described in appendix section A.1. This is the ITT impact. The corresponding standard error is reported in parentheses.

• The Percentage Impact, calculated as the impact divided by the control group mean, provides context for interpreting the relative magnitude of the treatment-control difference.

• The p-Value indicates how strong the evidence is in favor of rejecting the null hypothesis.

The smaller the p-value, the stronger the evidence that the null hypothesis should be rejected.

• In the Impact of Being Offered Services column, impacts marked with one or more asterisks are statistically significant, indicating that it is unlikely that the impact is due to chance. The number of asterisks indicates whether the impact is statistically significant at the The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 123

APPENDIX

p<0.10 level (*), p<0.05 level (**), or p<0.01 level (***) level. The more asterisks, the less likely the finding is due to chance.

• The 90-Percent Confidence Interval places bounds on the impact of being offered services.

Values that fall within the confidence interval are not statistically different from the estimated impact of having been offered services and are possible alternative estimates of the impact. Values outside the interval are statistically different from the impact.

• The Minimum Detectable Effect (MDE) is the smallest true intervention impact that can be detected with a given level of confidence. MDEs are helpful for understanding findings that are not statistically significant, because MDEs indicate how large the impact would have needed to be to be detected at a given level of confidence. In this application, we set the significance level to 10 percent and set statistical power to 80 percent. More detail on how MDEs are computed and interpreted appears in appendix section A.7.

• The Lower Bound of Impact of Taking Up Services column92 provides an estimate of the impact of taking up services, where service take-up is measured using administrative data.

This administrative measure either accurately represents (at 0 percent) or underestimates the share of control group members who received homebuyer education or counseling. As such, the estimate of the impact of taking up services, based on these administrative data, is either an accurate representation of or an underestimate of the actual impact of taking up services (as explained in appendix section A.2).

• The Upper Bound of Impact of Taking Up Services column provides an estimate of the impact of taking up services, where service take-up is measured using data from the Short-Term Follow-Up Survey. The survey-based measure of take-up likely overestimates the share of control group members who received homebuyer education or counseling similar to those offered to the treatment group (as described in appendix section A.2). As such, the estimate of the impact of taking up services, based on these self-reported survey data, is the upper bound of the actual impact of taking up services.

Similar to the Impact of Being Offered Services column, in the columns showing the Lower Bound and Upper Bound of Impact of Taking Up Services, impacts marked with one or more asterisks are statistically significant. The more asterisks, the less likely the finding is due to chance. As described in the textbox Calculating Impact Two Ways beginning on page 19, the ITT impact analysis and TOT impact analysis both yield the same pattern of results: the sign of the ITT and TOT estimates (that is, whether the impact is positive or negative, or favorable or 92 We refer to the TOT estimate computed by using the administrative data as a “lower-bound impact” although, as the discussion in appendix section A.2 highlights, it may also be an accurate representation of the true TOT impact, if indeed there are no control group crossovers.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 124

APPENDIX

unfavorable) will always be the same and the level of statistical significance of the ITT and TOT

estimates also will always be the same.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 125

APPENDIX

Exhibit C.1: Overal Impact of the Demonstration’s Homebuyer Education and Counseling, Expanded Results Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident that

2,537

2,006

74.7

69.8

4.9***

7.0%

0.004 ( 2.2, 7.5)

3.9

8.1***

16.0***

he/she could find the information

(1.6)

(2.6)

(5.1)

he/she needed about the

homebuying processa (%)

Number of lenders from which the

2,470

1,958

1.58

1.64

-0.05

-3.3%

0.136 ( -0.11, 0.01)

0.09

-0.09

-0.17

study participant received price

(0.03)

(0.06)

(0.11)

quotesa

Study participant purchased a

3,283

2,425

62.2

61.7

0.5

0.8%

0.629 ( -1.2, 2.2)

2.5

0.9

1.6

homeb (%)

(1.0)

(1.8)

(3.2)

Study participant was very satisfied

2,515

1,990

31.8

28.7

3.1**

10.7%

0.041 ( 0.6, 5.5)

3.6

5.1**

10.1**

with the homebuying processa (%)

(1.4)

(2.4)

(4.7)

Panel B: Financial Capability

Study participant knows how to

2,510

1,986

79.6

76.7

2.9***

3.8%

0.004 ( 1.3, 4.5)

2.3

4.8***

9.5***

correct inaccurate information in

(0.9)

(1.5)

(3.0)

credit reporta (%)

If in financial difficulty, study

2,443

1,925

21.3

16.9

4.4***

25.8%

0.001 ( 2.3, 6.4)

3.0

7.2***

14.0***

participant would contact his/her

(1.2)

(1.9)

(3.8)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

2,519

1,985

40.6

38.0

2.6

6.7%

0.121 ( -0.2, 5.3)

4.0

4.2

8.4

participant would contact his/her

(1.6)

(2.6)

(5.2)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

2,539

2,007

2.77

2.75

0.02

0.9%

0.340 ( -0.02, 0.07)

0.06

0.04

0.08

four) to mortgage literacy quiza

(0.02)

(0.04)

(0.08)

If study participant started having

2,527

1,993

82.1

82.7

-0.7

-0.8%

0.641 ( -3.1, 1.7)

3.5

-1.1

-2.2

financial problems and could not

(1.4)

(2.3)

(4.6)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 126

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has a budget and

2,529

1,995

34.0

35.8

-1.8

-5.0%

0.234 ( -4.3, 0.7)

3.7

-3.0

-5.9

often compares it against actual

(1.5)

(2.5)

(4.9)

spendinga (%)

Study participant usually pays

2,517

1,993

72.0

71.0

1.0

1.5%

0.398 ( -1.0, 3.1)

3.0

1.7

3.4

credit card balance in full to avoid

(1.2)

(2.0)

(3.9)

interest chargesa (%)

Regularly required mortgage

2,535

2,005

30.4

25.4

5.0***

19.6%

0.000 ( 3.0, 7.0)

2.9

8.3***

16.3***

payment is automatically deducted

(1.2)

(1.9)

(3.8)

from a bank accounta (%)

Credit score (out of 850)c

3,048

2,260

706.0

707.1

-1.0

-0.1%

0.479 ( -3.4, 1.4)

3.5

-1.9

-3.3

(1.4)

(2.6)

(4.6)

Study participant has a credit score

3,048

2,260

87.2

87.3

-0.1

-0.1%

0.844 ( -1.1, 0.9)

1.5

-0.2

-0.4

above or equal to 620c (%)

(0.6)

(1.1)

(2.0)

Student loan balancec ($)

3,073

2,273

10,392

9,843

549*

5.6%

0.067 ( 58, 1,039)

717

999*

1,784*

(288)

(524)

(936)

Credit card balancec ($)

3,073

2,273

3,850

3,791

58

1.5%

0.680 ( -180, 296)

348

106

190

(140)

(254)

(454)

Total nonhousing debtc ($)

3,073

2,273

24,506

23,715

791*

3.3%

0.072 ( 72, 1,510)

1,051

1,439*

2,570*

(422)

(768)

(1,372)

Total monthly debt-to-income ratio

3,043

2,261

26.6

25.6

1.0**

3.8%

0.027 ( 0.3, 1.7)

1.0

1.8**

3.1**

(back-end ratio)b

(0.4)

(0.8)

(1.3)

Total monthly debt-to-income ratio

3,043

2,261

17.0

16.1

0.9

5.6%

0.365 ( -0.8, 2.6)

2.4

1.6

2.9

(back-end ratio) greater than 0.43b

(1.0)

(1.8)

(3.1)

(%)

Study participant usually has

2,532

2,003

72.0

72.1

-0.0

-0.0%

0.981 ( -2.2, 2.2)

3.2

-0.1

-0.1

enough savings set aside to cover

(1.3)

(2.1)

(4.2)

3 months of expensesa (%)

Total savings and investmentsa ($)

2,404

1,891

50,216

49,220

996

2.0%

0.450 ( -1,216, 3,207)

3,233

1,628

3,130

(1,298)

(2,122)

(4,081)

Study participant occasionally does

2,535

2,003

10.9

9.2

1.7**

18.3%

0.050 ( 0.3, 3.1)

2.0

2.8**

5.5**

not have enough money to cover

(0.8)

(1.4)

(2.7)

all of the bil s at the end of the

montha (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 127

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Nonhousing debt bankruptcy,

3,089

2,280

5.1

4.6

0.4

9.6%

0.434 ( -0.5, 1.4)

1.4

0.8

1.5

foreclosure, or repossession

(0.6)

(1.0)

(1.8)

indicatorc (%)

Study participant has an electronic

2,531

1,998

54.7

56.4

-1.7

-3.1%

0.325 ( -4.7, 1.2)

4.3

-2.9

-5.7

or writ en budgeta (%)

(1.7)

(2.9)

(5.7)

Study participant never uses

2,457

1,947

85.6

86.8

-1.2

-1.4%

0.231 ( -2.8, 0.5)

2.4

-2.0

-3.8

payday lendersa (%)

(1.0)

(1.6)

(3.1)

Study participant has tried to figure

2,518

1,993

79.3

79.3

0.0

0.0%

0.998 ( -1.8, 1.8)

2.7

0.0

0.0

out how much he/she needs to

(1.1)

(1.8)

(3.5)

save for retirementa (%)

Study participant often saves

2,533

2,006

62.4

63.2

-0.8

-1.3%

0.523 ( -2.9, 1.3)

3.1

-1.3

-2.6

moneya (%)

(1.2)

(2.0)

(4.0)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

2,364

1,843

24.9

24.2

0.8

3.1%

0.240 ( -0.3, 1.8)

1.6

1.2

2.4

income ratioa

(0.6)

(1.0)

(2.0)

Monthly housing costs exceed 30

2,364

1,843

24.3

22.1

2.1*

9.6%

0.067 ( 0.2, 4.0)

2.8

3.5*

6.8*

percent of monthly incomea (%)

(1.1)

(1.8)

(3.6)

Ever 30 days delinquentb (%)

3,283

2,425

2.4

2.0

0.3

17.2%

0.307 ( -0.2, 0.9)

0.8

0.6

1.1

(0.3)

(0.6)

(1.1)

Ever 60 days delinquentb (%)

3,283

2,425

0.7

0.5

0.2

34.7%

0.417 ( -0.2, 0.5)

0.5

0.3

0.6

(0.2)

(0.4)

(0.7)

Ever 90 days delinquentb (%)

3,283

2,425

0.3

0.2

0.1

49.7%

0.452 ( -0.1, 0.3)

0.3

0.2

0.3

(0.1)

(0.2)

(0.4)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 128

APPENDIX

C.2 Alternative Estimates of Overal Impact

As described in chapter 3, the in-person treatment group took up services at a comparatively low rate. To determine whether the overall impact of homebuyer education and counseling services (as presented in chapters 5, 6, and 7 and appendix section C.1) is sensitive to the inclusion of the in-person treatment group, exhibit C.2 reports impact estimates for the subset of study participants assigned to the remote and choice treatment groups or the control group—that is, excluding the in-person treatment group.

We find no evidence that the estimates of the impact of being offered services presented in exhibit C.2 for this alternative sample are different from the corresponding impact estimates using the full study sample reported in exhibit C.1. For all outcomes, the impact of being offered services reported in exhibit C.1 is not statistically different from the corresponding estimate in exhibit C.2. 93 Additionally, we find that the standard errors are generally smaller when the in-person treatment group is included in the analysis, indicating that we have more precise estimates when we based the analysis on a larger sample, further motivating the use of the full sample when estimating the overall impact of homebuyer education and counseling services.

93 We do not necessarily expect the impact estimates to be the same when the full sample is included and when the in-person treatment group is excluded from the analysis. Excluding the in-person treatment group from the sample results in a lower share of the treatment group that was offered in-person services and a greater share that was offered remote services. To the extent that the offer of in-person services has a different impact from the offer of remote services, this sample exclusion could lead to different impacts.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 129

APPENDIX

Exhibit C.2: Overal Impact of the Demonstration’s Homebuyer Education and Counseling, Excluding the In-Person Treatment Group Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident

2,120

2,006

75.3

69.8

5.5***

7.9%

0.003 ( 2.7, 8.4)

4.2

8.2***

16.8***

that he/she could find the

(1.7)

(2.5)

(5.1)

information he/she needed about

the homebuying processa (%)

Number of lenders from which the

2,066

1,958

1.59

1.64

-0.05

-2.8%

0.258 ( -0.11, 0.02)

0.10

-0.07

-0.14

study participant received price

(0.04)

(0.06)

(0.12)

quotesa

Study participant purchased a

2,772

2,425

62.2

61.7

0.5

0.8%

0.662 ( -1.4, 2.3)

2.7

0.8

1.4

homeb (%)

(1.1)

(1.8)

(3.2)

Study participant was very

2,100

1,990

31.3

28.7

2.6

9.0%

0.104 ( -0.0, 5.2)

3.8

3.8

7.9

satisfied with the homebuying

(1.5)

(2.3)

(4.7)

processa (%)

Panel B: Financial Capability

Study participant knows how to

2,092

1,986

80.3

76.7

3.7***

4.8%

0.001 ( 1.9, 5.4)

2.5

5.4***

11.1***

correct inaccurate information in

(1.0)

(1.5)

(3.1)

credit reporta (%)

If in financial difficulty, study

2,047

1,925

21.9

16.9

5.0***

29.3%

0.001 ( 2.8, 7.2)

3.2

7.4***

14.8***

participant would contact his/her

(1.3)

(1.9)

(3.9)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

2,105

1,985

41.0

38.0

3.0*

8.0%

0.056 ( 0.4, 5.6)

3.8

4.5*

9.2*

participant would contact his/her

(1.5)

(2.3)

(4.6)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

2,120

2,007

2.76

2.75

0.01

0.4%

0.672 ( -0.03, 0.05)

0.06

0.02

0.03

four) to mortgage literacy quiza

(0.03)

(0.04)

(0.08)

If study participant started having

2,110

1,993

81.9

82.7

-0.8

-1.0%

0.501 ( -2.9, 1.2)

3.0

-1.2

-2.5

financial problems and could not

(1.2)

(1.8)

(3.7)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 130

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has a budget and

2,111

1,995

33.9

35.8

-1.9

-5.4%

0.221 ( -4.6, 0.7)

3.8

-2.9

-5.9

often compares it against actual

(1.5)

(2.3)

(4.7)

spendinga (%)

Study participant usually pays

2,101

1,993

72.4

71.0

1.4

2.0%

0.261 ( -0.7, 3.6)

3.1

2.2

4.4

credit card balance in full to avoid

(1.3)

(1.9)

(3.8)

interest chargesa (%)

Regularly required mortgage

2,117

2,005

30.9

25.4

5.5***

21.5%

0.000 ( 3.3, 7.6)

3.1

8.1***

16.6***

payment is automatically deducted

(1.3)

(1.9)

(3.8)

from a bank accounta (%)

Credit score (out of 850)c

2,577

2,260

705.1

707.1

-1.9

-0.3%

0.244 ( -4.7, 0.8)

4.1

-3.2

-5.8

(1.6)

(2.7)

(4.9)

Study participant has a credit

2,577

2,260

86.7

87.3

-0.6

-0.7%

0.404 ( -1.8, 0.6)

1.8

-1.0

-1.8

score above or equal to 620c (%)

(0.7)

(1.2)

(2.1)

Student loan balancec ($)

2,600

2,273

10,484

9,843

641**

6.5%

0.037 ( 143, 1,138)

727

1,053**

1,920**

(292)

(480)

(875)

Credit card balancec ($)

2,600

2,273

3,900

3,791

109

2.9%

0.501 ( -163, 381)

398

179

327

(160)

(263)

(479)

Total nonhousing debtc ($)

2,600

2,273

24,584

23,715

869*

3.7%

0.098 ( 6, 1,731)

1,261

1,428*

2,603*

(507)

(833)

(1,518)

Total monthly debt-to-income ratio

2,569

2,261

26.6

25.6

1.0**

3.9%

0.033 ( 0.2, 1.8)

1.1

1.6**

3.0**

(back-end ratio)b

(0.4)

(0.7)

(1.3)

Total monthly debt-to-income ratio

2,569

2,261

17.1

16.1

1.0

6.3%

0.399 ( -1.0, 3.0)

3.0

1.7

3.0

(back-end ratio) greater than 0.43b

(1.2)

(1.9)

(3.5)

(%)

Study participant usually has

2,115

2,003

73.1

72.1

1.0

1.4%

0.502 ( -1.5, 3.5)

3.6

1.5

3.0

enough savings set aside to cover

(1.5)

(2.2)

(4.5)

3 months of expensesa (%)

Total savings and investmentsa ($)

2,009

1,891

50,079

49,220

858

1.7%

0.576 ( -1,724, 3,441)

3,776

1,265

2,513

(1,516)

(2,235)

(4,439)

Study participant occasionally

2,116

2,003

10.8

9.2

1.6*

17.5%

0.059 ( 0.2, 3.0)

2.0

2.4*

4.9*

does not have enough money to

(0.8)

(1.2)

(2.5)

cover all of the bil s at the end of

the montha (%)

Nonhousing debt bankruptcy,

2,610

2,280

5.2

4.6

0.5

11.0%

0.347 ( -0.4, 1.4)

1.3

0.8

1.5

foreclosure, or repossession

(0.5)

(0.9)

(1.6)

indicatorc (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 131

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has an electronic

2,113

1,998

54.2

56.4

-2.2

-3.9%

0.260 ( -5.4, 1.0)

4.7

-3.3

-6.6

or writ en budgeta (%)

(1.9)

(2.8)

(5.8)

Study participant never uses

2,049

1,947

86.0

86.8

-0.8

-0.9%

0.497 ( -2.8, 1.2)

2.9

-1.2

-2.4

payday lendersa (%)

(1.2)

(1.7)

(3.5)

Study participant has tried to

2,105

1,993

79.2

79.3

-0.2

-0.2%

0.886 ( -2.1, 1.8)

2.9

-0.3

-0.5

figure out how much he/she needs

(1.2)

(1.7)

(3.5)

to save for retirementa (%)

Study participant often saves

2,116

2,006

62.9

63.2

-0.3

-0.5%

0.814 ( -2.6, 1.9)

3.3

-0.5

-0.9

moneya (%)

(1.3)

(2.0)

(4.0)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

1,981

1,843

24.6

24.2

0.4

1.8%

0.545 ( -0.8, 1.6)

1.8

0.6

1.3

income ratioa

(0.7)

(1.1)

(2.1)

Monthly housing costs exceed 30

1,981

1,843

23.2

22.1

1.1

4.8%

0.447 ( -1.3, 3.4)

3.5

1.6

3.2

percent of monthly incomea (%)

(1.4)

(2.1)

(4.1)

Ever 30 days delinquentb (%)

2,772

2,425

2.4

2.0

0.3

16.6%

0.376 ( -0.3, 1.0)

0.9

0.5

1.0

(0.4)

(0.6)

(1.1)

Ever 60 days delinquentb (%)

2,772

2,425

0.7

0.5

0.2

34.9%

0.424 ( -0.2, 0.5)

0.5

0.3

0.5

(0.2)

(0.3)

(0.6)

Ever 90 days delinquentb (%)

2,772

2,425

0.2

0.2

0.0

2.1%

0.977 ( -0.2, 0.3)

0.4

0.0

0.0

(0.1)

(0.2)

(0.4)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. For this analysis, the treatment group includes the choice and remote treatment groups (N=2,807), and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 132

APPENDIX

Appendix D: Expanded Results for Impacts by Service Delivery Mode

This appendix presents expanded results and

alternative estimates of the impact of

Key Findings: Expanded Results for

homebuyer education and counseling services

Impacts by Service Delivery Mode

by service delivery mode: Section D.1 presents

This appendix presents expanded results and

results separately for each service delivery

alternative estimates of the impact of homebuyer

education and counseling services by service

mode: in-person services, remote services, and

delivery mode.

choice of either in-person or remote services.

• We find no evidence that the in-person and

Section D.2. compares the impact of in-person

remote impacts presented in the main text are

sensitive to the inclusion of the choice

and remote services. Section D.3 compares the

treatment group: mode effects are statistically

impact of choice of services modes with the

similar when choice treatment group members

are omitted from the analysis.

impact of remote services. Section D.4

presents the impact of in-person services and

• We do not generally find evidence that the

impact on those given a choice of service

remote services for alternative samples.

modes is different from the impact on those

offered remote services without a choice.

D.1 Expanded Results by Service Delivery Mode: In-Person, Remote, and

Choice

This section presents expanded results for the impacts of in-person services and the impact of remote services, including additional information related to the mode effects presented in the main text and impacts on additional exploratory outcomes not presented in the main text. This section also presents the impacts of being offered a choice of in-person or remote services.

Exhibit D.1 presents expanded results for the impact of in-person homebuyer education and counseling. Exhibit D.2 presents expanded results for the impact of remote homebuyer education and counseling. Exhibit D.3 presents the impact of choice of in-person or remote homebuyer education and counseling.

For an explanation of how to read the exhibits, see appendix section C.1.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 133

APPENDIX

Exhibit D.1: Impact of In-Person Homebuyer Education and Counseling, Expanded Results Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident

640

990

70.4

69.1

1.3

1.9%

0.585 ( -2.8, 5.4)

6.0

4.2

6.3

that he/she could find the

(2.4)

(7.7)

(11.4)

information he/she needed about

the homebuying processa (%)

Number of lenders from which the

625

962

1.43

1.53

-0.11*

-6.9%

0.067 ( -0.20, -0.01)

0.14

-0.33*

-0.48*

study participant received price

(0.06)

(0.17)

(0.25)

quotesa

Study participant purchased a

800

1,173

58.9

57.7

1.2

2.1%

0.571 ( -2.4, 4.9)

5.3

4.4

5.8

homeb (%)

(2.1)

(7.7)

(10.2)

Study participant was very satisfied

636

981

35.0

26.1

8.9***

34.2%

0.001 ( 5.1, 12.8)

5.7

28.5***

42.5***

with the homebuying processa (%)

(2.3)

(7.3)

(10.8)

Panel B: Financial Capability

Study participant knows how to

636

973

75.1

76.3

-1.3

-1.7%

0.502 ( -4.4, 1.9)

4.6

-4.1

-6.0

correct inaccurate information in

(1.9)

(6.0)

(8.8)

credit reporta (%)

If in financial difficulty, study

613

943

18.4

17.7

0.7

3.9%

0.729 ( -2.6, 4.0)

4.9

2.2

3.2

participant would contact his/her

(2.0)

(6.2)

(9.2)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

635

977

37.1

35.8

1.3

3.5%

0.655 ( -3.5, 6.0)

6.9

4.0

5.9

participant would contact his/her

(2.8)

(8.8)

(13.1)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

642

990

2.78

2.69

0.08*

3.1%

0.054 ( 0.01, 0.16)

0.10

0.27*

0.40*

four) to mortgage literacy quiza

(0.04)

(0.13)

(0.20)

If study participant started having

640

984

84.7

82.6

2.0

2.4%

0.422 ( -2.2, 6.2)

6.2

6.4

9.5

financial problems and could not

(2.5)

(7.9)

(11.7)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

Study participant has a budget and

640

982

34.2

36.8

-2.6

-7.1%

0.217 ( -6.1, 0.9)

5.1

-8.3

-12.4

often compares it against actual

(2.1)

(6.6)

(9.8)

spendinga (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 134

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant usually pays

635

980

68.8

69.1

-0.3

-0.5%

0.852 ( -3.2, 2.5)

4.2

-1.0

-1.5

credit card balance in full to avoid

(1.7)

(5.3)

(7.8)

interest chargesa (%)

Regularly required mortgage

640

989

26.6

24.1

2.5

10.4%

0.260 ( -1.2, 6.2)

5.4

8.0

11.8

payment is automatically deducted

(2.2)

(7.0)

(10.3)

from a bank accounta (%)

Credit score (out of 850)c

736

1,081

700.2

697.3

2.9

0.4%

0.207 ( -0.9, 6.8)

5.6

10.5

13.7

(2.3)

(8.1)

(10.6)

Study participant has a credit score

736

1,081

86.7

84.5

2.2*

2.6%

0.051 ( 0.4, 4.1)

2.7

7.9*

10.4*

above or equal to 620c (%)

(1.1)

(3.9)

(5.1)

Student loan balancec ($)

740

1,091

8,977

9,272

-295

-3.2%

0.567 ( -1,163, 573)

1,269

-1,059

-1,418

(509)

(1,827)

(2,447)

Credit card balancec ($)

740

1,091

3,416

3,488

-71

-2.0%

0.751 ( -450, 307)

554

-256

-342

(222)

(797)

(1,068)

Total nonhousing debtc ($)

740

1,091

23,111

23,139

-28

-0.1%

0.967 ( -1,165, 1,109)

1,663

-101

-135

(668)

(2,394)

(3,207)

Total monthly debt-to-income ratio

728

1,075

26.4

25.2

1.3

5.0%

0.141 ( -0.2, 2.7)

2.1

4.5

5.7

(back-end ratio)b

(0.8)

(3.0)

(3.8)

Total monthly debt-to-income ratio

728

1,075

16.5

15.1

1.5

9.7%

0.242 ( -0.6, 3.5)

3.0

5.2

6.7

(back-end ratio) greater than 0.43b

(1.2)

(4.4)

(5.6)

(%)

Study participant usually has

638

988

68.7

69.8

-1.2

-1.7%

0.555 ( -4.4, 2.1)

4.8

-3.7

-5.5

enough savings set aside to cover

(1.9)

(6.2)

(9.3)

3 months of expensesa (%)

Total savings and investmentsa ($)

604

928

47,952

44,617

3,335

7.5%

0.168 ( -674, 7,344)

5,861

10,732

15,630

(2,354)

(7,575)

(11,028)

Study participant occasionally

641

988

12.8

10.8

1.9

17.9%

0.176 ( -0.4, 4.3)

3.5

6.2

9.3

does not have enough money to

(1.4)

(4.5)

(6.7)

cover all of the bil s at the end of

the montha (%)

Nonhousing debt bankruptcy,

746

1,097

5.7

5.0

0.7

14.0%

0.609 ( -1.6, 3.0)

3.4

2.5

3.4

foreclosure, or repossession

(1.4)

(4.9)

(6.6)

indicatorc (%)

Study participant has an electronic

640

984

54.9

57.2

-2.3

-4.0%

0.436 ( -7.3, 2.7)

7.3

-7.4

-11.0

or writ en budgeta (%)

(2.9)

(9.4)

(13.9)

Study participant never uses

625

958

84.1

84.4

-0.3

-0.3%

0.885 ( -3.6, 3.1)

4.9

-0.9

-1.3

payday lendersa (%)

(2.0)

(6.2)

(9.1)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 135

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has tried to figure

636

985

81.0

80.3

0.7

0.9%

0.720 ( -2.6, 4.1)

4.9

2.3

3.4

out how much he/she needs to

(2.0)

(6.3)

(9.3)

save for retirementa (%)

Study participant often saves

640

990

59.9

61.4

-1.5

-2.5%

0.496 ( -5.2, 2.2)

5.4

-4.8

-7.1

moneya (%)

(2.2)

(7.0)

(10.3)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

588

906

26.5

25.0

1.4

5.7%

0.108 ( -0.0, 2.9)

2.1

4.5

6.6

income ratioa

(0.9)

(2.7)

(3.9)

Monthly housing costs exceed 30

588

906

29.6

23.5

6.1***

26.0%

0.006 ( 2.6, 9.6)

5.1

19.2***

28.1***

percent of monthly incomea (%)

(2.0)

(6.4)

(9.4)

Ever 30 days delinquentb (%)

800

1,173

3.4

2.4

1.0

41.3%

0.117 ( -0.1, 2.0)

1.5

3.5

4.7

(0.6)

(2.2)

(2.9)

Ever 60 days delinquentb (%)

800

1,173

1.0

0.7

0.3

44.7%

0.531 ( -0.5, 1.1)

1.2

1.1

1.4

(0.5)

(1.7)

(2.3)

Ever 90 days delinquentb (%)

800

1,173

0.6

0.3

0.3

80.6%

0.389 ( -0.3, 0.8)

0.8

1.0

1.3

(0.3)

(1.1)

(1.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. For this analysis, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806), and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n=1,184). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 136

APPENDIX

Exhibit D.2: Impact of Remote Homebuyer Education and Counseling, Expanded Results Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident

1,897

2,006

75.3

69.8

5.6***

8.0%

0.003 ( 2.7, 8.5)

4.3

8.0***

16.6***

that he/she could find the

(1.7)

(2.4)

(5.1)

information he/she needed about

the homebuying processa (%)

Number of lenders from which the

1,845

1,958

1.60

1.64

-0.04

-2.3%

0.385 ( -0.11, 0.04)

0.11

-0.05

-0.11

study participant received price

(0.04)

(0.06)

(0.13)

quotesa

Study participant purchased a

2,483

2,425

61.9

61.7

0.1

0.2%

0.899 ( -1.7, 2.0)

2.7

0.2

0.4

homeb (%)

(1.1)

(1.7)

(3.2)

Study participant was very

1,879

1,990

30.3

28.7

1.5

5.4%

0.292 ( -0.9, 4.0)

3.6

2.2

4.6

satisfied with the homebuying

(1.4)

(2.1)

(4.3)

processa (%)

Panel B: Financial Capability

Study participant knows how to

1,874

1,986

80.7

76.7

4.1***

5.3%

0.000 ( 2.4, 5.8)

2.5

5.8***

12.2***

correct inaccurate information in

(1.0)

(1.4)

(2.9)

credit reporta (%)

If in financial difficulty, study

1,830

1,925

22.1

16.9

5.2***

30.6%

0.001 ( 2.7, 7.6)

3.6

7.3***

15.2***

participant would contact his/her

(1.4)

(2.0)

(4.2)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

1,884

1,985

41.0

38.0

3.0*

7.9%

0.085 ( 0.1, 5.8)

4.2

4.3*

8.9*

participant would contact his/her

(1.7)

(2.4)

(5.0)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

1,897

2,007

2.75

2.75

0.01

0.3%

0.730 ( -0.03, 0.05)

0.06

0.01

0.03

four) to mortgage literacy quiza

(0.03)

(0.04)

(0.08)

If study participant started having

1,887

1,993

81.7

82.7

-1.0

-1.3%

0.436 ( -3.3, 1.2)

3.3

-1.5

-3.1

financial problems and could not

(1.3)

(1.9)

(3.9)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

Study participant has a budget and

1,889

1,995

34.0

35.8

-1.8

-4.9%

0.283 ( -4.5, 1.0)

4.0

-2.5

-5.3

often compares it against actual

(1.6)

(2.3)

(4.8)

spendinga (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 137

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant usually pays

1,882

1,993

72.6

71.0

1.6

2.2%

0.265 ( -0.8, 4.0)

3.5

2.3

4.7

credit card balance in full to avoid

(1.4)

(2.0)

(4.2)

interest chargesa (%)

Regularly required mortgage

1,895

2,005

30.9

25.4

5.5***

21.5%

0.000 ( 3.3, 7.6)

3.2

7.8***

16.3***

payment is automatically deducted

(1.3)

(1.8)

(3.8)

from a bank accounta (%)

Credit score (out of 850)c

2,312

2,260

704.8

707.1

-2.3

-0.3%

0.177 ( -5.1, 0.5)

4.1

-3.6

-6.7

(1.6)

(2.6)

(4.8)

Study participant has a credit

2,312

2,260

86.4

87.3

-0.9

-1.0%

0.233 ( -2.1, 0.3)

1.8

-1.4

-2.5

score above or equal to 620c (%)

(0.7)

(1.1)

(2.1)

Student loan balancec ($)

2,333

2,273

10,622

9,843

779**

7.9%

0.025 ( 219, 1,339)

819

1,225**

2,291**

(329)

(517)

(967)

Credit card balancec ($)

2,333

2,273

3,891

3,791

100

2.6%

0.561 ( -189, 389)

422

157

294

(170)

(267)

(499)

Total nonhousing debtc ($)

2,333

2,273

24,686

23,715

971*

4.1%

0.080 ( 63, 1,879)

1,327

1,526*

2,856*

(533)

(838)

(1,568)

Total monthly debt-to-income ratio

2,315

2,261

26.4

25.6

0.9*

3.3%

0.079 ( 0.1, 1.7)

1.2

1.3*

2.5*

(back-end ratio)b

(0.5)

(0.7)

(1.4)

Total monthly debt-to-income ratio

2,315

2,261

16.7

16.1

0.7

4.3%

0.559 ( -1.3, 2.7)

2.9

1.1

2.0

(back-end ratio) greater than 0.43b

(1.2)

(1.8)

(3.4)

(%)

Study participant usually has

1,894

2,003

72.3

72.1

0.2

0.3%

0.896 ( -2.3, 2.6)

3.6

0.3

0.6

enough savings set aside to cover

(1.4)

(2.1)

(4.3)

3 months of expensesa (%)

Total savings and investmentsa ($)

1,800

1,891

49,542

49,220

322

0.7%

0.840 ( -2,364, 3,008)

3,927

454

925

(1,577)

(2,222)

(4,525)

Study participant occasionally

1,894

2,003

10.8

9.2

1.6

17.3%

0.111 ( -0.1, 3.2)

2.4

2.3

4.7

does not have enough money to

(1.0)

(1.4)

(2.9)

cover all of the bil s at the end of

the montha (%)

Nonhousing debt bankruptcy,

2,343

2,280

5.0

4.6

0.4

8.6%

0.453 ( -0.5, 1.3)

1.3

0.6

1.2

foreclosure, or repossession

(0.5)

(0.8)

(1.5)

indicatorc (%)

Study participant has an electronic

1,891

1,998

54.6

56.4

-1.8

-3.2%

0.367 ( -5.1, 1.5)

4.9

-2.6

-5.4

or writ en budgeta (%)

(2.0)

(2.8)

(5.8)

Study participant never uses

1,832

1,947

85.6

86.8

-1.2

-1.4%

0.302 ( -3.2, 0.8)

2.9

-1.7

-3.6

payday lendersa (%)

(1.2)

(1.7)

(3.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 138

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has tried to figure

1,882

1,993

79.0

79.3

-0.3

-0.4%

0.796 ( -2.3, 1.7)

2.9

-0.4

-0.9

out how much he/she needs to

(1.2)

(1.7)

(3.5)

save for retirementa (%)

Study participant often saves

1,893

2,006

62.6

63.2

-0.6

-0.9%

0.672 ( -2.9, 1.7)

3.4

-0.8

-1.7

moneya (%)

(1.4)

(2.0)

(4.1)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

1,776

1,843

24.6

24.2

0.4

1.7%

0.569 ( -0.8, 1.6)

1.7

0.6

1.2

income ratioa

(0.7)

(1.0)

(2.1)

Monthly housing costs exceed 30

1,776

1,843

22.9

22.1

0.7

3.3%

0.599 ( -1.6, 3.1)

3.4

1.0

2.1

percent of monthly incomea (%)

(1.4)

(2.0)

(4.0)

Ever 30 days delinquentb (%)

2,483

2,425

2.2

2.0

0.2

9.2%

0.607 ( -0.4, 0.8)

0.9

0.3

0.5

(0.4)

(0.6)

(1.1)

Ever 60 days delinquentb (%)

2,483

2,425

0.7

0.5

0.2

35.6%

0.374 ( -0.2, 0.5)

0.5

0.3

0.5

(0.2)

(0.3)

(0.6)

Ever 90 days delinquentb (%)

2,483

2,425

0.3

0.2

0.0

22.6%

0.770 ( -0.2, 0.3)

0.4

0.1

0.1

(0.2)

(0.2)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. For this analysis, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 139

APPENDIX

Exhibit D.3: Impact of Choice of In-Person or Remote Homebuyer Education and Counseling Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident

867

1,316

76.8

68.7

8.1***

11.7%

0.002 ( 4.0, 12.2)

6.0

13.2***

24.8***

that he/she could find the

(2.4)

(3.9)

(7.4)

information he/she needed about

the homebuying processa (%)

Number of lenders from which the

853

1,289

1.60

1.66

-0.06

-3.7%

0.244 ( -0.15, 0.03)

0.13

-0.10

-0.18

study participant received price

(0.05)

(0.08)

(0.15)

quotesa

Study participant purchased a

1,130

1,641

61.9

61.1

0.8

1.3%

0.602 ( -1.8, 3.4)

3.8

1.4

2.4

homeb (%)

(1.5)

(2.7)

(4.6)

Study participant was very

860

1,307

33.2

28.6

4.5*

15.8%

0.070 ( 0.4, 8.6)

6.0

7.4*

14.0*

satisfied with the homebuying

(2.4)

(3.9)

(7.4)

processa (%)

Panel B: Financial Capability

Study participant knows how to

856

1,305

80.5

75.9

4.6***

6.0%

0.007 ( 1.9, 7.3)

3.9

7.5***

14.0***

correct inaccurate information in

(1.6)

(2.6)

(4.8)

credit reporta (%)

If in financial difficulty, study

839

1,273

22.0

15.8

6.2***

38.9%

0.002 ( 3.1, 9.3)

4.5

10.0***

18.6***

participant would contact his/her

(1.8)

(3.0)

(5.5)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

865

1,306

40.3

36.8

3.5**

9.5%

0.040 ( 0.7, 6.3)

4.0

5.7**

10.8**

participant would contact his/her

(1.6)

(2.7)

(5.0)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

867

1,317

2.72

2.75

-0.03

-1.0%

0.546 ( -0.11, 0.05)

0.12

-0.05

-0.09

four) to mortgage literacy quiza

(0.05)

(0.08)

(0.14)

If study participant started having

865

1,308

82.1

83.1

-1.0

-1.2%

0.543 ( -3.8, 1.8)

4.0

-1.6

-3.1

financial problems and could not

(1.6)

(2.7)

(5.0)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

Study participant has a budget and

865

1,309

32.5

34.8

-2.3

-6.6%

0.213 ( -5.4, 0.8)

4.5

-3.8

-7.1

often compares it against actual

(1.8)

(2.9)

(5.5)

spendinga (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 140

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant usually pays

861

1,308

74.8

73.1

1.7

2.4%

0.213 ( -0.6, 4.1)

3.4

2.8

5.3

credit card balance in full to avoid

(1.4)

(2.2)

(4.2)

interest chargesa (%)

Regularly required mortgage

866

1,316

31.3

23.6

7.7***

32.8%

0.000 ( 4.9, 10.5)

4.1

12.6***

23.8***

payment is automatically deducted

(1.6)

(2.7)

(5.1)

from a bank accounta (%)

Credit score (out of 850)c

1,054

1,531

710.2

710.6

-0.4

-0.1%

0.842 ( -3.7, 2.9)

4.9

-0.7

-1.2

(2.0)

(3.5)

(5.9)

Study participant has a credit

1,054

1,531

89.0

88.6

0.3

0.4%

0.771 ( -1.6, 2.3)

2.9

0.6

1.0

score above or equal to 620c (%)

(1.2)

(2.1)

(3.5)

Student loan balancec ($)

1,063

1,537

10,313

9,585

728

7.6%

0.162 ( -134, 1,589)

1,260

1,287

2,194

(506)

(895)

(1,526)

Credit card balancec ($)

1,063

1,537

3,815

3,888

-73

-1.9%

0.710 ( -403, 257)

483

-129

-220

(194)

(343)

(585)

Total nonhousing debtc ($)

1,063

1,537

24,027

23,510

517

2.2%

0.489 ( -738, 1,772)

1,834

914

1,560

(737)

(1,303)

(2,222)

Total monthly debt-to-income ratio

1,042

1,535

27.1

25.4

1.7***

6.8%

0.002 ( 0.9, 2.6)

1.3

3.1***

5.1***

(back-end ratio)b

(0.5)

(0.9)

(1.5)

Total monthly debt-to-income ratio

1,042

1,535

19.1

16.4

2.7*

16.3%

0.084 ( 0.1, 5.2)

3.7

4.7*

7.9*

(back-end ratio) greater than 0.43b

(1.5)

(2.6)

(4.4)

(%)

Study participant usually has

864

1,313

78.2

73.7

4.5**

6.1%

0.023 ( 1.3, 7.6)

4.6

7.3**

13.7**

enough savings set aside to cover

(1.8)

(3.0)

(5.7)

3 months of expensesa (%)

Total savings and investmentsa ($)

816

1,242

50,136

48,759

1,378

2.8%

0.438 ( -1,605, 4,360)

4,360

2,245

4,095

(1,751)

(2,853)

(5,204)

Study participant occasionally

865

1,313

10.9

8.7

2.2*

25.2%

0.066 ( 0.2, 4.1)

2.8

3.6*

6.7*

does not have enough money to

(1.1)

(1.9)

(3.5)

cover all of the bil s at the end of

the montha (%)

Nonhousing debt bankruptcy,

1,064

1,538

5.5

4.6

0.9

20.8%

0.293 ( -0.6, 2.5)

2.2

1.7

2.9

foreclosure, or repossession

(0.9)

(1.6)

(2.7)

indicatorc (%)

Study participant has an electronic

865

1,311

52.8

54.6

-1.8

-3.2%

0.592 ( -7.3, 3.8)

8.1

-2.9

-5.5

or writ en budgeta (%)

(3.3)

(5.3)

(10.1)

Study participant never uses

838

1,281

86.9

87.8

-0.9

-1.0%

0.599 ( -3.6, 1.9)

4.0

-1.4

-2.6

payday lendersa (%)

(1.6)

(2.6)

(4.9)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 141

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has tried to figure

865

1,306

79.3

78.8

0.5

0.7%

0.791 ( -2.8, 3.9)

4.9

0.9

1.6

out how much he/she needs to

(2.0)

(3.2)

(6.1)

save for retirementa (%)

Study participant often saves

866

1,316

64.1

65.0

-0.9

-1.3%

0.724 ( -5.0, 3.3)

6.0

-1.4

-2.6

moneya (%)

(2.4)

(3.9)

(7.4)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

813

1,207

24.8

23.8

1.0

4.4%

0.143 ( -0.1, 2.2)

1.7

1.7

3.2

income ratioa

(0.7)

(1.1)

(2.1)

Monthly housing costs exceed 30

813

1,207

23.4

21.7

1.7

7.6%

0.345 ( -1.3, 4.6)

4.3

2.7

5.0

percent of monthly incomea (%)

(1.7)

(2.8)

(5.2)

Ever 30 days delinquentb (%)

1,130

1,641

2.2

1.7

0.5

27.4%

0.416 ( -0.5, 1.4)

1.4

0.8

1.4

(0.6)

(1.0)

(1.7)

Ever 60 days delinquentb (%)

1,130

1,641

0.4

0.4

0.1

15.5%

0.838 ( -0.4, 0.5)

0.7

0.1

0.2

(0.3)

(0.5)

(0.8)

Ever 90 days delinquentb (%)

1,130

1,641

0.1

0.1

-0.1

-43.0%

0.729 ( -0.3, 0.2)

0.4

-0.1

-0.2

(0.1)

(0.3)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. For this analysis, the treatment group includes the choice treatment group (n=1,142); and the control group includes the Modified Study Design control group (n=1,656). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 142

APPENDIX

D.2 Comparison of Impact of Taking Up In-Person Services with Impact of

Taking Up Remote Services

Section D.1 presented results separately for each delivery mode. In chapters 5, 6, and 7, we compared the impact of being offered in-person services with the impact of being offered remote services. As a supplement to these findings, exhibit D.4 compares the impact of taking up in-person services with the impact of taking up remote services. For this computation, we use the study’s preferred approach to estimating the TOT impact, which relies on administrative data and makes an adjustment for no-shows, assuming both that there are no impacts on no-shows and that there are no control group crossovers (see appendix section A.2 for an explanation of this approach and why it was chosen).

The rates at which treatment group members took up services differ meaningfully between the in-person and remote service modes, with take-up rates much higher for those offered remote services than for those offered in-person services. Almost two-thirds of those offered remote services (63.8 percent) took up online education or telephone counseling. In contrast, about one-fourth (28.0 percent) of those offered in-person services took up an in-person education workshop or in-person counseling. These mode-specific take-up rates imply that the impact of taking up remote services is about 1.6 times larger than the impact of being offered remote services, and that the impact of taking up in-person services is about 3.6 times larger than the impact of being offered in-person services.

As described in the textbox Calculating Impact Two Ways beginning on page 19, the impact of being offered services (the ITT impact) and the impact of taking up services (the TOT impact) both yield the same pattern of results: the sign of the ITT and TOT estimates (that is, whether the impact is positive or negative, or favorable or unfavorable) will always be the same and the level of statistical significance of the ITT and TOT estimates also will always be the same. Therefore, we refer readers to the main text, which presents the ITT estimates for in-person and remote services, for a discussion of the general pattern of findings related to mode effects.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 143

APPENDIX

Exhibit D.4: Comparison of Impact of Taking Up In-Person Services with Impact of Taking Up Remote Services

Impact of

Impact of Difference in

Taking Up

Taking Up the Impact of

In-Person

Remote

Taking Up

Outcome

Services

Services

Services

Panel A: Preparedness and Search

Study participant was confident that he/she could find the information

4.2

8.0***

-3.7

he/she needed about the homebuying processa (%)

Number of lenders from which the study participant received price quotesa

-0.33*

-0.05

-0.28

Study participant purchased a homeb (%)

4.4

0.2

4.2

Study participant was very satisfied with the homebuying processa (%)

28.5***

2.2

26.3***

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in credit

-4.1

5.8***

-9.9

reporta (%)

If in financial difficulty, study participant would contact his/her counseling

2.2

7.3***

-5.2

agency, consumer credit counseling agency, or other nonprofit

organization for assistance prior to missing a mortgage paymenta (%)

If in financial difficulty, study participant would contact his/her lender for

4.0

4.3*

-0.3

assistance prior to missing a mortgage paymenta (%)

Number of correct answers (out of four) to mortgage literacy quiza

0.27*

0.01

0.26*

If study participant started having financial problems and could not pay all

6.4

-1.5

7.9

of the bil s, he/she would pay the mortgage firsta (%)

Study participant has a budget and often compares it against actual

-8.3

-2.5

-5.8

spendinga (%)

Study participant usually pays credit card balance in full to avoid interest

-1.0

2.3

-3.3

chargesa (%)

Regularly required mortgage payment is automatically deducted from a

8.0

7.8***

0.2

bank accounta (%)

Credit score (out of 850)c

10.5

-3.6

14.0*

Study participant has a credit score above or equal to 620c (%)

7.9*

-1.4

9.3**

Student loan balancec ($)

-1,059

1,225**

-2,283

Credit card balancec ($)

-256

157

-413

Total nonhousing debtc ($)

-101

1,526*

-1,627

Total monthly debt-to-income ratio (back-end ratio)b

4.5

1.3*

3.1

Total monthly debt-to-income ratio (back-end ratio) greater than 0.43b (%)

5.2

1.1

4.1

Study participant usually has enough savings set aside to cover 3 months

-3.7

0.3

-4.0

of expensesa (%)

Total savings and investmentsa ($)

10,732

454

10,278

Study participant occasional y does not have enough money to cover all of

6.2

2.3

4.0

the bil s at the end of the montha (%)

Nonhousing debt bankruptcy, foreclosure, or repossession indicatorc (%)

2.5

0.6

1.9

Study participant has an electronic or written budgeta (%)

-7.4

-2.6

-4.8

Study participant never uses payday lendersa (%)

-0.9

-1.7

0.8

Study participant has tried to figure out how much he/she needs to save

2.3

-0.4

2.7

for retirementa (%)

Study participant often saves moneya (%)

-4.8

-0.8

-4.0

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

4.5

0.6

3.9

Monthly housing costs exceed 30 percent of monthly incomea (%)

19.2***

1.0

18.1***

Ever 30 days delinquentb (%)

3.5

0.3

3.2

Ever 60 days delinquentb (%)

1.1

0.3

0.8

Ever 90 days delinquentb (%)

1.0

0.1

0.9

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 144

APPENDIX

Impact of

Impact of Difference in

Taking Up

Taking Up the Impact of

In-Person

Remote

Taking Up

Outcome

Services

Services

Services

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of in-person services, the treatment group includes the in-person treatment group and choice treatment group members who stated a preference for in-person services (n=806); and the control group includes the Initial Study Design control group and Modified Study Design control group members who stated a preference for in-person services (n=1,184). For the analysis of remote services, the treatment group includes the remote treatment group and choice treatment group members who stated a preference for remote services (n=2,516); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

D.3 Comparison of Impact of Choice of Service Modes with Impact of Remote

Services

This section compares impacts for the group of study participants who were offered a choice between in-person and remote services and compares impacts for that group to impacts for those offered remote services without a choice. This comparison may be of interest because homebuyers outside the context of a study are likely to have their choice of service mode and because homebuyer education and counseling services may be more effective for individuals able to choose their mode of service.

Exhibit D.5 compares the impact of being offered the choice of in-person or remote services with the impact of being offered remote services without a choice. Exhibit D.6 compares the impact of taking up services when offered the choice of service modes with the impact of taking up services when offered remote services.

As explained in chapter 3, the study shifted from randomizing individuals into a purely in-person treatment group when it became clear that take-up of in-person services was low (about one-quarter of those offered in-person services took up that offer). Replacing the in-person offer with an offer of a choice of service modes allows the study to consider whether giving prospective homebuyers the choice of in-person or remote services modes would be an improvement on what might otherwise be a world in which only remote services are available. Indeed, having a choice might lead to larger impacts for treatment group members if they feel additionally empowered, are more motivated or engaged, and thereby experience greater benefits from services because they take up services that is in line with their preferences. Alternatively, to the extent that “one more choice” in a choice-filled homebuying process dissuades choice treatment group members from participating in services, we might expect that those offered remote services (without a choice) might experience more positive impacts relative to those offered a choice of service modes. However, as discussed in chapter 5, we generally find that those in the choice treatment The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 145

APPENDIX

group had similar service participation rates to those in the in-person or remote treatment group who were offered the same service.

The results reported in exhibit D.5 and D.6 address the question of whether being given a choice of service modes leads to different impacts than being offered remote services without a choice. 94 As exhibit D.5 and D.6 reveal, for 2 out of 32 outcomes, we observe a statistically significant difference (at the 10-percent significance level) between the impact for those study participants given a choice of service modes and those offered access to remote services without a choice; and this is no more than we would expect due to random chance. One possible explanation for this general lack of statistically significant differences is that three-fourths of those who were given a choice of service modes expressed a preference for remotely provided services, implying that the large majority of choice group members opted for the exact same type of services offered to remote treatment group members. Another explanation is that something about the nature of the choice intervention itself would compel differential program impacts. In brief, neither of the possible hypotheses—greater empowerment or information overload—about having a choice as part of the intervention is borne out in the data.

Exhibit D.5: Comparison of Impact of Being Offered Choice of In-Person or Remote Services with Impact of Being Offered Remote Services

Difference

Impact of

Impact of

Between

Being Offered Being Offered Choice and

Choice of

Remote

Remote

Outcome

Service Modes Services

Impacts

Panel A: Preparedness and Search

Study participant was confident that he/she could find the information

8.1***

4.2**

3.9

he/she needed about the homebuying processa (%)

Number of lenders from which the study participant received price

-0.06

-0.04

-0.02

quotesa

Study participant purchased a homeb (%)

0.8

-0.1

0.9

Study participant was very satisfied with the homebuying processa (%)

4.5*

1.5

3.0

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in credit

4.6***

2.8**

1.7

reporta (%)

If in financial difficulty, study participant would contact his/her counseling

6.2***

4.1**

2.1

agency, consumer credit counseling agency, or other nonprofit

organization for assistance prior to missing a mortgage paymenta (%)

If in financial dif iculty, study participant would contact his/her lender for

3.5**

2.5

1.0

assistance prior to missing a mortgage paymenta (%)

94 We estimate the impact of remote services by comparing the control group with the remote treatment group. That is, no choice treatment group members are included in this estimate of the impact of remote services, which is why the values differ slightly from the impact of remote services reported in exhibit D.2, where we included all study participants who were offered remote services, including those in the choice treatment group, in order to maximize sample size for that analysis. This enables us to make a clean comparison of the impact of having a choice of service modes relative to the impact of remote services without a choice.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 146

APPENDIX

Difference

Impact of

Impact of

Between

Being Offered Being Offered Choice and

Choice of

Remote

Remote

Outcome

Service Modes Services

Impacts

Number of correct answers (out of four) to mortgage literacy quiza

-0.03

0.03

-0.06

If study participant started having financial problems and could not pay

-1.0

-1.0

-0.0

all of the bil s, he/she would pay the mortgage firsta (%)

Study participant has a budget and often compares it against actual

-2.3

-2.2

-0.1

spendinga (%)

Study participant usually pays credit card balance in full to avoid interest

1.7

0.7

1.0

chargesa (%)

Regularly required mortgage payment is automatically deducted from a

7.7***

3.9**

3.9*

bank accounta (%)

Credit score (out of 850)c

-0.4

-3.0*

2.6

Study participant has a credit score greater than or equal to 620c (%)

0.3

-1.2*

1.6

Student loan balancec ($)

728

680**

48

Credit card balancec ($)

-73

190

-263

Total nonhousing debtc ($)

517

1,133**

-616

Total monthly debt-to-income ratio (back-end ratio)b

1.7***

0.6

1.1

Total monthly debt-to-income ratio (back-end ratio) greater than 0.43b

2.7*

0.2

2.4

(%)

Study participant usually has enough savings set aside to cover 3

4.5**

-1.5

6.0**

months of expensesa (%)

Total savings and investmentsa ($)

1,378

226

1,152

Study participant occasional y does not have enough money to cover all

2.2*

1.5

0.7

of the bil s at the end of the montha (%)

Nonhousing debt bankruptcy, foreclosure, or repossession indicatorc (%)

0.9

0.2

0.8

Study participant has an electronic or written budgeta (%)

-1.8

-2.7

0.9

Study participant never uses payday lendersa (%)

-0.9

-0.8

-0.1

Study participant has tried to figure out how much he/she needs to save

0.5

-0.7

1.2

for retirementa (%)

Study participant often saves moneya (%)

-0.9

-0.5

-0.4

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

1.0

0.2

0.9

Monthly housing costs exceed 30 percent of monthly incomea (%)

1.7

0.6

1.0

Ever 30 days delinquentb (%)

0.5

0.5

0.0

Ever 60 days delinquentb (%)

0.1

0.3

-0.2

Ever 90 days delinquentb (%)

-0.1

0.1

-0.1

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of choice of service modes, the treatment group includes the choice treatment group (n=1,142); and the control group includes the Modified Study Design control group (n=1,656). For the analysis of remote services, the treatment group includes the remote treatment group (n=1,665); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 147

APPENDIX

Exhibit D.6: Comparison of Impact of Taking Up Services When Given Choice of Service Modes With Impact of Taking Up Services When Offered Remote Services

Impact of

Taking Up

Services

Impact of Difference in

When Given Taking Up the Impact of

Choice of

Remote

Taking Up

Outcome

Service Modes Services

Services

Panel A: Preparedness and Search

Study participant was confident that he/she could find the information

13.2***

6.0**

7.2

he/she needed about the homebuying processa (%)

Number of lenders from which the study participant received price quotesa

-0.10

-0.06

-0.04

Study participant purchased a homeb (%)

1.4

-0.2

1.6

Study participant was very satisfied with the homebuying processa (%)

7.4*

2.1

5.3

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in credit

7.5***

4.0**

3.4

reporta (%)

If in financial difficulty, study participant would contact his/her counseling

10.0***

5.7**

4.3

agency, consumer credit counseling agency, or other nonprofit organization

for assistance prior to missing a mortgage paymenta (%)

If in financial difficulty, study participant would contact his/her lender for

5.7**

3.5

2.2

assistance prior to missing a mortgage paymenta (%)

Number of correct answers (out of four) to mortgage literacy quiza

-0.05

0.04

-0.08

If study participant started having financial problems and could not pay all

-1.6

-1.4

-0.2

of the bil s, he/she would pay the mortgage firsta (%)

Study participant has a budget and often compares it against actual

-3.8

-3.1

-0.7

spendinga (%)

Study participant usually pays credit card balance in full to avoid interest

2.8

1.0

1.8

chargesa (%)

Regularly required mortgage payment is automatically deducted from a

12.6***

5.5**

7.2**

bank accounta (%)

Credit score (out of 850)c

-0.7

-4.7*

4.0

Study participant has a credit score greater than or equal to 620c (%)

0.6

-1.9*

2.5

Student loan balancec ($)

1,287

1,078**

208

Credit card balancec ($)

-129

302

-431

Total nonhousing debtc ($)

914

1,797**

-883

Total monthly debt-to-income ratio (back-end ratio)b

3.1***

1.0

2.1*

Total monthly debt-to-income ratio (back-end ratio) greater than 0.43b (%)

4.7*

0.4

4.4

Study participant usually has enough savings set aside to cover 3 months

7.3**

-2.2

9.5**

of expensesa (%)

Total savings and investmentsa ($)

2,245

316

1,929

Study participant occasional y does not have enough money to cover all of

3.6*

2.1

1.5

the bil s at the end of the montha (%)

Nonhousing debt bankruptcy, foreclosure, or repossession indicatorc (%)

1.7

0.3

1.4

Study participant has an electronic or written budgeta (%)

-2.9

-3.9

1.0

Study participant never uses payday lendersa (%)

-1.4

-1.1

-0.3

Study participant has tried to figure out how much he/she needs to save for

0.9

-0.9

1.8

retirementa (%)

Study participant often saves moneya (%)

-1.4

-0.7

-0.7

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

1.7

0.2

1.5

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.7

0.9

1.8

Ever 30 days delinquentb (%)

0.8

0.7

0.1

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 148

APPENDIX

Impact of

Taking Up

Services

Impact of Difference in

When Given Taking Up the Impact of

Choice of

Remote

Taking Up

Outcome

Service Modes Services

Services

Ever 60 days delinquentb (%)

0.1

0.4

-0.3

Ever 90 days delinquentb (%)

-0.1

0.1

-0.2

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6

for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: For the analysis of choice of service modes, the treatment group includes the choice treatment group (n=1,142); and the control group includes the Modified Study Design control group (n=1,656). For the analysis of in-person services, the treatment group includes the remote treatment group (n=1,665); and the control group includes the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures; and appendix D provides expanded results, by service mode.

D.4 Sensitivity of Impacts of In-Person and Remote Services to Using

Alternative Samples

This subsection considers whether changing the sample used for estimating mode effects influences the results. For example, including study participants who had a choice of service mode in the samples used to estimate mode-specific impacts might have affected the impacts estimated for in-person and remote services (those presented in exhibit D.1 and exhibit D.2).

This is because those who had a choice of services may have responded differently to the offer and the use of services from those who did not have a choice. To determine whether these findings are sensitive to inclusion of the choice treatment group, we present alternative estimates of the impact of in-person and remote services omitting study participants assigned to the choice treatment group.

Exhibit D.7 presents the impact of in-person homebuyer education and counseling, where the sample includes study participants randomly assigned to the in-person treatment group and the control group, excluding study participants randomly assigned to the choice treatment group. We find no evidence that the estimates of the impact of being offered in-person services presented in exhibit D.7 for this alternative sample are different from the corresponding impact estimates that include the choice treatment group reported in exhibit D.1. For all outcomes, the impact of being offered in-person services reported in exhibit D.1 is not statistically different (at the 5-percent significance level) from the corresponding impact estimate in exhibit D.7.

Exhibit D.8 presents the impact of remote homebuyer education and counseling, where the sample includes study participants randomly assigned to the remote treatment group and control group, excluding study participants randomly assigned to the choice treatment group. We find no evidence that the estimates of the impact of being offered remote services presented in exhibit The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 149

APPENDIX

D.8 for this alternative sample are different from the corresponding impact estimates that include the choice treatment group reported in exhibit D.2. For all outcomes, the impact of being offered remote services reported in exhibit D.2 is not statistically different (at the 5-percent significance level) from the corresponding impact estimate in exhibit D.8.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 150

APPENDIX

Exhibit D.7: Impact of In-Person Homebuyer Education and Counseling, Excluding Choice Group Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident

417

690

71.5

71.8

-0.3

-0.5%

0.911 ( -5.3, 4.6)

7.2

-1.1

-1.6

that he/she could find the

(2.9)

(9.9)

(13.9)

information he/she needed about

the homebuying processa (%)

Number of lenders from which the

404

669

1.51

1.59

-0.09

-5.6%

0.245 ( -0.22, 0.04)

0.19

-0.30

-0.41

study participant received price

(0.07)

(0.25)

(0.35)

quotesa

Study participant purchased a

511

784

63.3

63.1

0.2

0.3%

0.952 ( -4.6, 5.0)

7.0

0.7

0.9

homeb (%)

(2.8)

(10.8)

(13.7)

Study participant was very

415

683

33.4

29.0

4.4*

15.3%

0.072 ( 0.4, 8.5)

5.9

15.2*

21.4*

satisfied with the homebuying

(2.4)

(8.1)

(11.4)

processa (%)

Panel B: Financial Capability

Study participant knows how to

418

681

77.9

78.2

-0.3

-0.4%

0.899 ( -4.1, 3.6)

5.6

-1.0

-1.4

correct inaccurate information in

(2.3)

(7.8)

(10.8)

credit reporta (%)

If in financial difficulty, study

396

652

19.7

19.3

0.5

2.4%

0.858 ( -3.8, 4.7)

6.3

1.6

2.2

participant would contact his/her

(2.5)

(8.6)

(12.1)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

414

679

41.6

40.5

1.2

2.9%

0.697 ( -4.0, 6.4)

7.5

4.1

5.7

participant would contact his/her

(3.0)

(10.4)

(14.4)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

419

690

2.87

2.74

0.12**

4.5%

0.031 ( 0.03, 0.22)

0.14

0.42**

0.60**

four) to mortgage literacy quiza

(0.05)

(0.19)

(0.26)

If study participant started having

417

685

83.3

82.0

1.3

1.6%

0.691 ( -4.1, 6.7)

7.9

4.4

6.1

financial problems and could not

(3.2)

(10.9)

(15.2)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

Study participant has a budget and

418

686

37.3

37.9

-0.6

-1.7%

0.824 ( -5.5, 4.2)

7.1

-2.2

-3.1

often compares it against actual

(2.8)

(9.7)

(13.6)

spendinga (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 151

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant usually pays

416

685

66.3

66.8

-0.5

-0.8%

0.793 ( -3.8, 2.8)

4.8

-1.8

-2.5

credit card balance in full to avoid

(1.9)

(6.6)

(9.3)

interest chargesa (%)

Regularly required mortgage

418

689

30.4

29.1

1.3

4.5%

0.623 ( -3.2, 5.8)

6.5

4.5

6.3

payment is automatically deducted

(2.6)

(9.0)

(12.7)

from a bank accounta (%)

Credit score (out of 850)c

471

729

704.8

699.5

5.3

0.8%

0.121 ( -0.3, 10.9)

8.2

20.2

25.5

(3.3)

(12.6)

(15.9)

Study participant has a credit

471

729

87.6

84.5

3.1**

3.7%

0.041 ( 0.6, 5.6)

3.6

11.9**

15.1**

score greater than or equal to 620c

(1.5)

(5.6)

(7.0)

(%)

Student loan balancec ($)

473

736

9,983

10,382

-398

-3.8%

0.571 ( -1,582, 785)

1,730

-1,520

-1,955

(695)

(2,652)

(3,421)

Credit card balancec ($)

473

736

3,381

3,589

-208

-5.8%

0.455 ( -676, 260)

684

-794

-1,023

(275)

(1,048)

(1,352)

Total nonhousing debtc ($)

473

736

24,309

24,144

165

0.7%

0.849 ( -1,297, 1,627)

2,137

630

830

(858)

(3,274)

(4,216)

Total monthly debt-to-income ratio

474

726

26.4

26.0

0.4

1.6%

0.661 ( -1.2, 2.0)

2.3

1.6

2.0

(back-end ratio)b

(0.9)

(3.5)

(4.5)

Total monthly debt-to-income ratio

474

726

15.1

15.3

-0.2

-1.5%

0.873 ( -2.6, 2.2)

3.5

-0.9

-1.1

(back-end ratio) greater than 0.43b

(1.4)

(5.4)

(6.9)

(%)

Study participant usually has

417

690

62.4

68.7

-6.3***

-9.2%

0.005 ( -9.9, -2.8)

5.2

-22.0***

-30.9***

enough savings set aside to cover

(2.1)

(7.3)

(10.2)

3 months of expensesa (%)

Total savings and investmentsa ($)

395

649

52,154

50,164

1,990

4.0%

0.563 ( -3,794, 7,774)

8,455

6,844

9,166

(3,396)

(11,680)

(15,636)

Study participant occasionally

419

690

12.4

10.2

2.2

21.4%

0.299 ( -1.3, 5.7)

5.2

7.6

10.6

does not have enough money to

(2.1)

(7.1)

(10.0)

cover all of the bil s at the end of

the montha (%)

Nonhousing debt bankruptcy,

479

742

4.2

4.9

-0.6

-13.0%

0.607 ( -2.7, 1.4)

3.0

-2.4

-3.2

foreclosure, or repossession

(1.2)

(4.7)

(6.1)

indicatorc (%)

Study participant has an electronic

418

687

60.6

60.1

0.5

0.8%

0.887 ( -5.2, 6.2)

8.3

1.7

2.3

or writ en budgeta (%)

(3.3)

(11.5)

(16.1)

Study participant never uses

408

666

82.3

84.8

-2.4

-2.9%

0.333 ( -6.6, 1.8)

6.2

-8.3

-11.5

payday lendersa (%)

(2.5)

(8.4)

(11.7)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 152

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has tried to figure

413

687

81.1

80.3

0.8

1.0%

0.777 ( -3.9, 5.5)

6.8

2.7

3.7

out how much he/she needs to

(2.7)

(9.4)

(13.1)

save for retirementa (%)

Study participant often saves

417

690

58.3

59.7

-1.3

-2.3%

0.666 ( -6.6, 3.9)

7.7

-4.6

-6.4

moneya (%)

(3.1)

(10.6)

(14.7)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

383

636

27.3

25.0

2.3***

9.3%

0.006 ( 1.0, 3.6)

1.9

7.9***

10.8***

income ratioa

(0.8)

(2.6)

(3.6)

Monthly housing costs exceed 30

383

636

30.3

22.9

7.4***

32.4%

0.009 ( 2.9, 11.9)

6.6

25.2***

34.4***

percent of monthly incomea (%)

(2.6)

(9.0)

(12.3)

Ever 30 days delinquentb (%)

511

784

2.5

2.7

-0.2

-7.5%

0.817 ( -1.7, 1.3)

2.1

-0.8

-1.0

(0.9)

(3.3)

(4.2)

Ever 60 days delinquentb (%)

511

784

0.9

0.8

0.1

13.8%

0.845 ( -0.8, 1.0)

1.3

0.4

0.5

(0.5)

(2.0)

(2.6)

Ever 90 days delinquentb (%)

511

784

0.9

0.4

0.6

146.5%

0.196 ( -0.2, 1.3)

1.1

2.1

2.7

(0.4)

(1.6)

(2.1)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. For this analysis, the treatment group is the in-person treatment group (n=515), and the control group is the Initial Study Design control group (n=789). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 153

APPENDIX

Exhibit D.8: Impact of Remote Homebuyer Education and Counseling, Excluding Choice Group Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Panel A: Preparedness and Search

Study participant was confident

1,253

2,006

74.0

69.8

4.2**

6.1%

0.023 ( 1.2, 7.2)

4.4

6.0**

13.0**

that he/she could find the

(1.8)

(2.5)

(5.4)

information he/she needed about

the homebuying processa (%)

Number of lenders from which the

1,213

1,958

1.60

1.64

-0.04

-2.4%

0.360 ( -0.11, 0.03)

0.10

-0.06

-0.12

study participant received price

(0.04)

(0.06)

(0.13)

quotesa

Study participant purchased a

1,642

2,425

61.6

61.7

-0.1

-0.2%

0.932 ( -2.2, 2.0)

3.1

-0.2

-0.3

homeb (%)

(1.2)

(1.9)

(3.7)

Study participant was very

1,240

1,990

30.2

28.7

1.5

5.2%

0.335 ( -1.1, 4.1)

3.8

2.1

4.7

satisfied with the homebuying

(1.5)

(2.2)

(4.7)

processa (%)

Panel B: Financial Capability

Study participant knows how to

1,236

1,986

79.5

76.7

2.8**

3.7%

0.033 ( 0.7, 5.0)

3.2

4.0**

8.7**

correct inaccurate information in

(1.3)

(1.8)

(3.9)

credit reporta (%)

If in financial difficulty, study

1,208

1,925

21.0

16.9

4.1**

24.0%

0.023 ( 1.2, 6.9)

4.2

5.7**

12.3**

participant would contact his/her

(1.7)

(2.4)

(5.1)

counseling agency, consumer

credit counseling agency, or other

nonprofit organization for

assistance prior to missing a

mortgage paymenta (%)

If in financial difficulty, study

1,240

1,985

40.5

38.0

2.5

6.5%

0.183 ( -0.6, 5.5)

4.5

3.5

7.6

participant would contact his/her

(1.8)

(2.6)

(5.6)

lender for assistance prior to

missing a mortgage paymenta (%)

Number of correct answers (out of

1,253

2,007

2.77

2.75

0.03

1.0%

0.325 ( -0.02, 0.07)

0.07

0.04

0.08

four) to mortgage literacy quiza

(0.03)

(0.04)

(0.08)

If study participant started having

1,245

1,993

81.8

82.7

-1.0

-1.2%

0.478 ( -3.3, 1.3)

3.4

-1.4

-3.0

financial problems and could not

(1.4)

(1.9)

(4.2)

pay all of the bil s, he/she would

pay the mortgage firsta (%)

Study participant has a budget and

1,246

1,995

33.6

35.8

-2.2

-6.0%

0.290 ( -5.6, 1.2)

5.0

-3.1

-6.7

often compares it against actual

(2.0)

(2.9)

(6.2)

spendinga (%)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 154

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant usually pays

1,240

1,993

71.7

71.0

0.7

1.0%

0.681 ( -2.2, 3.6)

4.3

1.0

2.2

credit card balance in full to avoid

(1.7)

(2.5)

(5.3)

interest chargesa (%)

Regularly required mortgage

1,251

2,005

29.3

25.4

3.9**

15.2%

0.026 ( 1.1, 6.6)

4.1

5.5**

11.8**

payment is automatically deducted

(1.6)

(2.3)

(5.0)

from a bank accounta (%)

Credit score (out of 850)c

1,523

2,260

704.1

707.1

-3.0*

-0.4%

0.093 ( -5.9, -0.1)

4.3

-4.7*

-9.0*

(1.7)

(2.7)

(5.1)

Study participant has a credit

1,523

2,260

86.1

87.3

-1.2*

-1.4%

0.098 ( -2.4, -0.0)

1.8

-1.9*

-3.7*

score greater than or equal to 620c

(0.7)

(1.1)

(2.2)

(%)

Student loan balancec ($)

1,537

2,273

10,523

9,843

680**

6.9%

0.048 ( 121, 1,239)

817

1,078**

2,058**

(328)

(521)

(994)

Credit card balancec ($)

1,537

2,273

3,982

3,791

190

5.0%

0.299 ( -116, 496)

447

302

576

(180)

(285)

(544)

Total nonhousing debtc ($)

1,537

2,273

24,848

23,715

1,133**

4.8%

0.049 ( 196, 2,070)

1,369

1,797**

3,430**

(550)

(872)

(1,665)

Total monthly debt-to-income ratio

1,527

2,261

26.2

25.6

0.6

2.4%

0.252 ( -0.3, 1.5)

1.3

1.0

1.8

(back-end ratio)b

(0.5)

(0.8)

(1.6)

Total monthly debt-to-income ratio

1,527

2,261

16.3

16.1

0.2

1.4%

0.860 ( -2.0, 2.4)

3.2

0.4

0.7

(back-end ratio) greater than 0.43b

(1.3)

(2.0)

(3.9)

(%)

Study participant usually has

1,251

2,003

70.5

72.1

-1.5

-2.1%

0.328 ( -4.1, 1.1)

3.8

-2.2

-4.7

enough savings set aside to cover

(1.5)

(2.2)

(4.7)

3 months of expensesa (%)

Total savings and investmentsa ($)

1,193

1,891

49,446

49,220

226

0.5%

0.897 ( -2,719, 3,170)

4,305

316

665

(1,729)

(2,423)

(5,097)

Study participant occasionally

1,251

2,003

10.6

9.2

1.5

15.8%

0.210 ( -0.5, 3.4)

2.8

2.1

4.5

does not have enough money to

(1.1)

(1.6)

(3.5)

cover all of the bil s at the end of

the montha (%)

Nonhousing debt bankruptcy,

1,546

2,280

4.8

4.6

0.2

4.0%

0.776 ( -0.9, 1.3)

1.6

0.3

0.6

foreclosure, or repossession

(0.6)

(1.0)

(1.9)

indicatorc (%)

Study participant has an electronic

1,248

1,998

53.7

56.4

-2.7

-4.8%

0.179 ( -6.0, 0.6)

4.9

-3.9

-8.3

or writ en budgeta (%)

(2.0)

(2.8)

(6.0)

Study participant never uses

1,211

1,947

86.0

86.8

-0.8

-0.9%

0.540 ( -3.0, 1.4)

3.2

-1.1

-2.4

payday lendersa (%)

(1.3)

(1.8)

(3.9)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 155

APPENDIX

Impact of

Lower Bound Upper Bound

Treatment Control Treatment Control

Being

90-Percent Minimum of Impact of of Impact of

Sample

Sample

Group

Group

Of ered Percentage

Confidence Detectable Taking Up

Taking Up

Outcome

Size

Size

Mean

Mean

Services Impact

p-Value

Interval

Effect

Services

Services

Study participant has tried to figure

1,240

1,993

78.7

79.3

-0.7

-0.8%

0.655 ( -3.2, 1.8)

3.7

-0.9

-2.0

out how much he/she needs to

(1.5)

(2.1)

(4.5)

save for retirementa (%)

Study participant often saves

1,250

2,006

62.7

63.2

-0.5

-0.8%

0.723 ( -2.9, 1.9)

3.5

-0.7

-1.6

moneya (%)

(1.4)

(2.0)

(4.3)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-

1,168

1,843

24.3

24.2

0.2

0.6%

0.869 ( -1.4, 1.7)

2.3

0.2

0.5

income ratioa

(0.9)

(1.3)

(2.7)

Monthly housing costs exceed 30

1,168

1,843

22.7

22.1

0.6

2.8%

0.718 ( -2.2, 3.5)

4.2

0.9

1.8

percent of monthly incomea (%)

(1.7)

(2.4)

(5.0)

Ever 30 days delinquentb (%)

1,642

2,425

2.5

2.0

0.5

22.3%

0.241 ( -0.2, 1.1)

0.9

0.7

1.4

(0.4)

(0.6)

(1.1)

Ever 60 days delinquentb (%)

1,642

2,425

0.8

0.5

0.3

54.9%

0.295 ( -0.2, 0.7)

0.6

0.4

0.8

(0.3)

(0.4)

(0.8)

Ever 90 days delinquentb (%)

1,642

2,425

0.3

0.2

0.1

32.0%

0.759 ( -0.3, 0.4)

0.5

0.1

0.2

(0.2)

(0.3)

(0.6)

a Outcome was constructed using Short-Term Fol ow-Up Survey data, which cover 78.8 percent of the study sample. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.) b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample.

c Outcome was constructed using credit bureau data, which cover 92.7 percent of the study sample.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Standard errors are reported in parentheses. Due to rounding, reported impacts (T-C differences) could differ from differences between reported regression-adjusted means for the treatment and control groups. For this analysis, the treatment group is the remote treatment group (n=1,665); and the control group is the full control group (n=2,448). Appendix A details the analytic methods and sample restrictions; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 156

APPENDIX

Appendix E: Impacts on Subgroups Defined by Baseline

Characteristics

The data collected by the baseline survey and

baseline credit bureau data offer a rich set of

Key Findings: Impacts on

demographic, socioeconomic, financial, and

Subgroups Defined by Baseline

housing market characteristics from which the

Characteristics

team defined subgroups of interest. In this

This appendix reports impacts on subgroups

defined by baseline demographic,

appendix, we report impacts on subgroups as

socioeconomic, financial, and housing market

follows:

characteristics. In most cases the analysis

revealed no systematic differences in impacts

between subgroups.

• Exhibit E.1: presents results for the

comparison of impacts on subpopulations

We did observe evidence of between-subgroup

differences in impacts for three subgroups,

defined by gender.

defined by gender, age at the time of study

enrol ment, and area housing market affordability.

• Exhibit E.2 presents results for the

• For women, the offer of services increased

comparison of impacts on subpopulations

the 30-day delinquency rate by 1.5

defined by age at baseline.

percentage points and increased the 60-day

delinquency rate by 0.7 percentage point.

• Exhibit E.3 presents results for the

However, these findings rely on a very small

number of delinquencies, and there is no

comparison of impacts on subpopulations

evidence that the offer of services had other

defined by area housing affordability.

adverse impacts for women.

• For those age 29 or younger at baseline, the

• Exhibits E.4 and E.5 present results for the

offer of services increased home purchase

comparison of impacts on subpopulations

rates by 3.3 percentage points. In contrast,

defined by race and ethnicity.

we find no detectable evidence that the offer

of services increased home purchase rates

for those age 30 or older.

• Exhibit E.6 presents results for the

comparison of impacts on subpopulations

• Considering area housing affordability, we

find that for those in the low housing

defined by educational attainment at

affordability subgroup (that is, those living in

baseline.

relatively expensive areas), the offer of

services improved self-reported

• Exhibit E.7 presents results for the

communication with lenders. However, the

offer of services had a negative impact on

comparison of impacts on subpopulations

budgeting practices for those in the low

defined by marital status at baseline.

housing affordability subgroup, as measured

by whether the study participant has a

• Exhibit E.8 presents results for the

budget and often compares it to actual

spending.

comparison of impacts on subpopulations

defined by English language preference.

• Exhibit E.9 presents results for the comparison of impacts on subpopulations defined by stage in homebuying process at baseline.

• Exhibit E.10 presents results for the comparison of impacts on subpopulations defined by credit score at baseline.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 157

APPENDIX

• Exhibit E.11 presents results for the comparison of impacts on subpopulations defined by cash on hand for downpayment and closing costs at baseline.

• Exhibit E.12 presents results for the comparison of impacts on subpopulations defined by savings at baseline.

• Exhibit E.13 presents results for the comparison of impacts on subpopulations defined by borrower income relative to area median income at baseline.

• Exhibit E.14 presents results for the comparison of impacts on subpopulations defined by nonhousing debt at baseline.

We report impacts on individual subgroups based on the comparison of mean outcomes between the pooled treatment group and the control group. Additionally, we report whether the impact of the intervention differs across subgroups.

We conduct a substantial number of tests in this subgroup analysis: we estimate and report the impact of being offered services on 16 outcomes of interest for each of 13 sets of subgroups. In conducting this many tests, some between-subgroup differences in impacts might appear simply due to random chance. Therefore, in this appendix, we focus on subgroups for which there is evidence of systematic between-subgroup differences in impacts. We operationalize this strategy by setting the minimum threshold for evidence of systematic between-subgroup differences in impacts as follows: For a given subgroup of interest, we must find a statistically significant between-subgroup difference in impacts (at the 10-percent significance level) for 3 or more of the 16 outcomes analyzed. If there are 2 or fewer between-subgroup differences (out of 16 total tests for each subgroup), then we conclude that there is not sufficient evidence of between-subgroup differences in impacts to warrant discussion.

We did find evidence of systematic between-subgroup differences in impacts for three subgroups: gender (section E.1); age at the time of study enrollment (section E.2); and area housing market affordability (section E.3). For these three sets of subgroups, the discussion focuses on outcomes for which there is a statistically significant difference in impacts between subgroups. If the impact on a given outcome is not statistically different between the two subgroups of interest, then there is no evidence that practitioners or policymakers should alter their actions. For the other ten sets of subgroups, the analysis revealed no systematic between-subgroup differences in impacts (section E.4).

E.1 Differences in Subgroup Impacts: Gender

We consider the impact of being offered homebuyer education and counseling services on men (60.2 percent of the study sample) and women (the remaining 39.8 percent of the study The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 158

APPENDIX

sample). 95 As described in chapter 2 (section 2.3), women were more likely than men to participate in homebuyer education and counseling services. This finding is consistent with hypotheses about gender differences: women might be more likely to participate because they want to try to offset perceived or actual discrimination in financial markets; or women are at least anecdotally more willing than men to ask for help. Exhibit E.1 reports the impacts of homebuyer education and counseling has for the subgroup of women and the subgroup of men and shows differences, if any, between these two subgroups’ impacts.

Overall, we observe few differences in impacts between men and women. There are, however, two that bear mentioning. First, among male study participants, treatment group members had followup levels of student loan debt $1,121 higher than control group members (exhibit E.1). We do not find any such impact for women.

Second, for the subsample of women, we do detect an (unfavorable) impact of the intervention on two measures of loan performance. Specifically, for women, we observed that 15 of 1,337 (or 1.1 percent) of treatment group women were ever 60 days delinquent compared to 3 of 938 (or 0.3 percent) of control group women. Multivariate regression estimates indicate that this 0.7-percentage point difference is statistically significant. We also observe an increase of the 30-day delinquency rate by 1.5 percentage points.

Although these findings—particularly with regards to loan performance—are potentially concerning, we caution readers that these treatment-control differences in delinquency rates represent a very small share of the sample because of the infrequency of delinquencies at this early stage of homebuying for both treatment and control group members. Only 12 more treatment group women than control group women were 60 days delinquent. Mainly for this reason, we do not judge these results to be substantively important at this time.

Moreover, this isolated finding is not part of a larger pattern of gender-differential results that might justify additional concern. Instead, impacts on other outcomes for women do not align with this finding. For example, treatment group women showed greater satisfaction than did control group women with the homebuying process. Furthermore, treatment group women had housing costs similar to those of control group women, indicating that homebuyer education and counseling did not lead women to consume more housing than they could afford.

Finally, no clear hypothesis explains why homebuyer education and counseling services would lead women to have higher delinquency rates, so this adverse finding might simply be due to random chance. The study examines many outcomes for many subgroups, and some of those will 95 The disproportionate number of men in our sample is due to the recruitment process: lenders reached out to the primary borrower on the application or pre-application. Among those who purchased a home, 17 percent of female participants and 26 percent of male participants had a co-borrower at followup, although the gender of these co-borrowers is not known.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 159

APPENDIX

appear to be statistically significant simply because we test such a large number of hypotheses.

We cannot know which results are false positives, but we posit that the small number of sample members who are delinquent and the lack of a clear hypothesis related to the finding suggest that this may be a false positive result.

We will continue to monitor the impact of homebuyer education and counseling both overall and for subgroups, including women, on delinquency rates in the Long-Term Impact Report, which will measure delinquencies about 3½ to 5 years after study enrollment.

Exhibit E.1: Comparison of Impacts on Subpopulations Defined by Gender

Male

Female

(60.2% of Sample) (39.8% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

6.1***

2.9

3.2

information he/she needed about the homebuying processa (%)

(1.8)

(2.3)

(2.6)

Study participant purchased a homeb (%)

1.0

-0.4

1.5

(0.8)

(1.8)

(1.7)

Study participant was very satisfied with the homebuying

2.5

4.0*

-1.5

processa (%)

(1.7)

(2.2)

(2.5)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

3.2

2.2

1.1

credit reporta (%)

(1.9)

(2.4)

(3.9)

If in financial difficulty, study participant would contact his/her

3.6**

5.5***

-1.9

counseling agency, consumer credit counseling agency, or other

(1.5)

(1.5)

(1.8)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial dif iculty, study participant would contact his/her

2.4

3.0

-0.6

lender for assistance prior to missing a mortgage paymenta (%)

(2.1)

(1.8)

(2.3)

Number of correct answers (out of four) to mortgage literacy quiza

0.01

0.05

-0.05

(0.04)

(0.04)

(0.06)

Study participant has a budget and often compares it against

-2.1

-1.7

-0.4

actual spendinga (%)

(1.9)

(2.4)

(3.2)

Credit score (out of 850)c

-1.1

-1.0

-0.1

(1.6)

(2.2)

(2.4)

Study participant has a credit score greater than or equal to 620c

0.2

-0.7

0.9

(%)

(0.8)

(1.1)

(1.5)

Student loan balancec ($)

1,121**

-301

1,422*

(412)

(522)

(737)

Total nonhousing debtc ($)

1,056

323

733

(633)

(583)

(910)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

1.1*

0.3

0.8

(0.6)

(1.0)

(0.8)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.8**

0.8

2.0

(1.2)

(1.8)

(2.0)

Ever 30 days delinquentb (%)

-0.4

1.5***

-2.0***

(0.4)

(0.5)

(0.6)

Ever 60 days delinquentb (%)

-0.2

0.7***

-0.9**

(0.3)

(0.2)

(0.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 160

APPENDIX

Male

Female

(60.2% of Sample) (39.8% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, 4,546

sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

E.2 Differences in Subgroup Impacts: Age

We consider the impact of being offered homebuyer education and counseling services on those study participants who were age 29 or younger at the time of study enrollment (31.7 percent of the study sample) and a complementary subgroup comprising those age 30 or older at the time of study enrollment (the remaining 68.3 percent of the study sample). These two groups, divided by age, are at different life cycle stages, and their life circumstances likely influence the ways in which they interact with the intervention and their subsequent experience. For example, the younger group, part of the millennial generation, was less likely to be married or have children relative to the older study participants. Interestingly, although the two groups had similar baseline incomes, those age 29 or younger had higher baseline credit scores relative to older study participants. Participation in services did not differ between the two groups: the younger and older subgroups took up services and completed all services at similar rates.

Among study participants age 29 or younger at baseline, 68.8 percent of those offered homebuyer education and counseling services purchased a home. This home purchase rate is 3.3

percentage points higher than the rate of their age 29 or younger control group counterparts. In contrast, we find no evidence that services increased home purchase rates for those age 30 or older. As discussed in more detail in chapter 8 (section 8.3.1), the success of homebuyer education and counseling at increasing home purchase rates for younger prospective homebuyers is important because homeownership rates among young people are at historic lows (Spader, McCue, and Herbert, 2016).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 161

APPENDIX

Exhibit E.2: Comparison of Impacts on Subpopulations Defined by Age at Baseline Age 30 or Older Age 29 or Younger

(68.3% of Sample) (31.7% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

4.0**

6.7**

-2.7

information he/she needed about the homebuying processa (%)

(1.6)

(2.6)

(2.8)

Study participant purchased a homeb (%)

-0.7

3.3**

-4.0**

(1.1)

(1.5)

(1.7)

Study participant was very satisfied with the homebuying

2.3

3.7

-1.4

processa (%)

(2.0)

(2.7)

(3.6)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

1.1

5.4**

-4.3*

credit reporta (%)

(1.2)

(2.2)

(2.5)

If in financial difficulty, study participant would contact his/her

4.3***

6.0**

-1.8

counseling agency, consumer credit counseling agency, or other

(1.5)

(2.4)

(2.9)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

2.1

3.4

-1.3

lender for assistance prior to missing a mortgage paymenta (%)

(1.9)

(3.2)

(3.5)

Number of correct answers (out of four) to mortgage literacy quiza

-0.03

0.11**

-0.14**

(0.03)

(0.05)

(0.06)

Study participant has a budget and often compares it against

-2.4

0.8

-3.1

actual spendinga (%)

(2.1)

(2.6)

(3.8)

Credit score (out of 850)c

-1.6

-0.4

-1.2

(1.6)

(2.5)

(2.8)

Study participant has a credit score greater than or equal to 620c

-0.3

0.3

-0.6

(%)

(0.8)

(1.2)

(1.5)

Student loan balancec ($)

879**

-87

965

(344)

(681)

(813)

Total nonhousing debtc ($)

919*

479

440

(482)

(727)

(818)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

0.6

0.8

-0.2

(1.0)

(1.1)

(1.6)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.1

3.8**

-1.7

(1.5)

(1.8)

(2.4)

Ever 30 days delinquentb (%)

0.3

0.9

-0.6

(0.4)

(0.7)

(0.9)

Ever 60 days delinquentb (%)

0.2

0.1

0.1

(0.3)

(0.3)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

This increase in home purchase rates for the younger subgroup also could be explained by the observed favorable impacts on outcomes in the financial capability domain for this group: the The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 162

APPENDIX

younger treatment subgroup showed higher scores on a four-question mortgage literacy quiz and greater confidence in their ability to correct credit report inaccuracies than did the younger control subgroup. In contrast, the offer of homebuyer education and counseling to those age 30

or older did not show these benefits. These gains in financial capability for the age 29 or younger subgroup might have helped them navigate the complex homebuying process.

These findings have implications for how homebuyer education and counseling services could be targeted to different populations. If services have greater benefits for younger prospective homebuyers, additional efforts to target services at this group might be warranted.

E.3 Differences in Subgroup Impacts: Area Housing Affordability

We consider the impact of being offered homebuyer education and counseling services on subgroups defined by area housing affordability. The “low housing affordability” subgroup consists of study participants living in metropolitan areas at the time of study enrollment where the ratio of the median value of an owner-occupied unit to the median income is greater than or equal to the corresponding ratio for the United States. 96 Examples of areas included in the low housing affordability subgroup include Boston-Quincy, MA; Washington-Arlington-Alexandria, DC-VA-MD-WV; and San Francisco-San Mateo-Redwood City, California. More than two-thirds of the study sample, 68.1 percent, lived in areas where this was the case. The “high housing affordability” subgroup, the remaining 31.9 percent of the study sample, consists of study participants living in metropolitan areas where the ratio of the median value of an owner-occupied unit to the median income is less than the nationwide ratio. Examples of areas included in the high housing affordability subgroup include St. Louis, MO; Atlanta-Sandy Springs-Marietta, GA; and Minneapolis-St. Paul-Bloomington, MN-WI.

Study participants in the low housing affordability subgroup had comparatively high baseline incomes and credit scores, with a mean income with co-borrowers of $63,000 and a mean credit score of 712, relative to those in the high housing affordability subgroup, who had a mean income with co-borrowers of $52,000 and mean credit score of 694. Homebuyer education and counseling service participation rates were similar for the two subgroups.

We find mixed evidence on the effect of homebuyer education and counseling services on subgroups defined by area housing affordability. For those in the low housing affordability subgroup (that is, those living in relatively expensive areas), services improved self-reported communication with lenders, whereas no evidence indicates that homebuyer education and counseling services improved communication with lenders for those within the high housing affordability subgroup (that is, those living in relatively inexpensive areas).

96 See exhibit B.3 for detailed operationalization.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 163

APPENDIX

Homebuyer education and counseling services had a negative impact on budgeting practices for those in the low housing affordability subgroup, as measured by whether the study participant has a budget and often compares it against actual spending. In contrast, those offered services in the high housing affordability subgroup were more likely to compare their budget against actual spending. The difference in impacts on this measure between the groups was relatively large, at 7.0 percentage points. Those living in relatively expensive metropolitan areas such as San Francisco or Washington, D.C. might have less time for budgeting because of the increased effort needed to find an affordable home and might view homebuyer education and counseling services as a substitute for more formal budgeting.

Exhibit E.3: Comparison of Impacts on Subpopulations Defined by Area Housing Affordability Low Housing

High Housing

Affordability

Affordability

(68.1% of Sample) (31.9% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

4.9**

5.1*

-0.1

information he/she needed about the homebuying processa (%)

(1.9)

(2.9)

(3.5)

Study participant purchased a homeb (%)

1.4

-1.4

2.8

(1.0)

(2.0)

(2.2)

Study participant was very satisfied with the homebuying

4.3**

0.5

3.7

processa (%)

(1.8)

(2.2)

(2.8)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

2.1*

4.5**

-2.4

credit reporta (%)

(1.1)

(1.9)

(2.2)

If in financial difficulty, study participant would contact his/her

4.0***

5.0***

-1.0

counseling agency, consumer credit counseling agency, or other

(1.3)

(1.8)

(2.1)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial dif iculty, study participant would contact his/her

3.8*

-0.1

4.0*

lender for assistance prior to missing a mortgage paymenta (%)

(2.0)

(1.5)

(2.3)

Number of correct answers (out of four) to mortgage literacy quiza

0.05

-0.02

0.07*

(0.03)

(0.03)

(0.04)

Study participant has a budget and often compares it against

-4.1**

3.0**

-7.1***

actual spendinga (%)

(1.5)

(1.1)

(1.8)

Credit score (out of 850)c

-1.2

-0.4

-0.8

(1.8)

(2.5)

(3.1)

Study participant has a credit score greater than or equal to 620c

-0.3

0.3

-0.5

(%)

(0.7)

(1.2)

(1.3)

Student loan balancec ($)

603*

616

-13

(320)

(671)

(749)

Total nonhousing debtc ($)

1,023*

525

499

(508)

(894)

(1,083)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

0.6

1.0

-0.4

(0.8)

(0.9)

(1.3)

Monthly housing costs exceed 30 percent of monthly incomea (%)

1.1

3.7**

-2.5

(1.5)

(1.5)

(2.3)

Ever 30 days delinquentb (%)

0.4

0.2

0.2

(0.4)

(0.6)

(0.7)

Ever 60 days delinquentb (%)

0.2

0.1

0.1

(0.3)

(0.3)

(0.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 164

APPENDIX

Low Housing

High Housing

Affordability

Affordability

(68.1% of Sample) (31.9% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

E.4 Additional Subgroup Results

The remainder of this appendix shows subgroup results for subgroups where there was no evidence of systematic between-group differences in impacts.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 165

APPENDIX

Exhibit E.4: Comparison of Impacts on Subpopulations Defined by Race/Ethnicity: White Non-Hispanic Versus Al Other Race/Ethnicities

White Non-

All Other

Hispanic

Race/Ethnicities

(38.5% of Sample) (61.5% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

3.2

6.3***

-3.0

information he/she needed about the homebuying processa (%)

(2.0)

(2.0)

(2.7)

Study participant purchased a homeb (%)

-0.4

0.9

-1.3

(1.3)

(1.5)

(2.1)

Study participant was very satisfied with the homebuying

3.7

2.2

1.5

processa (%)

(2.6)

(1.3)

(2.7)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

4.2*

1.7

2.6

credit reporta (%)

(2.3)

(1.3)

(3.0)

If in financial difficulty, study participant would contact his/her

6.6***

3.2*

3.5

counseling agency, consumer credit counseling agency, or other

(1.8)

(1.7)

(2.6)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

2.7

2.4

0.3

lender for assistance prior to missing a mortgage paymenta (%)

(2.4)

(1.9)

(2.8)

Number of correct answers (out of four) to mortgage literacy quiza

0.02

0.03

-0.01

(0.04)

(0.03)

(0.04)

Study participant has a budget and often compares it against

-5.5***

0.6

-6.1***

actual spendinga (%)

(1.5)

(2.0)

(2.1)

Credit score (out of 850)c

0.6

-1.7

2.2

(2.3)

(1.8)

(3.0)

Study participant has a credit score greater than or equal to 620c

0.2

-0.2

0.4

(%)

(1.2)

(0.9)

(1.7)

Student loan balancec ($)

239

742**

-503

(694)

(326)

(856)

Total nonhousing debtc ($)

375

1,049

-674

(677)

(624)

(1,006)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

1.9**

-0.0

1.9**

(0.7)

(0.7)

(0.7)

Monthly housing costs exceed 30 percent of monthly incomea (%)

1.6

2.2*

-0.6

(1.9)

(1.2)

(2.3)

Ever 30 days delinquentb (%)

0.4

0.4

0.0

(0.5)

(0.4)

(0.7)

Ever 60 days delinquentb (%)

-0.2

0.4

-0.6

(0.2)

(0.3)

(0.4)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 166

APPENDIX

Exhibit E.5: Comparison of Impacts on Subpopulations Defined by Race/Ethnicity: White Non-Hispanic Versus African-American Non-Hispanic

White Non-

African-American

Hispanic

Non-Hispanic

(38.5% of Sample) (20.5% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

3.2

6.5*

-3.2

information he/she needed about the homebuying processa (%)

(2.1)

(3.4)

(3.7)

Study participant purchased a homeb (%)

-0.4

0.8

-1.2

(1.3)

(2.9)

(3.6)

Study participant was very satisfied with the homebuying

3.9

3.0

0.9

processa (%)

(2.5)

(2.6)

(3.3)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

4.0*

0.7

3.4

credit reporta (%)

(2.4)

(2.4)

(3.4)

If in financial difficulty, study participant would contact his/her

6.7***

4.6*

2.1

counseling agency, consumer credit counseling agency, or other

(1.8)

(2.7)

(3.6)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

2.8

4.7*

-1.9

lender for assistance prior to missing a mortgage paymenta (%)

(2.3)

(2.4)

(3.4)

Number of correct answers (out of four) to mortgage literacy quiza

0.02

-0.07

0.10

(0.04)

(0.07)

(0.08)

Study participant has a budget and often compares it against

-5.5***

2.6

-8.1*

actual spendinga (%)

(1.5)

(4.3)

(4.3)

Credit score (out of 850)c

0.6

-6.9*

7.4

(2.3)

(3.9)

(4.7)

Study participant has a credit score greater than or equal to 620c

0.1

-4.3

4.5

(%)

(1.2)

(2.7)

(3.2)

Student loan balancec ($)

332

1,309**

-978

(625)

(636)

(962)

Total nonhousing debtc ($)

582

1,474

-892

(681)

(1,120)

(1,451)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

1.9**

0.3

1.6

(0.7)

(1.3)

(1.3)

Monthly housing costs exceed 30 percent of monthly incomea (%)

1.9

2.5

-0.6

(1.8)

(2.5)

(2.9)

Ever 30 days delinquentb (%)

0.3

0.8

-0.5

(0.5)

(0.7)

(0.9)

Ever 60 days delinquentb (%)

-0.2

0.5

-0.7

(0.2)

(0.5)

(0.6)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Notes: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 167

APPENDIX

Exhibit E.6: Comparison of Impacts on Subpopulations Defined by Educational Attainment at Baseline Bachelor’s Degree Less Than a

or Higher

Bachelor’s Degree

(53.4% of Sample) (46.6% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

6.1***

3.4

2.8

information he/she needed about the homebuying processa (%)

(1.4)

(2.5)

(2.4)

Study participant purchased a homeb (%)

-1.5

2.6*

-4.1

(2.0)

(1.4)

(2.8)

Study participant was very satisfied with the homebuying

2.5

3.7

-1.3

processa (%)

(1.8)

(2.4)

(3.0)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

1.7

4.4**

-2.7

credit reporta (%)

(1.3)

(1.9)

(2.6)

If in financial difficulty, study participant would contact his/her

3.3**

5.7***

-2.4

counseling agency, consumer credit counseling agency, or other

(1.4)

(1.6)

(1.9)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial dif iculty, study participant would contact his/her

2.7

2.4

0.3

lender for assistance prior to missing a mortgage paymenta (%)

(2.3)

(2.3)

(3.3)

Number of correct answers (out of four) to mortgage literacy quiza

0.02

0.03

-0.01

(0.03)

(0.05)

(0.06)

Study participant has a budget and often compares it against

-0.3

-3.5

3.2

actual spendinga (%)

(1.7)

(2.3)

(2.7)

Credit score (out of 850)c

-1.6

-0.4

-1.2

(1.6)

(1.8)

(2.0)

Study participant has a credit score greater than or equal to 620c

-0.3

0.0

-0.4

(%)

(0.7)

(1.1)

(1.3)

Student loan balancec ($)

688

407

281

(545)

(342)

(720)

Total nonhousing debtc ($)

1,188

366

822

(707)

(538)

(951)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

0.6

0.9

-0.3

(0.7)

(1.1)

(1.3)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.1

2.2

-0.0

(1.6)

(2.4)

(3.4)

Ever 30 days delinquentb (%)

0.6

0.1

0.5

(0.4)

(0.6)

(0.7)

Ever 60 days delinquentb (%)

0.3

0.0

0.3

(0.3)

(0.3)

(0.4)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 168

APPENDIX

Exhibit E.7: Comparison of Impacts on Subpopulations Defined by Marital Status at Baseline Married

Not Married

(38.2% of Sample) (61.8% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

9.3***

2.2

7.1**

information he/she needed about the homebuying processa (%)

(2.0)

(2.2)

(3.0)

Study participant purchased a homeb (%)

0.5

0.5

0.0

(1.3)

(1.6)

(2.3)

Study participant was very satisfied with the homebuying

4.0

2.5

1.6

processa (%)

(2.4)

(2.2)

(3.7)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

2.6

3.1**

-0.5

credit reporta (%)

(1.9)

(1.3)

(2.6)

If in financial difficulty, study participant would contact his/her

4.9***

4.1***

0.8

counseling agency, consumer credit counseling agency, or other

(1.7)

(1.5)

(2.1)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

2.2

2.8

-0.5

lender for assistance prior to missing a mortgage paymenta (%)

(2.1)

(2.0)

(2.6)

Number of correct answers (out of four) to mortgage literacy quiza

0.01

0.03

-0.02

(0.04)

(0.04)

(0.06)

Study participant has a budget and often compares it against

-0.7

-2.6

2.0

actual spendinga (%)

(2.0)

(1.9)

(2.5)

Credit score (out of 850)c

-2.8

0.1

-2.8

(2.0)

(1.8)

(2.6)

Study participant has a credit score greater than or equal to 620c

-1.6

0.8

-2.4

(%)

(1.1)

(1.0)

(1.6)

Student loan balancec ($)

872**

346

526

(377)

(441)

(622)

Total nonhousing debtc ($)

881

754

127

(695)

(587)

(963)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

1.0

0.6

0.5

(1.0)

(0.7)

(1.2)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.4

1.9

0.6

(2.0)

(1.6)

(2.8)

Ever 30 days delinquentb (%)

0.4

0.3

0.0

(0.7)

(0.5)

(1.0)

Ever 60 days delinquentb (%)

0.2

0.2

0.1

(0.5)

(0.2)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 169

APPENDIX

Exhibit E.8: Comparison of Impacts on Subpopulations Defined by English Language Preference English Language Spanish Language

Preference

Preference

(90.9% of Sample) (9.1% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

4.7***

7.9

-3.2

information he/she needed about the homebuying processa (%)

(1.6)

(7.6)

(7.7)

Study participant purchased a homeb (%)

0.6

-1.1

1.7

(1.2)

(2.5)

(3.5)

Study participant was very satisfied with the homebuying

2.8*

5.0

-2.2

processa (%)

(1.5)

(4.7)

(4.9)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

2.7***

6.3

-3.6

credit reporta (%)

(0.9)

(4.2)

(4.2)

If in financial difficulty, study participant would contact his/her

4.5***

1.5

3.0

counseling agency, consumer credit counseling agency, or other

(1.2)

(3.4)

(3.6)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

2.7

1.6

1.0

lender for assistance prior to missing a mortgage paymenta (%)

(1.6)

(3.8)

(3.7)

Number of correct answers (out of four) to mortgage literacy quiza

0.03

-0.06

0.09

(0.03)

(0.08)

(0.09)

Study participant has a budget and often compares it against

-1.8

-1.9

0.1

actual spendinga (%)

(1.6)

(3.5)

(3.7)

Credit score (out of 850)c

-1.8

6.7*

-8.5**

(1.4)

(3.7)

(3.4)

Study participant has a credit score greater than or equal to 620c

-0.4

2.3

-2.8

(%)

(0.7)

(1.8)

(2.1)

Student loan balancec ($)

617*

-35

652

(304)

(586)

(633)

Total nonhousing debtc ($)

1,008**

-1,373*

2,382**

(478)

(711)

(990)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

0.8

-0.6

1.4

(0.5)

(2.3)

(2.1)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.6**

-3.8

6.4

(1.1)

(4.7)

(4.7)

Ever 30 days delinquentb (%)

0.4

-0.1

0.6

(0.3)

(0.7)

(0.8)

Ever 60 days delinquentb (%)

0.2

0.2

0.0

(0.2)

(0.6)

(0.7)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 170

APPENDIX

Exhibit E.9: Comparison of Impacts on Subpopulations Defined by Stage in Homebuying Process at Baseline Made an Of er,

Signed an

Early Stage in

Agreement, or

Homebuying

Purchased a Home

Process

(51.7% of Sample) (48.3% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

3.7**

6.2**

-2.5

information he/she needed about the homebuying processa (%)

(1.8)

(2.4)

(2.7)

Study participant purchased a homeb (%)

0.4

0.7

-0.3

(1.1)

(1.5)

(1.6)

Study participant was very satisfied with the homebuying

3.9*

1.8

2.1

processa (%)

(2.0)

(2.1)

(2.8)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

3.3**

1.8

1.5

credit reporta (%)

(1.5)

(1.5)

(2.3)

If in financial difficulty, study participant would contact his/her

5.6***

3.0*

2.7

counseling agency, consumer credit counseling agency, or other

(2.0)

(1.5)

(2.5)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

4.1*

0.9

3.2

lender for assistance prior to missing a mortgage paymenta (%)

(2.3)

(1.9)

(2.8)

Number of correct answers (out of four) to mortgage literacy quiza

0.03

0.02

0.01

(0.03)

(0.04)

(0.05)

Study participant has a budget and often compares it against

-3.3*

0.1

-3.4

actual spendinga (%)

(1.7)

(2.1)

(2.4)

Credit score (out of 850)c

0.2

-2.9

3.1

(1.7)

(2.7)

(3.4)

Study participant has a credit score greater than or equal to 620c

0.3

-1.0

1.3

(%)

(1.0)

(1.4)

(2.0)

Student loan balancec ($)

103

1,025***

-922

(549)

(338)

(727)

Total nonhousing debtc ($)

515

1,038*

-523

(675)

(553)

(918)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

1.2*

0.3

0.9

(0.6)

(0.8)

(0.7)

Monthly housing costs exceed 30 percent of monthly incomea (%)

3.2*

1.0

2.2

(1.6)

(2.2)

(3.1)

Ever 30 days delinquentb (%)

0.2

0.5

-0.3

(0.6)

(0.4)

(0.7)

Ever 60 days delinquentb (%)

0.1

0.2

-0.1

(0.4)

(0.2)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 171

APPENDIX

Exhibit E.10: Comparison of Impacts on Subpopulations Defined by Credit Score at Baseline Credit Score 680

Credit Score

or Above

Below 680

(64.9% of Sample) (35.1% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the information

5.3***

2.9

2.4

he/she needed about the homebuying processa (%)

(1.3)

(2.9)

(3.0)

Study participant purchased a homeb (%)

0.7

-0.1

0.8

(1.2)

(1.8)

(2.1)

Study participant was very satisfied with the homebuying processa (%)

3.4*

2.8

0.6

(1.7)

(2.2)

(2.3)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in credit

3.6**

1.4

2.1

reporta (%)

(1.7)

(2.3)

(3.1)

If in financial dif iculty, study participant would contact his/her counseling

4.9***

4.2***

0.7

agency, consumer credit counseling agency, or other nonprofit

(1.6)

(1.5)

(1.7)

organization for assistance prior to missing a mortgage paymenta (%)

If in financial dif iculty, study participant would contact his/her lender for

3.9*

-0.6

4.6*

assistance prior to missing a mortgage paymenta (%)

(2.1)

(2.2)

(2.4)

Number of correct answers (out of four) to mortgage literacy quiza

0.05

-0.02

0.07

(0.04)

(0.04)

(0.06)

Study participant has a budget and often compares it against actual

-1.0

-4.0

3.0

spendinga (%)

(1.4)

(3.2)

(3.3)

Credit score (out of 850)c

-1.8

0.3

-2.0

(1.5)

(3.1)

(3.8)

Study participant has a credit score greater than or equal to 620c (%)

0.4

-0.0

0.4

(0.4)

(1.8)

(2.0)

Student loan balancec ($)

-27

1,583***

-1,609**

(368)

(539)

(650)

Total nonhousing debtc ($)

340

1,585*

-1,245

(559)

(845)

(1,004)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

0.7

0.5

0.1

(0.8)

(1.1)

(1.3)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.2

2.1

0.0

(1.4)

(3.1)

(3.7)

Ever 30 days delinquentb (%)

0.6

0.5

0.1

(0.4)

(0.7)

(0.9)

Ever 60 days delinquentb (%)

-0.0

0.6

-0.6

(0.2)

(0.5)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 172

APPENDIX

Exhibit E.11: Comparison of Impacts on Subpopulations Defined by Cash on Hand for Downpayment and Closing Costs at Baseline

$15,000 or More Less Than $15,000

for Downpayment for Downpayment

(51.1% of Sample) (48.9% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

5.3**

4.3*

1.0

information he/she needed about the homebuying processa (%)

(2.0)

(2.4)

(3.1)

Study participant purchased a homeb (%)

2.2

-1.2

3.4*

(1.4)

(1.4)

(1.9)

Study participant was very satisfied with the homebuying

2.6

3.3

-0.7

processa (%)

(1.6)

(2.2)

(2.5)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

2.9*

2.7

0.2

credit reporta (%)

(1.4)

(1.7)

(2.5)

If in financial difficulty, study participant would contact his/her

5.0***

3.6**

1.3

counseling agency, consumer credit counseling agency, or other

(1.3)

(1.5)

(1.5)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

6.0***

-1.1

7.0***

lender for assistance prior to missing a mortgage paymenta (%)

(1.9)

(2.1)

(2.4)

Number of correct answers (out of four) to mortgage literacy quiza

0.04

0.00

0.04

(0.04)

(0.04)

(0.06)

Study participant has a budget and often compares it against

-1.5

-1.9

0.4

actual spendinga (%)

(1.6)

(1.9)

(1.7)

Credit score (out of 850)c

-1.2

-1.2

-0.1

(1.4)

(2.1)

(2.2)

Study participant has a credit score greater than or equal to 620c

-0.6

0.1

-0.6

(%)

(0.8)

(1.3)

(1.8)

Student loan balancec ($)

262

764

-502

(600)

(464)

(906)

Total nonhousing debtc ($)

118

1,444***

-1,327

(707)

(468)

(859)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

0.8

0.4

0.4

(1.0)

(0.7)

(1.1)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.7

1.5

1.2

(1.6)

(1.7)

(2.4)

Ever 30 days delinquentb (%)

0.3

0.4

-0.1

(0.5)

(0.5)

(0.8)

Ever 60 days delinquentb (%)

0.4

0.1

0.3

(0.2)

(0.4)

(0.4)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 173

APPENDIX

Exhibit E.12: Comparison of Impacts on Subpopulations Defined by Savings at Baseline Savings $20,000 or Savings Less Than

More

$20,000

(52.6% of Sample) (47.4% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

5.7***

4.0

1.7

information he/she needed about the homebuying processa (%)

(1.5)

(2.4)

(2.5)

Study participant purchased a homeb (%)

2.3

-1.3

3.6

(1.6)

(1.4)

(2.3)

Study participant was very satisfied with the homebuying

4.2**

1.7

2.4

processa (%)

(2.0)

(1.6)

(2.2)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

1.8

4.3**

-2.6

credit reporta (%)

(1.3)

(1.8)

(2.4)

If in financial difficulty, study participant would contact his/her

6.3***

2.1

4.2

counseling agency, consumer credit counseling agency, or other

(1.9)

(1.7)

(2.6)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

4.9**

-0.3

5.2*

lender for assistance prior to missing a mortgage paymenta (%)

(2.3)

(2.1)

(2.8)

Number of correct answers (out of four) to mortgage literacy quiza

-0.01

0.06*

-0.07

(0.03)

(0.03)

(0.04)

Study participant has a budget and often compares it against

-3.4**

-0.3

-3.1

actual spendinga (%)

(1.5)

(2.5)

(2.7)

Credit score (out of 850)c

-1.5

-0.4

-1.1

(1.6)

(2.3)

(2.5)

Study participant has a credit score greater than or equal to 620c

-0.9

0.8

-1.7

(%)

(0.7)

(1.2)

(1.4)

Student loan balancec ($)

719

218

501

(499)

(487)

(785)

Total nonhousing debtc ($)

692

953*

-261

(714)

(544)

(927)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

1.3*

0.3

1.0

(0.7)

(0.9)

(1.0)

Monthly housing costs exceed 30 percent of monthly incomea (%)

1.4

2.8

-1.4

(1.5)

(1.7)

(2.2)

Ever 30 days delinquentb (%)

0.6

0.1

0.5

(0.5)

(0.6)

(0.8)

Ever 60 days delinquentb (%)

0.2

0.2

0.1

(0.4)

(0.3)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for sSurvey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 174

APPENDIX

Exhibit E.13: Comparison of Impacts on Subpopulations Defined by Borrower Income Relative to Area Median Income at Baseline

Income

Income

Higher Than 80

Lower Than 80

Percent of Area Percent of Area

Median

Median

(44.6% of Sample) (55.4% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

6.2**

3.8*

2.4

information he/she needed about the homebuying processa (%)

(2.3)

(2.0)

(2.8)

Study participant purchased a homeb (%)

1.0

-0.2

1.2

(1.5)

(1.2)

(1.9)

Study participant was very satisfied with the homebuying

2.9

3.4

-0.5

processa (%)

(1.7)

(2.2)

(2.8)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

3.2*

2.6*

0.6

credit reporta (%)

(1.6)

(1.5)

(2.6)

If in financial difficulty, study participant would contact his/her

4.2**

4.3***

-0.1

counseling agency, consumer credit counseling agency, or other

(2.0)

(1.2)

(2.2)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

3.1

1.6

1.5

lender for assistance prior to missing a mortgage paymenta (%)

(2.8)

(1.2)

(2.7)

Number of correct answers (out of four) to mortgage literacy quiza

0.01

0.04

-0.04

(0.02)

(0.04)

(0.04)

Study participant has a budget and often compares it against

-2.3

-1.6

-0.7

actual spendinga (%)

(1.5)

(2.3)

(2.5)

Credit score (out of 850)c

-2.2

-0.1

-2.2

(2.0)

(1.7)

(2.3)

Study participant has a credit score greater than or equal to 620c

-1.1

0.4

-1.5

(%)

(0.7)

(0.9)

(1.0)

Student loan balancec ($)

-82

1,037***

-1,119**

(492)

(326)

(534)

Total nonhousing debtc ($)

384

1,078*

-694

(922)

(595)

(1,272)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

1.0

0.5

0.5

(0.6)

(0.9)

(0.8)

Monthly housing costs exceed 30 percent of monthly incomea (%)

0.2

3.2*

-2.9

(1.4)

(1.8)

(2.5)

Ever 30 days delinquentb (%)

0.4

0.5

-0.1

(0.5)

(0.5)

(0.6)

Ever 60 days delinquentb (%)

0.4

0.1

0.3

(0.3)

(0.3)

(0.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 175

APPENDIX

Income

Income

Higher Than 80

Lower Than 80

Percent of Area Percent of Area

Median

Median

(44.6% of Sample) (55.4% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

Exhibit E.14: Comparison of Impacts on Subpopulations Defined by Nonhousing Debt at Baseline Nonhousing Debt Nonhousing Debt

$10,000 or More Less Than $10,000

(49.8% of Sample) (50.2% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

4.1**

4.9***

-0.8

information he/she needed about the homebuying processa (%)

(1.9)

(1.6)

(2.3)

Study participant purchased a homeb (%)

-1.4

1.9

-3.3

(1.9)

(1.5)

(2.8)

Study participant was very satisfied with the homebuying

4.8***

1.1

3.8

processa (%)

(1.7)

(2.5)

(2.8)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

2.0

3.2*

-1.2

credit reporta (%)

(1.9)

(1.8)

(2.8)

If in financial difficulty, study participant would contact his/her

5.8***

4.2**

1.6

counseling agency, consumer credit counseling agency, or other

(1.7)

(1.7)

(2.1)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

2.0

2.7

-0.7

lender for assistance prior to missing a mortgage paymenta (%)

(2.1)

(2.1)

(2.0)

Number of correct answers (out of four) to mortgage literacy quiza

0.02

0.03

-0.01

(0.03)

(0.04)

(0.05)

Study participant has a budget and often compares it against

-2.0

-1.6

-0.3

actual spendinga (%)

(1.9)

(2.1)

(2.8)

Credit score (out of 850)c

-0.1

-2.0

2.0

(1.5)

(2.0)

(2.4)

Study participant has a credit score greater than or equal to 620c

-0.9

1.2

-2.2

(%)

(0.9)

(1.1)

(1.6)

Student loan balancec ($)

1,202**

-56

1,257*

(541)

(188)

(613)

Total nonhousing debtc ($)

1,339

143

1,195

(788)

(457)

(868)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 176

APPENDIX

Nonhousing Debt Nonhousing Debt

$10,000 or More Less Than $10,000

(49.8% of Sample) (50.2% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

0.8

0.5

0.3

(0.9)

(1.0)

(1.4)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.2

2.4

-0.2

(1.9)

(2.2)

(3.4)

Ever 30 days delinquentb (%)

0.2

0.6

-0.4

(0.6)

(0.5)

(0.9)

Ever 60 days delinquentb (%)

0.0

0.3

-0.3

(0.3)

(0.4)

(0.6)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix A details the analytic methods; appendix B provides additional detail on the construction of measures.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 177

APPENDIX

Appendix F: Impacts on Subgroups Defined by Likelihood of Service

Participation

Policymakers, scholars, and practitioners

are interested not only in the impacts on

Key Findings: Impacts on Subgroups

first-time homebuyers of having

Defined by Likelihood of Service

Participation

homebuyer education and counseling

available, but also in the impacts of taking

Characteristics that Predict Service Participation

up that offer (using any service).

• For both in-person and remote service modes,

women and those with higher educational

attainment were more likely to take up and

Throughout this report, we have labeled

complete services.

these two types of impacts as intent-to-

• Those referred to in-person services were more

treat (ITT) and treatment-on-the-treated

likely to participate in services if they were at an

(TOT), respectively. As discussed in

early stage of the homebuying process at the time

of study enrollment. They were also more likely to

chapter 2 and further detailed in appendix

complete the education curriculum if they reported

A, section A.2, a standard adjustment for

being “pretty good at math” or if they planned to

purchase a home without a co-borrower.

no-shows (assuming no impact of the

• Those referred to remote services were more

intervention on them and no crossovers)

likely to participate in services if they planned to

provides the TOT estimate. That estimate

spend more years living in their purchased home;

scored better on a baseline mortgage literacy quiz;

is computed from the experimental ITT

or had a baseline credit score of 740 or higher.

estimate by essentially dividing the ITT

• Race/ethnicity, age, marital status, and household

estimate by the take-up rate.

size were not associated with service participation.

Impacts on Subgroups Defined by Likelihood of

This appendix, however, takes an

Service Participation

alternative approach to considering the

• Chapter 7 reported that homebuyer education and

impact of homebuyer education and

counseling led to increased prevalence of high

counseling by asking the question: What is

monthly housing costs. This appendix findings

reveal that the overal effect is driven by those

the impact of homebuyer education and

least likely to participate in services.

counseling on those most likely to

• Across all four measures of service receipt—took

participate in services? 97 We undertake

up any services, completed the education

curriculum, completed one-on-one counseling,

this alternative analysis because it allows

and completed all homebuyer education and

us to identify an experimental subgroup

counseling services—those treatment group

members who are least likely to participate in

comprising treatment group members who

services are more likely to have high monthly

are “most likely” to participate in services

housing costs relative to their control group

and their control group counterparts who

counterparts. Other than this finding, there are no

differences in impacts associated with service

participation.

97 As detailed here, we consider four separate measures of participation in homebuyer education and counseling services: (1) whether the study participant took up services; (2) whether the study participant completed the education curriculum; (3) whether the study participant completed one-on-one counseling; and (4) whether the study participant completed all homebuyer education and counseling services. We use the term participation to refer to these four measures generally.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 178

APPENDIX

would have been “most likely” to participate in services had they been offered services. Doing so enables us to compute an experimental estimate of the impact on the subgroup of study participants most likely to participate in services.

For this analysis, the idea is to compare outcomes for treatment group members who participated in homebuyer education and counseling services with outcomes for their control group counterparts who would have participated in the services had they been offered services.

However, making this comparison poses an empirical challenge because we do not directly observe who in the control group would have participated in services had they been assigned to the treatment group. That is, because control group members are not offered services, we cannot observe directly whether they would have participated in services had they been offered.

Therefore, we require more complex analytic techniques to conduct this analysis.

The analysis we use is called Analysis of Symmetrically Predicted Endogenous Subgroups (ASPES). In brief, this analysis identifies (using multivariate regression) those in the treatment group who are most likely to participate in services and then uses the resulting model coefficients to identify their counterparts in the control group who would be most likely to participate in services. We then compare outcomes between treatment and control group members within this subgroup of those “most likely” to participate in services.

The estimated impact on those most likely to participate in services (based on the ASPES

analysis) is purely experimental. Similar to the subgroups constructed using baseline characteristics presented in appendix E, the ASPES analysis defines those most likely to participate in services using characteristics that are exogenous to the intervention—meaning that they could not be affected by the offer of the treatment, because they are measured at baseline, before members of the study sample were randomly assigned to receive the treatment offer or not. This results in an experimental estimate of the impact of the offer of homebuyer education and counseling services on the subgroup of study participants who are most likely to participate in services and a complementary estimate of the impact of the offer of services for the subgroup of study participants least likely to participate in services.

This analysis is useful for two reasons: (1) it can identify the types of individuals who are most likely to participate in homebuyer education and counseling services, which might help program targeting; and (2) by providing evidence on the impact the intervention has on those study participants who are most likely to participate in services, it adds to the information provided by the TOT estimate.

The next section offers more detail on the specific ASPES procedures that were applied. ASPES

was used both in this appendix (which creates subgroups based on the likelihood of participation in services) and in appendix G (which creates subgroups based on the likelihood of home purchase).

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 179

APPENDIX

F.1 Method for Analyzing the Role of Service Participation

Given an experimental setting, where study participants are randomly assigned to treatment and control conditions, the expectation is that any subset of one experimental group has a counterpart in the other. For example, treatment group members who complete all services will have counterparts in the control group who would have completed all services had they been offered the treatment. Continuing this example treatment group completers and control group would-be completers will have the same measurable and unmeasurable characteristics, on average, as would any experimentally defined subgroup (Moulton, Peck, and Greeney, 2017). Based on this insight, the ASPES method established in Peck (2003) uses baseline characteristics to construct subgroups with high propensities for a post-randomization event; in this example, that event is the completion of all homebuyer education and counseling services.

As detailed by Moulton, Peck, and Bell (2014), the ASPES method proceeds through the two stages, in which the analysts (1) use baseline characteristics to sort the study sample into subgroups defined by likelihood of participation in services and (2) estimate the impacts on these subgroups. 98 This section provides an overview of the ASPES method. We refer interested readers to published documents to learn more about this methodological approach. 99

Stage 1: Construct Predicted Subgroups

In this stage, we predict which treatment and control group members are in each service participation subgroup of interest based on their baseline characteristics. Following the recommendation of Harvill, Peck, and Bell (2013), we use a cross-validation approach to predict service participation. This method ensures that subgroup membership for every study participant in the sample (both treatment and control) is estimated by the same process (that is,

“symmetrically”), through out-of-sample prediction using baseline characteristics. This 98 A third analytic stage that sometimes is applied converts these estimated impacts on predicted endogenous subgroups to represent impacts on subgroups that actually had the experience of interest. Estimates of impacts on actual endogenous subgroups require additional assumptions. For example, the assumption required to produce unbiased impact estimates on the subgroup of study participants who actually completed the education curriculum (or would have completed the curriculum in the case of the control group) is the following: the baseline characteristics used to predict whether a study participant completed the education curriculum affect impact magnitude only through completion of the education curriculum. In this application, this assumption is implausible because these same baseline characteristics are also related to whether study participants completed one-on-one counseling, and one-on-one counseling could affect impact magnitude. For this reason, we forego this conversion step and examine the impact differences only for the likelihood-defined groups.

99 The ASPES method was established by Peck (2002, 2003) and revisited by Peck (2013). Bell and Peck (2013) further considered the method’s assumptions. Moulton, Peck, and Bell (2014) detailed the analytic steps.

Applications of the method include that by Peck and Bell (2014), who considered the case of Head Start quality, and by Moulton, Peck, and Dillman (2014), who considered the case of neighborhood quality in the Moving to Opportunity Experiment. For more details on the method, see http://aspes.abtassociates.com/.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 180

APPENDIX

symmetric process to prediction using only baseline characteristics allows us to maintain the integrity of the experimental design. 100, 101

We consider four different measures of homebuyer education and counseling service participation, which serve as the outcomes of interest in the first-stage prediction model:

1. Take-up of any homebuyer education and counseling services.

2. Completion of the education curriculum.

3. Completion of one-on-one counseling.

4. Completion of all homebuyer education and counseling services. 102

We include in the prediction model a wide range of baseline characteristics as covariates, which capture demographics, attitudes and beliefs, housing arrangements, financial capability and knowledge, and creditworthiness. We selected measures—in consultation with HUD—that either were relevant to policy or practice or were explicitly hypothesized to have a relationship with service participation. For interested readers, the baseline characteristics included in the prediction model and their estimated relationship with completion of in-person and remote services are documented in detail in Moulton et al. (2018).

Estimating this model produces a predicted probability that each study participant in the sample (both treatment and control) is a member of the subgroup of interest. We then use this predicted probability to determine whether each study participant is a member of the services participation subgroup of interest. For example, when completion of all homebuyer education and counseling services (measure 4) is the outcome in the prediction model, estimating the model allows us to produce a predicted probability that each study participant (both treatment and control) completed (or would have completed, in the case of control group members) all homebuyer 100 In contrast to this “symmetric” approach to prediction (which relies on an “out-of-sample” process), conducting

“in-sample” prediction for the treatment group can create an imbalance between the experimental groups due to the potentially better prediction of service participation in the treatment group relative to the control group.

101 In this application, we implemented the cross-validation approach as follows. Randomly partition the experimental sample (both treatment and control) into 10 groups of equal size. To obtain predictions for group 1, estimate the prediction model on the subsample of treatment group members in groups 2–10 (because we only observe actual service participation in the treatment group). Using the parameters obtained from estimating this prediction model, out-of-sample predict the likelihood of home purchase for both treatment and control individuals in group 1. To obtain out-of-sample predictions for group 2, estimate the prediction model on the subsample of treatment group members in groups 1, 3–10. Using the parameters obtained from estimating this prediction model, predict the likelihood of home purchase for both treatment and control individuals in group 2. Repeat this process for groups 3–10. This process provides each individual in the sample (both treatment and control) with a continuous score that represents their probability of purchasing a home based on their baseline characteristics.

102 The indicators for whether the study participant took up or completed services were defined using administrative data from eHome America, ClearPoint, and local housing counseling agencies.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 181

APPENDIX

education and counseling services. We then convert this predicted probability into a binary indicator that divides the full study sample into two subgroups of interest: a subgroup of treatment and control group members predicted to complete all services and a complementary subgroup predicted not to complete all services. The cut-point for dividing the sample into two subgroups was chosen so that the subgroups of treatment and control group members considered most likely to complete all services matches the share of treatment group members that actually did complete all services. We use a similar approach to construct the subgroups defined by service take-up, completion of the education curriculum, and completion of one-on-one counseling (measures 1, 2, and 3). Each of these subgroups is constructed using baseline, exogenous characteristics and, therefore, is experimentally valid.

Although the impact estimates presented in this appendix are experimental, it is important to note that, due to imperfect prediction, not everyone in a given most-likely-to-participate-in-services subgroup actually participated in services and not everyone in the corresponding least-likely-to-participate-in-services subgroup actually did not. For instance, among treatment group members in the most-likely-to-take-up-services subgroup, 68.5 percent took up services. Among treatment group members in the least-likely-to-take-up-services subgroup, 41.8 percent took up services.

The implications of this imperfect prediction are discussed further in section F.7.

Stage 2: Estimate Impacts on Predicted Subgroups

In Stage 2, the impact of the intervention is estimated for each predicted subgroup constructed in Stage 1. We estimate subgroup impacts using the same “interaction model” used to produce the subgroup estimates reported in appendix E. As described in appendix A, each set of subgroup impacts is estimated in a separate “interaction model,” where the treatment indicator is interacted with a subgroup identifier, and the coefficient on this interaction term provides an estimate of the difference in impacts between subgroups. Because the subgroups are defined by exogenous factors, the impact estimate can be interpreted as experimentally valid.

F.2 What Characteristics Predict Participation in Services?

A prediction model with the same set of baseline covariates was used by Moulton et al. (2018) to explore whether a wide range of measures related to study participants’ demographics, attitudes and beliefs, housing arrangements, financial capability and knowledge, and creditworthiness predict participation in in-person and remote homebuyer education and counseling services. 103

Across both in-person and remote service modes, Moulton et al. found the following—

103 Moulton et al. (2018) estimate the prediction model separately for those offered in-person services and those offered remote services, and the results of this analysis are presented in this section. In contrast, the ASPES analysis used to construct subgroups defined by service participation includes all study participants offered services (both in-person and remote) in the prediction model. This allows for larger sample size (and therefore statistical precision) for estimating subgroup effects presented in later sections of this appendix.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 182

APPENDIX

• Women were more likely to participate in homebuyer education and counseling services.

This finding is in line with our prior hypothesis that women might be more likely to participate, because they want to try to offset perceived or actual discrimination in financial markets or because women are more willing than men to ask for help.

• Those with relatively greater education were more likely to participate in homebuyer education and counseling services. Our prior hypotheses posited an explanation for either direction of effect. The results provide support for the hypothesis that those with greater education are more likely to participate, perhaps because of their being “education seekers.”

• Race or ethnicity, age, marital status, and household size were not statistically significant predictors of participation in homebuyer education and counseling services.

Considering in-person services specifically:

• Those referred to in-person services were more likely to participate in homebuyer education and counseling services if they were at an early stage of the homebuying process at the time of study enrollment. They also were more likely to complete the education curriculum if they reported being “pretty good at math” or if they planned to purchase a home without a co-borrower. We do not find that these same traits were associated with participation in remote services.

Considering remote services specifically:

• Those referred to remote services were more likely to participate in services if they planned to spend more years living in their purchased home, scored better on a baseline mortgage literacy quiz, or had a baseline credit score of 740 or higher. We do not find evidence that these same traits were associated with participation in in-person services.

Agencies providing homebuyer education and counseling services might find these findings informative to their messaging, outreach, and approach to providing services. For example, that certain traits are associated with participation in services could inform the way agencies decide to market and provide their services to meet potential clients’ interests, preferences, and needs.

F.3 What Is the Impact on Those Most Likely to Take Up Any Services?

This section describes the impact of homebuyer education and counseling services on those most likely to take up services (55.1 percent of the study sample) and a complementary subgroup comprising those least likely to take up services (44.9 percent of the study sample).

The impact estimates for the full sample of treatment and control groups found that homebuyer education and counseling services led to the somewhat puzzling finding that the offer of homebuyer education and counseling led to increased “high” monthly housing costs relative to income, as measured by the share of study participants who spent more than 30 percent of their monthly income on housing. The ASPES estimates indicate that this effect is driven by those least likely to take up homebuyer education and counseling services (exhibit F.1). Among those least likely to take up services, the offer of homebuyer education and counseling services is The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 183

APPENDIX

associated with a 1.7-percentage point increase in the share of monthly income spent on housing.

In addition, among those least likely to take up services, treatment group members were 6.1

percentage points more likely to spend more than 30 percent of their monthly income on housing relative to their control group counterparts. In contrast, we find no evidence that homebuyer education and counseling had an effect on these measures for those most likely to take up services.

Despite observing some within-subgroup impacts, there is no detectable evidence of between-subgroup differences in impacts on other outcomes.

Exhibit F.1: Comparison of Impacts on Subpopulations Defined by Likelihood of Taking Up Any Service Most Likely to

Least Likely to

Take Up Services Take Up Services

(55.1% of Sample) (44.9% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

5.9***

4.5*

1.4

information he/she needed about the homebuying processa (%)

(1.9)

(2.6)

(3.0)

Study participant purchased a homeb (%)

-0.6

1.9

-2.5

(1.6)

(1.6)

(2.6)

Study participant was very satisfied with the homebuying

2.5

2.5

-0.0

processa (%)

(1.8)

(2.4)

(3.0)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

3.7**

3.1

0.6

credit reporta (%)

(1.6)

(1.9)

(2.9)

If in financial difficulty, study participant would contact his/her

5.5***

3.1**

2.5

counseling agency, consumer credit counseling agency, or other

(1.6)

(1.5)

(1.9)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

3.8**

1.6

2.2

lender for assistance prior to missing a mortgage paymenta (%)

(1.6)

(2.5)

(2.6)

Number of correct answers (out of four) to mortgage literacy

0.00

0.09

-0.08

quiza

(0.04)

(0.05)

(0.08)

Study participant has a budget and often compares it against

-0.1

-3.8

3.7

actual spendinga (%)

(2.0)

(2.6)

(3.5)

Credit score (out of 850)c

-1.5

-0.8

-0.7

(2.0)

(1.9)

(2.6)

Study participant has a credit score greater than or equal to 620c

-0.4

0.4

-0.8

(%)

(1.0)

(1.2)

(1.7)

Student loan balancec ($)

530

940**

-410

(450)

(350)

(532)

Total nonhousing debtc ($)

699

1,107*

-408

(646)

(637)

(863)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

0.1

1.7**

-1.6*

(0.8)

(0.7)

(0.9)

Monthly housing costs exceed 30 percent of monthly incomea

-1.2

6.1***

-7.3**

(%)

(2.1)

(2.0)

(3.4)

Ever 30 days delinquentb (%)

0.5

0.1

0.4

(0.5)

(0.6)

(0.8)

Ever 60 days delinquentb (%)

0.3

0.1

0.2

(0.2)

(0.4)

(0.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 184

APPENDIX

Most Likely to

Least Likely to

Take Up Services Take Up Services

(55.1% of Sample) (44.9% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix B provides additional detail on the construction of measures.

F.4 What Is the Impact on Those Most Likely to Complete the Education

Curriculum?

This section describes the impact of homebuyer education and counseling services on those most likely to complete the education curriculum (31.0 percent of the study sample) and a complementary subgroup comprising those least likely to complete the education curriculum (69.0 percent of the study sample).

Among those least likely to complete the education curriculum, the offer of homebuyer education and counseling services is associated with a 1.4-percentage point increase in the share of monthly income spent on housing. Additionally, treatment group members who were least likely to complete the education curriculum were 3.8 percentage points more likely to spend more than 30 percent of their monthly income on housing relative to their control group counterparts. 104

Similar to the findings presented in section F.3, the estimates presented in exhibit F.2 indicate that the increased prevalence of high housing costs among treatment group members is driven by those least likely to complete the education curriculum.

Despite observing some within-subgroup impacts, there is no detectable evidence of between-subgroup differences in impacts on other outcomes.

104 In contrast, among those most likely to complete the education curriculum, treatment group members were 1.7

percentage points less likely to experience these high housing costs. Though this impact is not statistically different from zero, it is statistically different from the impact on the subgroup predicted to not complete the education curriculum.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 185

APPENDIX

Exhibit F.2: Comparison of Impacts on Subpopulations Defined by Likelihood of Completing Education Curriculum

Most Likely to

Least Likely to

Complete

Complete

Education

Education

Curriculum

Curriculum

(31.0% of Sample) (69.0% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

6.0**

4.3**

1.7

information he/she needed about the homebuying processa (%)

(2.4)

(1.9)

(2.9)

Study participant purchased a homeb (%)

-1.7

1.5

-3.2

(2.0)

(1.2)

(2.2)

Study participant was very satisfied with the homebuying

3.5

2.7*

0.8

processa (%)

(2.7)

(1.5)

(2.9)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

1.8

3.5**

-1.7

credit reporta (%)

(2.3)

(1.6)

(3.4)

If in financial difficulty, study participant would contact his/her

5.5**

3.8**

1.7

counseling agency, consumer credit counseling agency, or other

(2.1)

(1.5)

(2.7)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

1.7

3.0

-1.3

lender for assistance prior to missing a mortgage paymenta (%)

(1.8)

(2.2)

(2.7)

Number of correct answers (out of four) to mortgage literacy quiza

-0.03

0.05*

-0.08

(0.04)

(0.03)

(0.06)

Study participant has a budget and often compares it against

-0.1

-2.5

2.4

actual spendinga (%)

(3.0)

(1.7)

(3.4)

Credit score (out of 850)c

1.2

-1.9

3.1

(2.6)

(1.3)

(2.4)

Study participant has a credit score greater than or equal to 620c

1.6

-0.9

2.5*

(%)

(1.1)

(0.7)

(1.3)

Student loan balancec ($)

390

637

-248

(810)

(398)

(1,038)

Total nonhousing debtc ($)

-74

1,233**

-1,306

(977)

(523)

(1,202)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

-0.3

1.2

-1.6

(0.8)

(0.8)

(1.0)

Monthly housing costs exceed 30 percent of monthly incomea (%)

-1.5

3.7**

-5.3*

(2.3)

(1.4)

(2.9)

Ever 30 days delinquentb (%)

0.3

0.4

-0.0

(0.6)

(0.4)

(0.7)

Ever 60 days delinquentb (%)

-0.0

0.3

-0.3

(0.3)

(0.3)

(0.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 186

APPENDIX

Most Likely to

Least Likely to

Complete

Complete

Education

Education

Curriculum

Curriculum

(31.0% of Sample) (69.0% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix B provides additional detail on the construction of measures.

F.5 What Is the Impact on Those Most Likely to Complete One-on-One

Counseling?

This section describes the impact of homebuyer education and counseling services on those most likely to complete one-on-one counseling (37.3 percent of the study sample) and a complementary subgroup comprising those least likely to complete one-on-one counseling (62.7

percent of the study sample). Although we find that those least likely to complete one-on-one counseling were 3.5 percentage points more likely to have “high” monthly housing costs, this impact is not statistically different from the impact on the subgroup predicted to complete one-on-one counseling. Similarly, we do not observe any between-subgroup differences in impacts on other outcomes.

Exhibit F.3: Comparison of Impacts on Subpopulations Defined by Likelihood of Completing One-on-One Counseling

Most Likely to

Least Likely to

Complete One-on- Complete One-on-

One Counseling One Counseling

(37.3% of Sample) (62.7% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

5.1**

5.3**

-0.2

information he/she needed about the homebuying processa (%)

(2.0)

(2.0)

(2.6)

Study participant purchased a homeb (%)

-0.9

1.7

-2.5

(2.2)

(1.3)

(2.8)

Study participant was very satisfied with the homebuying

2.4

3.4*

-1.0

processa (%)

(3.2)

(1.8)

(4.2)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

3.8**

3.2**

0.6

credit reporta (%)

(1.7)

(1.4)

(2.4)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 187

APPENDIX

Most Likely to

Least Likely to

Complete One-on- Complete One-on-

One Counseling One Counseling

(37.3% of Sample) (62.7% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Of ered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

If in financial difficulty, study participant would contact his/her

6.0***

3.3**

2.7

counseling agency, consumer credit counseling agency, or other

(1.8)

(1.3)

(1.8)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

3.5

2.3

1.2

lender for assistance prior to missing a mortgage paymenta (%)

(2.1)

(2.1)

(2.9)

Number of correct answers (out of four) to mortgage literacy quiza

-0.03

0.08**

-0.11*

(0.04)

(0.03)

(0.06)

Study participant has a budget and often compares it against

1.1

-3.0

4.1

actual spendinga (%)

(2.4)

(2.1)

(3.2)

Credit score (out of 850)c

-0.5

-1.1

0.6

(2.5)

(1.4)

(2.6)

Study participant has a credit score greater than or equal to 620c

-0.4

0.3

-0.7

(%)

(1.0)

(0.8)

(1.2)

Student loan balancec ($)

1,034**

416

618

(471)

(335)

(475)

Total nonhousing debtc ($)

336

1,268**

-932

(636)

(601)

(868)

Panel C: Sustainable Homeownership

Monthly-housing-costs to monthly-income ratioa

0.4

0.9

-0.5

(1.1)

(0.9)

(1.5)

Monthly housing costs exceed 30 percent of monthly incomea (%)

-0.4

3.5*

-4.0

(2.2)

(1.7)

(3.1)

Ever 30 days delinquentb (%)

-0.3

0.6

-1.0

(0.5)

(0.5)

(0.7)

Ever 60 days delinquentb (%)

-0.1

0.3

-0.4

(0.2)

(0.3)

(0.3)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix B provides additional detail on the construction of measures.

F.6 What Is the Impact on Those Most Likely to Complete Al Services?

This section describes the impact of homebuyer education and counseling services on those most likely to complete all homebuyer education and counseling services (25.3 percent of the study sample) and a complementary subgroup comprising those least likely to complete all homebuyer education and counseling services (74.7 percent of the study sample).

The estimates presented in exhibit F.4 indicate that among those least likely to complete all services, treatment group members were 3.7 percentage points more likely to spend more than 30

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 188

APPENDIX

percent of their monthly income on housing relative to their control group counterparts. In contrast, among those most likely to complete all services, treatment group members were 2.4

percentage points less likely to experience these high housing costs. 105

Despite observing some within-subgroup impacts, there is no detectable evidence of between-subgroup differences in impacts on other outcomes.

Exhibit F.4: Comparison of Impacts on Subpopulations Defined by Likelihood of Completing Al Services Most Likely to

Least Likely to

Complete All

Complete All

Services

Services

(25.3% of Sample) (74.7% of Sample)

Difference in the

Impact of Being Impact of Being

Impact of Being

Of ered Services Of ered Services Offered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

5.7**

4.8**

1.0

information he/she needed about the homebuying processa (%)

(2.6)

(1.7)

(2.9)

Study participant purchased a homeb (%)

-1.9

1.4

-3.3

(2.5)

(1.1)

(2.8)

Study participant was very satisfied with the homebuying

2.3

3.1**

-0.7

processa (%)

(3.2)

(1.3)

(3.2)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

3.8

2.8*

1.0

credit reporta (%)

(2.6)

(1.5)

(3.6)

If in financial difficulty, study participant would contact his/her

5.8**

3.8***

1.9

counseling agency, consumer credit counseling agency, or other

(2.4)

(1.1)

(2.3)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

4.8**

1.8

3.0

lender for assistance prior to missing a mortgage paymenta (%)

(2.2)

(1.9)

(2.7)

Number of correct answers (out of four) to mortgage literacy quiza

-0.02

0.05*

-0.07

(0.04)

(0.03)

(0.05)

Study participant has a budget and often compares it against

2.3

-2.8*

5.1

actual spendinga (%)

(3.4)

(1.6)

(3.9)

Credit score (out of 850)c

3.0

-2.3*

5.3**

(2.7)

(1.2)

(2.5)

Study participant has a credit score greater than or equal to 620c

1.6

-0.7

2.3

(%)

(1.3)

(0.6)

(1.4)

Student loan balancec ($)

694

539

156

(732)

(325)

(837)

Total nonhousing debtc ($)

-20

1,165**

-1,184

(873)

(467)

(984)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

-0.8

1.3

-2.1*

(1.0)

(0.8)

(1.2)

Monthly housing costs exceed 30 percent of monthly incomea (%)

-2.4

3.7**

-6.1*

(2.4)

(1.5)

(3.0)

105 Although this impact is not statistically different from zero, it is statistically different from the impact on the subgroup predicted to not complete all homebuyer education and counseling services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 189

APPENDIX

Most Likely to

Least Likely to

Complete All

Complete All

Services

Services

(25.3% of Sample) (74.7% of Sample)

Difference in the

Impact of Being Impact of Being

Impact of Being

Of ered Services Of ered Services Offered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Ever 30 days delinquentb (%)

-0.3

0.5

-0.8

(0.8)

(0.4)

(0.8)

Ever 60 days delinquentb (%)

-0.2

0.3

-0.5

(0.2)

(0.3)

(0.4)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix B provides additional detail on the construction of measures.

F.7 Discussion

This appendix presents impacts of homebuyer education and counseling services on those most likely (or least likely) to participate in homebuyer education and counseling services. As described earlier, the impacts on these subgroups of those most (or least) likely to take up or complete services are experimental impacts because subgroups are symmetrically constructed for the treatment and control groups using only baseline characteristics (Harvill, Peck, and Bell, 2013).

Although the impact estimates presented in this appendix are experimental, it is important to note that, due to imperfect prediction, they reflect a blend of participants and nonparticipants: that is, not everyone in a given most-likely-to-participate-in-services subgroup actually participated in services, and not everyone in the corresponding least-likely-to-participate-in-services subgroup actually did not. For instance, among treatment group members in the most-likely-to-take-up-services subgroup, 68.5 percent took up services. Among treatment group members in the least-likely-to-take-up-services subgroup, 41.8 percent took up services. The implication of this imperfect prediction is that both the most-likely-to-participate subgroup and the least-likely-to-participate subgroup contain a mix of study participants who did and did not participate in services. Given this, the findings speak to the effect homebuyer education and counseling has on the set of study participants with characteristics that make them most likely to participate in services (as opposed to the set of study participants that actually participate in services). As such, the findings are most relevant to program targeting of people most likely to participate.

Across all four measures of service receipt—took up any services, completed the education curriculum, completed one-on-one counseling, and completed all homebuyer education and counseling services—we find that those treatment group members who are least likely to The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 190

APPENDIX

participate in services are more likely to have high monthly housing costs relative to their control group counterparts. In chapter 7, we found that homebuyer education and counseling services led to increased prevalence of high monthly housing costs. The findings presented in this appendix indicate that that overall effect is driven by those least likely to take up or complete services.

Why would treatment group members who are least likely to complete homebuyer education and counseling services experience increased prevalence of high monthly housing costs? One explanation is that the offer of homebuyer education and counseling services gave service noncompleters a false sense of confidence in their ability to gauge how much housing is affordable given their financial situation. For instance, perhaps service noncompleters looked into the contents of the homebuyer education and counseling services and made the determination that they knew enough to go forth in the homebuying process without homebuyer education and counseling services but also without doing additional independent research on how much housing they could afford. In contrast, their control group counterparts—who would have been least likely to take up or complete homebuyer education and counseling services had the services been offered—might have felt compelled to conduct more independent research to determine what amount of housing is affordable, which led them to make a more appropriate housing choice given their income.

Alternatively, perhaps the offer of homebuyer education and counseling services encouraged treatment group members to trust professionals in the industry, even if they do not complete services. Many noncompleters noted that completing services was inconvenient (for example, because of scheduling conflicts) and, as an alternative to services, they were motived to simply trust the lender or realtor, which led them to consume as much home as the lender would allow, rather than an amount of housing that is affordable given their financial situation. Their control group counterparts, who were not offered services, might have been more skeptical of industry professionals and were motived to conduct independent research to determine what they could afford.

In sum, this analysis flags one difference—as just discussed—between groups that were likely versus not likely to participate. Instead, what characterizes most of this appendix’s results is a lack of differences between the most- and least-likely-to-participate subgroups. Possible reasons for this include that there are no meaningful differential impacts across subgroups or that the sample size is too small to detect meaningful differential impacts.

This appendix moves beyond simple take-up of services as is provided in the report’s TOT

impact estimates. Instead, it considers the impact homebuyer education and counseling had on those who were most likely to participate in services. Although we might hypothesize that homebuyer education and counseling would have a different impact on likely completers, because they are those most fully engaged in the intervention, findings are generally similar to those most likely to simply take up services. This indicates that take-up of services is just as important as completion of services. Because of this observation, and because the TOT estimates are based on a familiar approach, in the main text, we place greater emphasis on the TOT results.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 191

APPENDIX

An exception is the impact on housing cost relative to income, where these ASPES estimates help clarify a puzzling finding.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 192

APPENDIX

Appendix G: Impacts on Subgroups Defined by Likelihood of Home

Purchase

Policymakers, scholars, and

practitioners might wonder whether

Key Findings: Impacts on Subgroups

different—perhaps larger—impacts of

Defined by Likelihood of Home Purchase

homebuyer education and counseling

This appendix identifies the types of individuals who are

occur on those who purchase a home

most likely to purchase a home as of followup, and the

relative to those who do not. In

impact the intervention has on those study participants

who are most likely (and least likely) to purchase a home.

response, this appendix addresses the

What Characteristics Predict Home Purchase as of

research question: What is the impact of

Fol owup?

homebuyer education and counseling on

A number of baseline measures of financial and credit

those most likely to purchase a home?

characteristics, housing arrangements, and demographic

To address this question we use the

characteristics are statistically significant predictors of

whether a study participant purchased a home as of

Analysis of Symmetrically Predicted

followup. Specifically:

Endogenous Subgroups (ASPES)

• Both employment and income are predictors of home

method to create subgroups defined by

purchase: those who were employed part-time at

baseline and those with annual incomes of less than

their likelihood to purchase a home.

$25,000 were less likely to purchase a home; and

This analysis results in an experimental

those with a credit score less than 620 were less

likely to purchase a home.

estimate of the impact on the subgroup

of study participants (both treatment and • Those who reported that they set aside extra money for retirement, education, or to build a financial

control) who are most likely to purchase

cushion were more likely to purchase a home.

a home and a complementary estimate

Similarly, those who searched for information about

the mortgage loans available from multiple lenders

of the impact on the subgroup of study

were more likely to purchase a home.

participants (again, both treatment and

• Not surprisingly, study participants who were at later

control) least likely to purchase a home

stages in the homebuying process at baseline (that

as of the 12–18 month followup.

is, made an offer, signed a purchase agreement, or

had already purchased a home) were more likely to

have purchased a home as of followup. Additionally,

Ideally, for this analysis we would

those who planned to live in their purchased home for

compare outcomes for control group

6 or more years were more likely to purchase a home

relative to those who planned to live in their

members who purchased a home versus

purchased home for less than 6 years.

their treatment group counterparts who

• We find that certain demographic characteristics—

would have purchased a home even if

namely, race/ethnicity, marital status, and

educational attainment—are also predictors of

they were not offered homebuyer

whether the study participant purchased a home as

education and counseling services.

of the study’s followup. We found no evidence that

However, making this comparison poses

homebuyer education and counseling had a different

impact on those most likely to purchase a home

an empirical challenge because we do

relative to those least likely to purchase a home.

not directly observe who in the

• This finding implies that there is no evidence that

treatment group would have purchased a

agencies providing homebuyer education and

counseling would produce greater impacts if they

home had they been assigned to the

targeted their services at those most likely to

control group. In response, we use the

purchase a home.

ASPES method to create experimental

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 193

APPENDIX

subgroups defined by their likelihood to purchase a home. (Appendix F describes the use of ASPES to create experimental subgroups defined by their likelihood of participation in homebuyer education and counseling services.)

This analysis might be of interest to policymakers for two reasons: (1) it can identify the types of individuals who are most likely to purchase a home, which might help program targeting, in that agencies may find it beneficial to devote their limited resources to those clients who are most likely to purchase a home; and (2) it can identify the impact on those study participants who are most likely to purchase and therefore, the impact on outcomes that are particularly relevant to home purchasers such as loan performance.

G.1 Method for Analyzing the Role of Home Purchase

The ASPES method proceeds in the following two stages: the analysts (1) use baseline characteristics to sort the study sample into subgroups defined by likelihood of home purchase; and (2) estimate the impacts on these subgroups. 106

Stage 1: Construct Predicted Subgroups

In this stage, we predict which treatment and control group members are in each subgroup of interest, most likely to purchase a home and least likely to purchase a home, based on their baseline characteristics. Following the recommendation of Harvill, Peck, and Bell (2013), we use a cross-validation approach to predict service participation. This method ensures that subgroup membership for every study participant in the sample (both treatment and control) is estimated by the same process (that is, “symmetrically”); through out-of-sample prediction using baseline characteristics. This symmetric process to prediction using only baseline characteristics allows us to maintain the integrity of the experimental design. 107

In this application of ASPES, we predict the probability of home purchase for control group members, where the prediction model includes a wide range of baseline characteristics as covariates, which capture demographics, attitudes and beliefs, housing arrangements, financial capability and knowledge, and creditworthiness. Estimating the prediction model for the control group allows us to construct subgroups defined by their likelihood of home purchase in absence of the intervention. We then use the model coefficients, estimated through a cross-validation 106 For more details on the method, see http://aspes.abtassociates.com/.

107 In contrast to this “symmetric” approach to prediction (which relies on an “out-of-sample” process), conducting

“in-sample” prediction for the control group can create an imbalance between the experimental groups due to the potentially better prediction of service participation in the control group relative to the treatment group.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 194

APPENDIX

approach, to assign all study participants in the sample an out-of-sample predicted probability of home purchase as of followup. 108

We then convert this continuous measure of predicted probability into a binary indicator that divides the full study sample into two subgroups of interest: a subgroup of treatment and control group members predicted to purchase a home in absence of being offered intervention services and a complementary subgroup of treatment and control group members predicted not to purchase a home. The cut-point for dividing the sample into these two subgroups was chosen such that the share of the sample in the subgroup predicted to purchase a home matched the share of all control group members who purchased a home (because the goal is to construct subgroups defined by their likelihood of home purchase in absence of the intervention).

Although the impact estimates presented in this appendix are experimental, it is important to note that, due to imperfect prediction, not everyone in a given most-likely-to-purchase-a-home subgroup actually purchased a home (and not everyone in the corresponding least-likely-to-purchase-a-home subgroup actually did not). For instance, 21 percent of control group members who were predicted to purchase a home did not actually do so. Conversely, 34 percent of control group members who were predicted to not purchase a home actually did. The implications of this imperfect prediction are discussed further in section G.4.

Stage 2: Estimate Impacts on Predicted Subgroups

In Stage 2, the impact of the intervention is estimated for the subgroup of study participants likely to purchase a home and the subgroup of study participants not likely to purchase a home.

We estimate subgroup impacts using the same “interaction model” used to produce the subgroup estimates reported in appendix E. As described in appendix A, each set of subgroup impacts is estimated in a separate “interaction model,” where the treatment indicator is interacted with a subgroup identifier, and the coefficient on this interaction term provides an estimate of the difference in impacts between subgroups.

108 In this application, we implemented the cross-validation approach as follows. Randomly partition the experimental sample (both treatment and control) into 10 groups of equal size. To obtain predictions for group 1, estimate the prediction model on the subsample of control group members in groups 2–10 (estimating the prediction model for control group members enables us to define subgroups defined by their likelihood of home purchase in absence of the intervention). Using the parameters obtained from estimating this prediction model, out-of-sample predict the likelihood of home purchase for both treatment and control individuals in group 1. To obtain out-of-sample predictions for group 2, estimate the prediction model on the subsample of control group members in groups 1 and 3–10. Using the parameters obtained from estimating this prediction model, predict the likelihood of home purchase for both treatment and control individuals in group 2. Repeat this process for groups 3–10. This process provides each individual in the sample (both treatment and control) with a continuous score that represents their probability of purchasing a home based on their baseline characteristics.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 195

APPENDIX

G.2 What Characteristics Predict Home Purchase?

Exhibit G.1 reports how baseline characteristics relate to study participants’ home purchase status as of followup (12–18 months after random assignment).

Unsurprisingly, financial and credit characteristics, which align with lender underwriting standards, predict home purchase. For example, both employment and income are predictors of home purchase, as those who were employed part-time at baseline and those with annual incomes of less than $25,000 were less likely to purchase a home. Similarly, those with a credit score less than 620 were less likely to purchase a home relative to those with a credit score of 740 or more. These results reflect mortgage underwriting standards at the time. During the study period, few mortgages were made to borrowers with scores below 620, reflecting a tightening of lending standards after the 2008 financial crisis. Lenders and secondary market actors put in place credit score floors at or above 620 during this period. In addition, lenders increased scrutiny of employment: new standards required borrowers to prove longer periods of steady income.

Exhibit G.1: Predictors of Home Purchase for Control Group

Purchased a Home

as of Followupa

Demographic Measures

Race/Ethnicity (reference: non-Hispanic White)

Hispanic

–

Black/African American

–

Other race

Male (reference: female)

Age (reference: under 30)

30 to 45

46 or older

Marital Status (reference: single and never married)

Married

+

Divorced, widowed, or separated

Household Size (reference: one)

Two

Three or four

Five or more

Plans to have children in purchased home

Education (reference: bachelor’s degree or higher)

Associate’s degree

–

Some college, but no degree

High school diploma or less

–

Attitudes and Beliefs (respondent strongly agreed with the statement)

As a student, I enjoyed going to school

–

I usually spend a lot of time planning for large purchases

It is easy for me to stick to and accomplish my goals

I am pretty good at math

Housing Arrangements

Satisfied with pre-purchase housing arrangements

Made an offer, signed a purchase agreement, or purchased a home (reference: early stage)

+

Years Planned to Live in Purchased Home (reference: less than 6 years)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 196

APPENDIX

Purchased a Home

as of Followupa

6 to 10 years

+

11 years or more

+

Measures of Financial Capability and Knowledge

Uses a written budget

Usually pays credit card balance in full to avoid interest charges

Over the past year, was short on money sometimes or often

Sets aside extra money for retirement, education, or to build a financial cushion sometimes or often

+

Searched for information about the mortgage loans available from multiple lenders

+

Number of correct answers (out of four) to mortgage literacy quiz

Would rather receive 40 dollars now than 50 dollars a month from now

Measures of Creditworthiness

Employment (reference: full-time employment (30+ hours per week))

Part-time employment (1–29 hours per week)

–

Not currently employed

Plans to purchase the home with a co-borrower

Income Received by Participant and Any Co-Borrowers in Last 12 Months (reference: $75,000 or more)

$24,999 or less

–

$25,000 to $49,999

$50,000 to $74,999

Credit Score (reference category: 740 or more)

Less than 620

–

620 to 679

680 to 739

Had mortgage loan application denied by another lender

Previously lost a home or other property through foreclosure

Cash on hand for downpayment and closing costs (in thousands)

Monthly payment nonhousing debt (in hundreds)

Given Choice of In-Person or Remote Services?

Choice treatment group indicator

Summary Information

Sample size

2,425

R-squared

0.254

a The “+” symbol indicates that the baseline characteristic has a statistically significant positive effect on home purchase at the p<0.05 significance level. The “–” symbol indicates that the baseline characteristic has a statistically significant negative effect on home purchase at the p<0.05 significance level.

Notes: The sample includes all control group members with nonmissing data on whether they purchased a home as of followup.

Study participants who withdrew from the study are excluded. Regression includes a set of site fixed effects. Standard errors are clustered at the site level. Missing values for baseline characteristic data are coded as the reference category for binary or categorical measures and are coded as the sample mean for continuous measures.

Sources: Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. Data on baseline characteristics are from the baseline survey of study participants and credit bureau data

We also find that select measures of financial capability and knowledge are predictors of whether the study participant purchased a home as of followup. Specifically, we find that those who reported that they set aside extra money for retirement, education, or to build a financial cushion were more likely to purchase a home. Perhaps those more likely to report savings behaviors such as these feel more confident in their ability to sustain homeownership.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 197

APPENDIX

Similarly, those who searched for information about the mortgage loans available from multiple lenders were more likely to purchase a home. A more exhaustive search for the best financing might indicate that the study participant was more serious about purchasing a home. It is also possible that borrowers who were turned down by one lender became discouraged or didn’t believe they would be successful at obtaining a mortgage and, as a result, did not reach out to multiple lenders.

Not surprisingly, study participants who were at later stages in the homebuying process at baseline (that is, made an offer, signed a purchase agreement, or had already purchased a home) were more likely to have purchased a home as of followup relative to study participants who were at earlier stages in the homebuying process.

Additionally, those who planned to live in their purchased home for 6 or more years were more likely to purchase a home relative to those who planned to live in their purchased home for less than 6 years. This suggests that at least some individuals understand that homeownership is less risky if one plans on being in the home long enough to wait out any short-term home price drops and to recoup the transactional costs of home purchase. Perhaps those with longer planning horizons prefer owning to renting, which might reflect that such individuals feel more confident in the stability of their job or a stronger commitment to live in particular place.

In addition to these characteristics, which we would expect to be associated with home purchase, we find that certain demographic characteristics—namely, race/ethnicity, marital status, and educational attainment—are also predictors of whether the study participant purchased a home as of the study’s followup. 109 We find that Hispanics and African-Americans were less likely to purchase a home relative to non-Hispanic Whites. Study participants who were married at baseline were more likely to purchase a home relative to those single and never married at baseline. Those with an associate’s degree and those with a high school diploma or less were less likely to purchase a home relative to those with a bachelor’s degree or higher at baseline.

The findings related to which demographic characteristics predict home purchase might reflect differences in preferences (for example, marriage is a well-established “trigger event” for purchasing a home). They could also be capturing other, correlated factors that are not directly measured. For example, those with higher education tend to have more stable employment, an important underwriting consideration for lenders. Discrimination against African-American and Hispanic prospective homebuyers could also contribute to disparate purchase behavior.

Finally, demographic characteristics might be associated with housing market characteristics that could affect the likelihood that a study participant purchases a home. For example, non-White prospective homebuyers might be concentrated in “tighter” housing markets compared to White 109 In contrast, gender, age, household size, and whether the study participant planned to have children living in the purchased home were not predictors of home purchase.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 198

APPENDIX

homebuyers. Married couples might have been in the market for a different kind of home or a home in a different type of neighborhood compared to nonmarried potential buyers. Housing markets vary greatly in availability and affordability even within the same metropolitan region.

G.3 What Is the Impact on Those Most Likely to Purchase a Home?

This section describes the impact of homebuyer education and counseling services on those most likely to purchase a home (61.6 percent of the study sample) and a complementary subgroup comprising those least likely to purchase a home (38.4 percent of the study sample). Despite observing some within-subgroup impacts, there are no detectable between-subgroup differences in impacts on any outcomes. That is, if the subgroups do not differ, then our best estimate is the overall impact. For this reason, we generally do not focus on impacts that are not statistically different across subgroups, both in this appendix as well as in the rest of the report’s subgroup analyses.

Exhibit G.2: Comparison of Impacts on Subpopulations Defined by Likelihood of Purchasing a Home Most Likely to

Least Likely to

Purchase a Home Purchase a Home

(61.6% of Sample) (38.4% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Offered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Panel A: Preparedness and Search

Study participant was confident that he/she could find the

5.1***

4.5

0.6

information he/she needed about the homebuying processa (%)

(1.6)

(2.7)

(2.9)

Study participant purchased a homeb (%)

1.0

-0.4

1.4

(1.1)

(1.8)

(2.0)

Study participant was very satisfied with the homebuying

3.4*

2.5

0.8

processa (%)

(1.8)

(2.2)

(2.8)

Panel B: Financial Capability

Study participant knows how to correct inaccurate information in

2.8**

3.2

-0.4

credit reporta (%)

(1.2)

(2.0)

(2.5)

If in financial difficulty, study participant would contact his/her

5.4***

2.7

2.8

counseling agency, consumer credit counseling agency, or other

(1.8)

(1.8)

(2.8)

nonprofit organization for assistance prior to missing a mortgage

paymenta (%)

If in financial difficulty, study participant would contact his/her

3.6

1.0

2.6

lender for assistance prior to missing a mortgage paymenta (%)

(2.1)

(2.4)

(3.1)

Number of correct answers (out of four) to mortgage literacy quiza

0.03

0.02

0.01

(0.03)

(0.05)

(0.06)

Study participant has a budget and often compares it against

-1.9

-1.7

-0.2

actual spendinga (%)

(1.7)

(2.5)

(2.8)

Credit score (out of 850)c

-1.0

-1.0

0.0

(1.6)

(2.6)

(3.0)

Study participant has a credit score greater than or equal to 620c

-0.5

0.5

-1.0

(%)

(0.8)

(1.5)

(1.9)

Student loan balancec ($)

492

648

-156

(428)

(425)

(646)

Total nonhousing debtc ($)

657

1,029

-372

(647)

(698)

(1,052)

Panel C: Sustainable Homeownership

Monthly housing costs to monthly income ratioa

0.9

0.5

0.4

(0.7)

(1.0)

(1.1)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 199

APPENDIX

Most Likely to

Least Likely to

Purchase a Home Purchase a Home

(61.6% of Sample) (38.4% of Sample) Difference in the

Impact of Being Impact of Being Impact of Being

Of ered Services Of ered Services Offered Services

Outcome

(Standard Error) (Standard Error) (Standard Error)

Monthly housing costs exceed 30 percent of monthly incomea (%)

2.0

2.4

-0.4

(1.6)

(2.9)

(3.9)

Ever 30 days delinquentb (%)

0.3

0.4

-0.1

(0.5)

(0.4)

(0.7)

Ever 60 days delinquentb (%)

0.1

0.3

-0.2

(0.4)

(0.2)

(0.5)

a Outcome was constructed using Short-Term Fol ow-Up Survey data. These data cover 78.8 percent of the study sample, which is 4,546 sample members. We applied sample weights that adjust for survey nonresponse for analyses of outcomes collected from the survey. (See appendix section A.6 for details.)

b Outcome was constructed using Short-Term Fol ow-Up Survey data, credit bureau data, lender loan and servicing data, and Federal Housing Administration data. These data cover 98.9 percent of the study sample, which is 5,708 sample members.

c Outcome was constructed using credit bureau data. These data cover 92.7 percent of the study sample, which is 5,346 sample members.

*** Difference is statistical y significant (two-sided t-test) at the p<0.01 level.

** Difference is statistically significant (two-sided t-test) at the p<0.05 level.

* Difference is statistically significant (two-sided t-test) at the p<0.10 level.

Note: Appendix B provides additional detail on the construction of measures.

G.4 Discussion

We found that a number of baseline measures of financial and credit characteristics, housing arrangements, and demographic characteristics are statistically significant predictors of whether a study participant purchased a home as of followup. Agencies providing homebuyer education and counseling services might find these findings informative to their approach to providing services. For example, that certain traits are associated with home purchase could inform the way agencies decide to market and provide their services to meet potential clients’ interests, preferences, and needs.

In addition, understanding why these traits are predictive of home purchase might matter for ensuring that all prospective homebuyers are being fairly and adequately served. On the one hand, it is not surprising that certain characteristics—for example, credit history and income—

are correlated with home purchase. On the other hand, it is not self-evident why other traits, such as race/ethnicity, are also correlated with home purchase. We encourage policymakers, researchers, and housing counseling agencies to further explore these disparities and determine whether and how various actors in the housing industry can better serve groups who could be facing additional barriers to home purchase.

Because the way in which we identify subgroup membership is imperfect, not everyone in the most-likely-to-purchase-a-home subgroup actually purchased a home, and not everyone in the least-likely-to-purchase-a-home subgroup actually didn’t. Indeed, we find the 21 percent of control group members who were predicted to purchase a home did not actually do so.

Conversely, 34 percent of control group members who predicted to not purchase a home actually did. The implication of this imperfect prediction is that both the most-likely-to-purchase-a-home The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 200

APPENDIX

subgroup and the least-likely-to-purchase-a-home subgroup contain a mix of study participants who did and did not purchase a home. Given this observation, the findings speak to the effect homebuyer education and counseling has on the set of study participants with characteristics that make them most likely to purchase a home (as opposed to the set of study participants that actually purchased a home). Because of the empirical challenges associated with analyzing a subgroup of individuals who select into that experience (of home purchase), we believe this analysis is a strong as we can get in an effort to examine the topic of the effects of homebuyer education and counseling on the subset of those who purchase a home.

In sum, this appendix provides an estimate of the impact of homebuyer education and counseling services on those most and least likely to purchase a home. We found no evidence that homebuyer education and counseling had a different impact on those most likely to purchase a home relative to those least likely to purchase a home (see exhibit G.2). Given the lack of differential impacts, there is no evidence that agencies providing homebuyer education and counseling would produce greater impacts if they targeted their services at those most likely to purchase a home.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 201

APPENDIX

Appendix H: Short-Term Fol ow-Up Survey

HUD’s Pre-Purchase Homeownership Counseling

Demonstration and Impact Evaluation

Short-Term Follow-Up Survey

Hello, this is ________________. I’m calling from Abt SRBI on behalf of the HUD First-Time Homebuyer Study. May I please speak to (RESPONDENT NAME)?

INTERVIEWER: IF NECESSARY, READ: “(RESPONDENT) has

agreed to help with a study on first-time homebuyers.

ALTERNATE CO-BORROWER INTRODUCTION

Hello, this is ________________. I am calling because (RESPONDENT NAME) is participating in the HUD First-Time Homebuyer Study and mentioned that (COBORROWER) is a listed co-borrower on their loan. May I please speak with (COBORROWER).

IF NECESSARY: I’m calling from Abt SRBI on behalf of the HUD for the First-Time

Homebuyer Study. We have already interviewed (BORROWER), who told us you were a co-borrower on their loan.

IF RESPONDENT IS NOT AVAILABLE COLLECT INFORMATION ON BEST TIME TO

CALL BACK.

1 YES [GO TO INTRO2 IF LANDLINE OR CELL1 IF CELL PHONE]

2 NO/DK, SCHEDULE CALLBACK [GO TO INTRO1A]

3 WRONG PERSON/NUMBER [THANK AND END, DISPO AS WRONG

PERSON/NUMBER]

4. GATEKEEPER REFUSAL - GIVE CONTACT INFO: Please call 877-251-5323 and ask

for Study 30223. Mention your ID is [QKEY]. [DISPO AS SOFT REFUSAL]

8. (VOL) Soft Refusal [DISPO AS SOFT REFUSAL]

9 (VOL) HARD REFUSAL [THANK AND END, DISPO AS HARD REFUSAL]

INTRO1A. INTERVIEWER: RECORD STATUS OF R

1 R NOT AVAILABLE – CALLBACK AT SAME NUMBER

2

R NOT AVAILABLE – CALLBACK AT DIFFERENT NUMBER [GO TO UP1]

UP1. [INTERVIEWER: UPDATE PHONE NUMBER]

UP2. Is that a landline or cell phone?

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 202

APPENDIX

1 LANDLINE [UPDATE CELL VARIABLE = 0]

2 CELL PHONE [UPDATE CELL VARIABLE = 1]]

[CATI – START CALLBACK AT INTRO1]

INTRO1. Hello, my name is [NAME] and I’m calling from Abt SRBI. I’m calling you about the HUD First-Time Homebuyer Study you joined about a year ago.

[CATI: ASK IF CELL PHONE SAMPLE]

CELL1: If you are now driving a car or doing an activity that requires your full attention, I need to call you back. Are you in a safe place that you can talk?

1. Yes, continue [GO TO INTRO2]

2. No, callback

INTRO2. When you joined the study, you completed a survey and we told you that we would be contacting you again in a year to learn how you are doing and ask you about the status of your home search process. The interview will take about 35 minutes and you will receive $35 to thank you for your time.

Participation in this study is voluntary. All information you provide will be kept secure and confidential. You may refuse to answer any individual questions.

Is now a good time to do the interview?

1. OK to continue

2. Not a good time [SCHEDULE CALLBACK]

9. REFUSED INTERVIEW [PLEASE COPY REFUSAL CODING FROM 5753]

[SET QUALIFIED LEVEL=1 FOR INTRO2=1, 2]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 203

APPENDIX

CONTINUE TO VERIFICATION:

DOB. First I just need to verify that I am speaking with the correct person.

What is your date of birth? Let’s start with the month and day.

Respondent’s Birthday: ________ / ________

MM

DD

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

COMPARE RESPONSE GIVEN TO THE BIRTH DATE ON SAMPLE FILE.

1. IF INFORMATION IS CORRECT CONTINUE WITH INTERVIEW

[SET QUALIFIED LEVEL=2]

2. IF THERE IS A MISMATCH IN DOB, DK OR REF:

I’m sorry. I was unable to pull up the correct questionnaire. I will need to check with my supervisor to look into the problem. I will re-contact you when the problem is resolved.

Thank you for your time.

DOBYR. And what is your year of birth?

1. Gave response [RANGE: 1920 – 1997]

9. (VOL) REFUSED [CONTINUE]

INTRO3. Before we begin, I am required to tell you that the questions in this survey have been reviewed by the Office of Management and Budget (OMB) under the Paperwork Reduction Act of 1995 OMB control # 2528-029, expiration date March 31, 2018. (IF NECESSARY: 12 U.S.C.

1701z-2(g)). That means that the federal government has reviewed and approved these survey questions.

We’d like to ask you some questions about yourself and if applicable, the people you might be buying or have bought a house with. Your name will not be linked with your answers. All information you provide is confidential and will be protected to the fullest extent possible by the law, including the Privacy Act of 1974 (IF NECESSARY: 5.U.S.C. 552a).

The survey will take about 35 minutes. Please stop me at any time if you have questions.

Section A: Home Purchase Status

Note: This section applies to all study participants.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 204

APPENDIX

A1. Since you first learned about this study in about [REGDTM] of [REGDTY] did you purchase or acquire any homes or properties?

1. Yes

2. No [SKIP TO A2]

8. (VOL) DON’T KNOW [SKIP TO A2]

9. (VOL) REFUSED [SKIP TO A2]

A1a. Do you currently live in a home that you purchased or acquired since then?

1. Yes [SKIP TO A3]

2. No

3. (VOL): Purchased more than one home, only live in one [SKIP TO A3]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

A2. And what is your current housing situation? Do you…

1. Rent your house or apartment

2. [CATI: HIDE PUNCH; code A2=2 if A1a=1, 3] Own your home

3. Live in someone else’s house or apartment without paying rent

4. Live in some other housing arrangement (SPECIFY: ___________________)

5. A MILITARY SETTING (BASE, CAMP, DEPLOYMENT, OR COMBAT ZONE)

6. EDUCATIONAL INSTITUTION (RESIDENTIAL COLLEGE, DORM)

7. HOTEL/MOTEL

8. SUBSIDIZED HOUSING

9. HOMELESS LIVING SITUATION (SHELTER)

10. INSTITUTIONAL FACILITY (MENTAL HEALTH, SUBSTANCE ABUSE) [GO

TO A2a]

11. CORRECTIONAL FACILITY/JAIL OR DETENTION CENTER [GO TO A2a]

98. DON’T KNOW

99. REFUSED

100.

[ASK IF A2 = 10 or 11]

A2a. To confirm, you currently live in a (n) [INSERT ANSWER FROM A2]. Did I get

that right?

1. Yes [TERMINATE CALL. DISPO AS SCREENOUT A2]

2. No [GO BACK TO A2]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 205

APPENDIX

[ASK IF A1 = 1, IF A1>1 SKIP TO A8]

A3. What month and year did you purchase or acquire your (first) home since you

learned of this study around [REGDTM] of [REGDTY]?

[PROMPT: If you inherited or otherwise did not purchase your home, please tell us the month and year that you became the owner of the property. If you purchased more than one property since then, please think about the first one.]

Month __________ Year__________ [RANGE: 2013-2015]

[IF DK, ASK FOR BEST GUESS.]

[ASK IF A1a=2, 8, 9 OTHERWISE SKIP TO A4]

A3a. Did you ever live in the home you purchased or acquired since you learned of this study?

1. Yes  From when to when? MM___/YYYY_______- MM

_____/YYYY_______ [RANGE: 2013-2015]

2. No

8. DON’T KNOW

9. REFUSED

A3b. Do you still own that home?

1. Yes [ASK A3c THEN SKIP TO A4]

2. No [SKIP TO A3d]

8. DON’T KNOW [SKIP TO A3d]

9. REFUSED [SKIP TO A3d]

A3c. What is the current use of the property? Is it a:

1. Rental property

2. Unoccupied investment property

3. Home for a relative or friend

4. Something else: Specify_______________

8. DON’T KNOW

9. REFUSED

A3d. What happened to the home you purchased or acquired on that date?

1. I sold the home

2. I lost the home through foreclosure

3. The home was damaged in a fire, flood, or some other disaster

4. Other (specify): __________________

8. DON’T KNOW [SKIP TO A4]

9. REFUSED[SKIP TO A4]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 206

APPENDIX

A3e. What month and year did that happen?

Month __________ Year__________ [RANGE: 2013-2015]

[IF DK, ASK FOR BEST GUESS.]

A4. Did you inherit the home?

1. Yes

2. No [Skip to A5]

8. DON’T KNOW [Skip to A5]

9. REFUSED [Skip to A5]

A4a. What was the estimated value of the home at the time that you inherited it?

Your best guess is fine.

1. $________________ (VALUE OF THE HOME) [RANGE: 0 – 999,999]

8. DON’T KNOW

9. REFUSED

A4b. When you inherited the home, did you take out a mortgage on this property?

1. YES

2. NO

8. DON’T KNOW

9. REFUSED

A5. Throughout the rest of the survey I will be asking you about the home you purchased since you learned about our study, sometime around [REGDTM] of [REGDTY]. Does the home you own include more than one housing unit? For example, some homes include a separate rental unit. [IF A1a=3: Please think about the first home you purchased, even if it isn’t the one you live in now.]

[Prompt: Answer yes only if you purchased more than one unit. Answer no if you purchased one unit of a multi-unit property.]

1. Yes

2. No [SKIP TO A5b]

8. DON’T KNOW [SKIP TO A5b]

9. REFUSED [SKIP TO A5b]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 207

APPENDIX

A5a. How many housing units does it include?

[Prompt: Include only those housing units that you purchased. Do not include any

housing units that may be located in the same structure but that you did not purchase.]

1. _________________# housing units owned [RANGE: 2 to 30]

8. DON’T KNOW

9. REFUSED

A5b. Now think about the overall structure. Is it [READ LIST]:

1. A single-family home

2. A townhouse or row house

3. A mobile home

4. A multi-unit condominium or co-op building

5. Some other type of housing (SPECIFY: _______________________)

8. DON’T KNOW

9. REFUSED

A6. When was this home originally built? Was it [READ LIST IF NECESSARY]:

1. Before 1970

2. Between 1970 and 1989

3. Between 1990 and 1999

4. Between 2000 and 2009

5. In 2010 or later

8. DON’T KNOW

9. REFUSED

A6a. How many separate rooms are in the house? Include bedrooms and all rooms that are separated from adjoining rooms by walls, archways, or half walls. Do not include bathrooms, foyers, half-rooms, garages, attics or unfinished basements.

1. ______________________#

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

A6b. How many of these rooms would count as bedrooms if this home were for sale or rent?

1. ______________________#

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 208

APPENDIX

A7. And what is the address of this home?

1. GAVE RESPONSE

8. DON’T KNOW

9. REFUSED

[STANDARD ADDRESS COLLECTION MODULE]

Street 1:_________________________

Street 2:_________________________

City:____________________________

State:___________________________

Zip:_____________________________

CREATE VARIABLE TO HOLD GROUP FOR ALTERNATE WORDING:

Group 1=A1=1 and A3≤BASEDT (Sample variable)

Group 2=A1=1 and A3>BASEDT (INCLUDING A3=DK/REF) (Sample variable)

Group 3=A1>1 (including A1=DK/REF)

A8. GROUP=3: Now think about your plans for the future. If you purchase a home, how many years do you think you will own it? [READ LIST]

GROUP=1, 2: Now think about your plans for the future. How many years do you think you will own the home? [READ LIST]

(Prompt: If you don’t know, please give us your best guess.)

1. Less than 1 year

2. 1 to 5 years

3. 6 to 10 years

4. 11-20 years

5. More than 20 years

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[GROUP=1, 2 SKIP TO B1.]

GROUP=3

A.9 The last time we talked with you, you were searching for a home to purchase. Are you still actively searching for a home to purchase?

1. Yes [SKIP TO SECTION B]

2. No

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 209

APPENDIX

A9a. I am going to read you a list of common reasons individuals postpone their search for a home, please let me know if any of them describe the reason you postponed your search for a home.

[READ LIST, CHECK ALL THAT APPLY.]

1. Learned I could not afford to buy a home

2. Learned I needed to repair my credit first

3. Did not like the houses I could afford

4. Did not like the neighborhoods I could afford

5. The person I was planning to purchase a home with is no longer interested in

purchasing a home

6. There was a change in my (or my co purchaser’s) employment situation

7. The current economic climate has made it more difficult to get a mortgage

8. The information I got from a counseling agency workshop made me better aware

of my personal situation in the home buying process

98. DON’T KNOW

99. REFUSED

A9b. Were there any other reasons you have postponed your search for a home?

1. NO

2. OTHER SPECIFY (____________________)

8. DON’T KNOW

9. REFUSED

10.

Section B: Income and Financial Management

[Note: This section applies to all study participants.]

INTRO: Next, I would like to ask you a few questions about your financial experiences.

B1. For each of the following statements, please tell me whether you strongly disagree, disagree, agree, or strongly agree with the statement. [CATI: RANDOMIZE]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 210

APPENDIX

STATEMENTS:

B1a. I occasionally overdraw my checking account.

B1b. I occasionally don’t have enough money to cover all of my bills at the end of the month.

B1c. I usually have enough savings set aside to cover three months of expenses.

B1d. I’ve tried to figure out how much I need to save for retirement.

B1e. I never use payday lenders.

B1f. I usually pay my credit card balance in full to avoid interest charges.

B1g. I usually shop around when choosing a new credit card.

B1h. I know how to correct inaccurate information in my credit report.

B1i. I trust banks with my money.

ANSWER CHOICES: 1. Strongly Disagree

2. Disagree

3. Agree

4. Strongly Agree

8. DON’T KNOW

9. REFUSED

INTRO: Now, I would like to ask you a few questions about different mortgage options. For each statement I read, tell me whether it is a true statement or a false statement.

B2. The interest rate on a mortgage loan is the same thing as the annual percentage rate (APR). 1. True

2. False

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

B3. A home equity loan is secured by your house.

1. True

2. False

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

B4. When you first get a mortgage loan, only a small portion of your monthly payment, if any, reduces the amount you owe. Most of your monthly payment is applied to interest.

1. True

2. False

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 211

APPENDIX

B5. The loan officer is legally obligated to tell you if you qualify for a different loan product that has a lower cost.

1. True

2. False

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

INTRO: Next, I would like to ask you a few questions about different sources of income you may receive.

B6. GROUP=3: Please think about the total income you have available to pay your monthly mortgage payment each month. If you expect to have co-borrowers on your loan, think about the total for yourself and the co-borrowers. We refer to “co-borrowers” as people who will share ownership and financial responsibility for the home you may buy. In other words, these are individuals who will also be named on the home loan. They may or may not currently live with you.

GROUP=1, GROUP=2: Please think about the total income you have available to pay

your monthly mortgage payment each month. If you have co-borrowers on your loan,

think about the total for yourself and the co-borrowers. We refer to “co-borrowers” as people who share ownership and financial responsibility for the home you bought. In other words, these are individuals who are also to be named on the home loan. They may or may not currently live with you.

Thinking about the last year, does this income include:

STATEMENTS:

B6a Income from self-employment where you work for yourself and not through an employer.

B6b Wages or salary from a job

B6c Rent or other income from an investment property

B6d Interest, dividend, or other investment income

B6e Child support payments, alimony, or maintenance payments

B6f Social Security retirement or disability benefits

B6g Other pensions or retirement income

B6h Public assistance or Earned Income Tax Credit benefits

B6i Unemployment benefits

B6j Veterans’ benefits

B6k Other income

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 212

APPENDIX

ANSWER CHOICES:

1. Yes

2. No

8. DON’T KNOW

9. REFUSED

B7. What is the total amount of income that you and any co-borrowers received in the last 12

months? Include income from all sources and tell me the total amount before any taxes or deductions are removed. [IF R IS UNSURE PROMPT: Your best estimate is fine.]

1. GAVE RESPONSE $ ______________________ [RANGE: 0-999,999+]

8. (VOL) DON’T KNOW [SKIP TO B7b]

9. (VOL) REFUSED [SKIP TO B8]

B7a. To confirm, you said ___________________. Did I get that right?

1. YES [SKIP TO B7c]

2. NO[GO BACK TO B7]

B7b. If you are unsure of this total income that you and any co-borrowers received in the last 12 months, was it……..

1. Less than $10,000

2. $10,000 to less than $25,000

3. $25,000 to less than $40,000

4. $40,000 to less than $55,000

5. $55,000 to less than $70,000

6. $70,000 to less than $85,000

7. $85,000 to less than $100,000

8. $100,000 or greater

98. (VOL) DON’T KNOW [SKIP TO B8]

99. (VOL) REFUSED [SKIP TO B8]

B7c. Does this amount include income from anyone other than yourself?

1. Yes

2. No [Skip to B9]

8. (VOL) DON’T KNOW [Skip to B9]

9. (VOL) REFUSED [Skip to B9]

B8. Thinking only about your own income, what is the total amount of income you received in the last 12 months? Tell me the total amount before any taxes or deductions are removed. . [IF R IS UNSURE PROMPT: Your best estimate is fine.] [CATI: B7a

CANNOT BE GREATER THAN B8 or B8b]

1. GAVE RESPONSE $ ______________________ [RANGE: 0-

999,999+]

8. (VOL) DON’T KNOW [Skip to B8b]

9. (VOL) REFUSED [Skip to B9]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 213

APPENDIX

B8a. To confirm, you said ___________________. Did I get that right?

1. YES [Skip to B9]

2. NO [GO BACK TO B8]

B8b.

If you are unsure of your total amount of income received in the last 12

months, was it…..

1. Less than $10,000

2. $10,000 to less than $25,000

3. $25,000 to less than $40,000

4. $40,000 to less than $55,000

5. $55,000 to less than $70,000

6. $70,000 to less than $85,000

7. $85,000 to less than $100,000

8. $100,000 or greater

98. (VOL) DON’T KNOW

99. (VOL) REFUSED

B9.

A budget is a spending plan for your monthly household expenses. Do you have an

electronic or written budget?

1. Yes

2. No [SKIP TO B12]

8. (VOL) DON’T KNOW [SKIP TO B12]

9. (VOL) REFUSED [SKIP TO B12]

B10. About how often do you compare your electronic or written budget to your actual spending? [READ LIST]

1. Often

2. Sometimes

3. Rarely

4. Never

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

B11. Thinking back on the past year, about how often were you able to stick to your electronic or written budget? [READ LIST]

1. Often

2. Sometimes

3. Rarely

4. Never

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 214

APPENDIX

B12. Being short of money means that you brought in less money than you spent and had to do something to get through the month, like using credit cards, taking money out of savings, or borrowing money. Over the past year, how often were you short of money? [READ

LIST] 1. Often

2. Sometimes

3. Rarely

4. Never

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

B13. Saving money means spending less than you earn, and putting the extra money aside for retirement, education, or to build a financial cushion. About how often do you save money? [READ LIST]

1. Often

2. Sometimes

3. Rarely

4. Never

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

B14. Do you currently have a checking account?

1. YES

2. NO (SKIP TO B15)

8. (VOL) DON’T KNOW (SKIP TO B15)

9. (VOL) REFUSED (SKIP TO B15)

B14a. How much money do you currently have in checking accounts? Please round to the nearest 100. [Prompt: If you have more than one checking account, please tell us the total amount in these accounts. ANSWER MUST END IN 00] [CATI – ANSWER

MUST BE ROUNDED TO THE NEAREST 100. LAST TWO DIGITS SHOULD

BE 00]

1. GAVE RESPONSE______________ [RANGE: 0-999,000+]

8. (VOL) DON’T KNOW (SKIP TO B15)

9. (VOL) REFUSED (SKIP TO B15)

B14b. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO B14a]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 215

APPENDIX

B15. Do you currently have a savings account?

1. YES

2. NO (SKIP TO B16)

8. (VOL) DON’T KNOW (SKIP TO B16)

9. (VOL) REFUSED (SKIP TO B16)

B15a. How much money do you currently have in savings accounts? Please round to the nearest 100. [Prompt: If you have more than one savings account, please tell us the total amount in these accounts. ANSWER MUST END IN 00] [CATI – ANSWER

MUST BE ROUNDED TO THE NEAREST 100. LAST TWO DIGITS SHOULD

BE 00]

1. GAVE RESPONSE______________ [RANGE: 0-999,000+]

8. (VOL) DON’T KNOW (SKIP TO B16)

9. (VOL) REFUSED (SKIP TO B16)

B15b. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO B15a]

B16. Do you currently have any retirement accounts, like 401(k) or 403(b) accounts, IRAs, or other pension accounts? Please round to the nearest 100. [Prompt: If you have more than one retirement account, please tell us the total amount in these accounts. ANSWER

MUST END IN 00] [CATI – ANSWER MUST BE ROUNDED TO THE NEAREST

100. LAST TWO DIGITS SHOULD BE 00]

1. YES

2. NO (SKIP TO B17)

8. (VOL) DON’T KNOW (SKIP TO B17)

9. (VOL) REFUSED (SKIP TO B17)

B16a. How much money do you currently have in such accounts?

1. GAVE RESPONSE_____________ [RANGE: 0-999,000+]

8. (VOL) DON’T KNOW (Skip to B17)

9. (VOL) REFUSED (Skip to B17)

B16b. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO B16a]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 216

APPENDIX

B17. Aside from your savings accounts and retirement accounts, do you currently have any other money market accounts, certificates of deposit, mutual funds, stocks, or brokerage accounts? Please round to the nearest 100. [Prompt: If you have more than one account, please tell us the total amount in these accounts. ANSWER MUST END IN 00] [CATI –

ANSWER MUST BE ROUNDED TO THE NEAREST 100. LAST TWO DIGITS

SHOULD BE 00]

1. YES

2. NO (SKIP TO B18)

8. (VOL) DON’T KNOW (SKIP TO B18)

9. (VOL) REFUSED (SKIP TO B18)

B17a. How much money do you currently have in such accounts?

1. GAVE RESPONSE______________ [RANGE: 0-999,000+]

8. (VOL) DON’T KNOW (SKIP TO B18)

9. (VOL) REFUSED (SKIP TO B18)

B17b. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO B17a]

B18. Do you have any other source of savings that would be available if you lost your job or had a financial emergency? For example, this might include savings at home or savings with others who are keeping it safe. Please round to the nearest 100. [Prompt: If you have more than one savings account, please tell us the total amount in these accounts.

ANSWER MUST END IN 00] [CATI – ANSWER MUST BE ROUNDED TO THE

NEAREST 100. LAST TWO DIGITS SHOULD BE 00]

1. YES

2. NO (SKIP TO B19)

8. (VOL) DON’T KNOW (SKIP TO B19)

9. (VOL) REFUSED (SKIP TO B19)

B18a. About how much would be available?

1. GAVE RESPONSE_________________ [RANGE 0-999,000+]

8. (VOL) DON’T KNOW (SKIP TO B19)

9. (VOL) REFUSED (SKIP TO B19)

B18b. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO B18a]

[SKIP TO B20 IF SUM OF B14a + B15a + B16a + B17a + B18a = 0 OR COMBINATION OF

ALL = 0 AND DK/REF]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 217

APPENDIX

B19. To confirm, your responses include a total of __________ in savings and investments.

Does that sound about right? [CATI: INSERT SUM OF B14a + B15a + B16a + B17a +

B18a IF = 1 GAVE RESPONSE]

1. YES (SKIP TO B20)

2. NO

8. (VOL) DON’T KNOW (SKIP TO B20)

9. (VOL) REFUSED (SKIP TO B20)

[GO BACK TO B14 AND CORRECT THE RESPONSES TO THE PREVIOUS QUESTIONS]

B20. GROUP=3: Now pretend that you have already purchased a home. If you started having financial problems and could not pay all of your bills, which bill would you pay first?

[READ LIST]

GROUP=1, GROUP=2: As a recent homeowner, if you started having financial problems and could not pay all of your bills, which bill would you pay first? [CATI: RANDOMIZE

LIST] 1. Credit card

2. Utilities (gas, electricity, water, etc.)

3. Car payment

4. Mortgage

5. Student loan

6. Health insurance

7. Other [SPECIFY_________________]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

B23. How frequently do you review your credit report information? [READ LIST]

1. I have never looked at my credit report

2. It has been 5 years or more since I have looked at my report

3. It has been a few years since I have looked at my report

4. Once a year

5. A few times during the year

6. Monthly

7. (VOL) PREFER NOT TO SAY

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 218

APPENDIX

Section C: Home and Mortgage Search

Intro: The next section asks questions about your home purchase process, whether you recently purchased a home or have not purchased a home.

C1. GROUP=3: Since you started your home search how many homes have you visited?

GROUP=1, GROUP=2: During your home search process, how many homes did you

visit? 1. ___________________HOMES [RANGE = 0-97]

98. (VOL) DON’T KNOW

99. (VOL) REFUSED

[IF C1=0, SKIP TO C4]

C2. GROUP=3: How many offers did you make on any homes you visited?

GROUP=1, GROUP=2: During your home search process, how many offers did you make on any homes you visited?

How many offers? ________________ (IF 0, Skip to C4) [RANGE: 0-97]

98. (VOL) DON’T KNOW [SKIP TO C4]

99. (VOL) REFUSED [SKIP TO C4]

C3. GROUP=3 ONLY: - Was the last offer you made accepted, rejected, or is it still outstanding?

1. Accepted

2. Rejected

3. Outstanding

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

C4. GROUP=3: In total, how many lenders have you contacted since starting the home search process?

GROUP=1, GROUP=2: In total during your home search process, how many lenders did

you contact?

1. _______________ # [IF ZERO, SKIP TO C7] [RANGE: 0-97]

98. (VOL) DON’T KNOW

99. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 219

APPENDIX

C5. GROUP=1, GROUP=2: Did you ask any of these lenders for a price quote of the interest rate and other costs associated with loans that you might apply for??

GROUP=3: Have you asked any lender for a price quote of the interest rate and other costs associated with loans that you might apply for?

1. YES

2. NO [SKIP TO C7 IF GROUP=3; SKIP TO C8 IF GROUP=1 OR

GROUP=2]

8. (VOL) DON’T KNOW [SKIP TO C7 IF GROUP=3; SKIP TO C8 IF

GROUP=1 OR GROUP=2]

9. (VOL) REFUSED [SKIP TO C7 IF GROUP=3; SKIP TO C8 IF

GROUP=1 OR GROUP=2]

C5a. Did you get price quotes from more than one lender?

1. YES; How many # ________________ [RANGE 2-97]

2. NO

98. (VOL) DON’T KNOW

99. (VOL) REFUSED

C6. GROUP=3: Have you had a loan application denied by any of the lenders you

contacted??

GROUP=2: Did you have a loan application denied by any of the lenders you contacted?

1. YES (AT LEAST ONE APPLICATION HAS BEEN DENIED)

2. NO (NONE OF THE APPLICATIONS HAVE BEEN DENIED)

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

C7. GROUP=3: Have you signed a purchase agreement for a home?

1. YES

2. NO

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[IF C1=0 OR C2=0, SKIP TO C9]

C8. GROUP=3- A home inspection is an examination of the physical structures and systems of a house, to identify any problems or needed repairs. Have you had the home inspected?

GROUP=1, GROUP=2- A home inspection is an examination of the physical structures and systems of a house, to identify any problems or needed repairs. Before you purchased your home, did you have the home inspected?

1. YES

2. NO

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 220

APPENDIX

C9. GROUP=1, GROUP=2: In general during the home purchase process, how confident were you that you could find the information you needed about the home purchase

process?

GROUP=3: In general during the home purchase process, how confident are you that you can find the information you need about the home purchase process? [READ LIST]

1. Very Confident

2. Confident

3. Somewhat Confident

4. Not Confident at All

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

C10. GROUP=1, GROUP=2: In general, how satisfied were you with the home purchase

process?

GROUP=3: In general, how satisfied are you with the home purchase process? [READ

LIST]

1. Very Satisfied

2. Somewhat Satisfied

3. Somewhat Dissatisfied

4. Very Dissatisfied

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[GROUP=3, SKIP TO Cl2]

C11. GROUP=1, GROUP=2: Now think about when you made your initial offer for the home you purchased. When you made your offer, was your offer above, below, or about the same as the property’s listing price? Choose ‘about the same’ if your offer was within $1,000 of the property’s listing price. [Prompt: A listing price is the price that is shown in advertisements for potential buyers before any offers are made.]

1. Above

2. Below

3. About the same

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 221

APPENDIX

C11a. Did you negotiate with the seller for any seller-provided home repairs, closing costs, or other costs associated with finalizing the home purchase?

1. Yes

2. No

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

GROUP=1/GROUP=2: Intro: Now think about the factors you considered in purchasing your home.

GROUP=3: Intro- Now think about the factors you might consider when purchasing a home.

C12. GROUP=3= For each of the following home features, please tell me whether the feature is very important, important, somewhat important, or not at all important to you in selecting a home.

GROUP=1/GROUP=2= For each of the following home features, please tell me whether

the feature was very important, important, somewhat important, or not at all important to you in selecting a home

[CATI: RANDOMIZE]

CATEGORIES:

C12a The number of bedrooms and bathrooms

C12b The attractiveness of a home’s interior

C12c The attractiveness of a home’s exterior

C12d The yard or landscaping

C12e A home’s maintenance and repair needs

C12f The age of a home

ANSWER CHOICES:

1. Very Important

2. Important

3. Somewhat Important

4. Not at all Important

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 222

APPENDIX

C13. GROUP=3: For each of the following neighborhood features, please tell me whether the feature is very important, important, somewhat important, or not at all important to you in selecting a home.

GROUP=2: For each of the following neighborhood features, please tell me whether the feature was very important, important, somewhat important, or not at all important to you in selecting a home.

[CATI: RANDOMIZE]

CATEGORIES

C13a School quality

C13b Safety

C13c The length of your commute to work

C13d Neighborhood amenities like parks or nearby restaurants

C13e Access to public transportation

C13f The appearance of other homes in the neighborhood

ANSWER CHOICES:

1. Very Important

2. Important

3. Somewhat Important

4. Not at all Important

8. DON’T KNOW

9. REFUSED

Section D: Home and Mortgage Features

This section is for study participants who have purchased a home or inherited a home with a mortgage (GROUP=1 or GROUP=2). GROUP=3 should skip to Section F.

If you own a home, it will be very helpful to have your Settlement Statement or Closing Disclosure on hand. You probably received one of these documents a few days before closing or when you signed the settlement documents. The Settlement Statement is also called a HUD-1. It is okay if you don’t have both documents—most people only get one or the other. Each document is about 3 to 5 pages long and should say either ‘Settlement Statement’ or ‘Closing Disclosure’ at the top of the first page.]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 223

APPENDIX

D0. Do you have your Settlement Statement (or HUD-1 form) or Closing Disclosure on hand?

1. Settlement Statement [PROCEED WITH SURVEY]

2. Closing Disclosure [PROCEED WITH SURVEY]

3. No [INTERVIEWER ASK: Do you want to go and get the form? IF YES,

INTERVIEWER WAIT UNTIL RESPONDENT COMES BACK. IF NO, PROCEED

WITH SURVEY]

D1. What was the purchase price of the home you purchased? That is, what was the final amount you paid for this home? [IF D0=1: This can be found on Line 101, labeled Contract Sales Price on your Settlement Statement. IF D0=2: This can be found on the top left hand corner on page one of the Closing Disclosure under the Closing Information heading. It is labeled as Sale Price.]

[Probe: This price does not include closing costs or any subsidy you received from the seller.]

1. $__________________ [RANGE: 1-999,999+]

8. DON’T KNOW

9. REFUSED

D2. How much was your initial deposit on the purchase? This is also known as earnest money. It is the amount that bound you and the seller to the terms of the Purchase and Sale agreement. [IF

D0=1: This can be found on Line 201, labeled Deposit or earnest money on your Settlement Statement. IF D0=2: This can be found in the middle of page three of the Closing Disclosure in section L under the heading Paid Already by or on Behalf of Borrower at Closing. It is labeled as Deposit.]

1. $_______________________ [RANGE: 0-999,999+]

8. DON’T KNOW

9. REFUSED

D3. How much in total did you pay at closing? Please think of all costs including the down payment, closing costs and fees. If you are not sure, please provide your best estimate. [IF D0=1: This can be found on Line 303, labeled Cash from borrower on your Settlement Statement. IF

D0=2: This can be found at the very bottom of page three of the Closing Disclosure under the heading Calculation. The amount is listed next to the label Cash to Close From Borrower.]

1. $_______________________ [SKIP TO D4] [RANGE: 0-999,999+]

8. DON’T KNOW [GO TO D3a]

9. REFUSED [SKIP TO D4]

D3a. What is your best estimate of the amount you paid at closing?

1. $________________ [RANGE: 0-999,999+]

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 224

APPENDIX

D4. For the next question, please think only about the amount of your down payment. How much was the down payment amount?

1. $_______________________ [RANGE: 0-999,999+]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[SKIP D5 IF ALL D2-D3 = DK AND/OR REF]

D5. Together that would be a total of [INSERT $$] for the down payment, closing costs, and other fees covered by these payments. Does that sound about right?

1. Yes

2. No [RETURN TO D2 – D4 TO UPDATE AMOUNTS]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

D7. Did you receive any down payment assistance, grants, or forgivable loans that you may not be obligated to pay back, such as loans or grants from a city or county government agency, a community organization, or a local housing agency? [INTERVIEW: IF R MENTIONS FUNDS

RECEIVED FROM HUD PROGRAMS, “HOME” PROGRAM, OR FAMILY SELF

SUFFICIENCY PROGRAM, CODE RESPONSE AS YES.]

1. YES

2. NO [Skip to D9]

8. DON’T KNOW [Skip to D9]

9. REFUSED [Skip to D9]

D7a. What was the total amount of the grant or other form of assistance you received? If you are not sure, give your best estimate.

$________________ [RANGE: 1-999,999+]

8. DON’T KNOW

9. REFUSED

The next set of questions asks about how you financed the purchase of your home. This could include a mortgage loan which is a loan that you must repay. Please do not include any down payment assistance, grants, or forgivable loans that you have previously described.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 225

APPENDIX

D9. How did you finance the acquisition of this home? (READ LIST, CHECK ALL THAT

APPLY)

1. Took on one new mortgage [SKIP TO D10]

2. Took on two or more new mortgages [GO TO D9a]

3. Assumed one or more mortgages already on the property [SKIP TO D10]

4. Borrowed using assets other than this property as collateral [SKIP TO E1]

5. Gift or loan from friends or relatives [SKIP TO E1]

6. Paid all cash – no borrowing [SKIP TO E1]

7. Other (specify) [SKIP TO E1]

8. (VOL) DON’T KNOW [SKIP TO E1]

9. (VOL) REFUSED [SKIP TO E1]

NOTE ON D9: SOME COMBINATIONS ARE MUTUALLY EXCLUSIVE:

1. List of unallowed combos:

2. 1&2

6 with 1,2,3,4

And then skip to the earliest skip based on multiple selections.

D9a. And how many mortgages did you take out?

1. Two

2. Three

3. More than three

8. DON’T KNOW

9. REFUSED

[If D9=2]: The next set of questions focus on your first mortgage. Please exclude any subordinate mortgages. We will talk about them later.

D10. How much is the loan amount for your first mortgage on this home?

1. $___________________ [RANGE: 1-999,999+]

8. DON’T KNOW

9. REFUSED

(IF R SAYS DK, NOT SURE, PROBE: Think about the purchase price of the home and

the amount of your total down payment. If you have one mortgage, the remaining amount after the down payment would be the mortgage, or loan amount you needed, to purchase this home)

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 226

APPENDIX

D12. What is the initial annual interest rate on this mortgage? [IF D0=1: This can be found on your Settlement Statement in The Loan Terms box on about page 3 that has a line that says

‘Your initial interest rate is …’ IF D0=2: This can be found at the top half of the Closing Disclosure in the Loan Terms section. It is printed in bold and is called Interest Rate. ]

______________% Annual Interest Rate [RANGE: 0.00-15.00%]

8. DON’T KNOW

9. REFUSED

D13. Is your mortgage a standard, fixed-rate mortgage or adjustable-rate mortgage? Or is it some other type of mortgage? IF D0=2: This can be found at the top of page one of the Closing Disclosure in the right side of the page under the heading Loan Information.]

1. Fixed rate mortgage [GO TO D14]

2. Adjustable rate mortgage (ARM) [GO TO D13a]

3. Or some other type of mortgage

(Specify with any notes listed on the SETTLEMENT

STATEMENT:________________________) [GO TO D14]

8. DON’T KNOW [GO TO D14]

9. REFUSED [GO TO D14]

D13a. Has your interest rate changed since you purchased this home?

1. Yes

2. No [SKIP TO D13c]

8. (VOL) DON’T KNOW [SKIP TO D13c]

9. (VOL) REFUSED [SKIP TO D13c]

D13b. What is your current interest rate on your mortgage?

1. ___________________ [RANGE: 0.00-15.00%]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[SKIP D13c IF D13a=1]

D13c. When will the interest rate change, or adjust, on this mortgage?

1. 3 years after the loan was made

2. 5 years after the loan was made

3. 7 years after the loan was made

4. Or after some other number of years (Specify: _____________YEARS)

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 227

APPENDIX

D14. What is the term of the mortgage? That is, what is the total number of months or years over which mortgage payments are to be made? [IF D0=1: This can be found on your Settlement Statement in The Loan Terms box on about page 3 that has a line that says ‘Your loan term is … IF D0=2: This can be found at the top of page one of the Closing Disclosure in the right side of the page under the heading Loan Information.]

1. 30 years or 360 months

2. 25 years or 300 months

3. 20 years or 240 months

4. 15 years or 180 months

5. 10 years or 120 months

6. Or some other number of months (Specify: _____________MONTHS)

8. DON’T KNOW

9. REFUSED

D15. Do you have mortgage insurance? This insurance is sometimes called PMI or MIP for FHA loans, and may have been required by the bank or lender, to protect them against possible nonpayment. Answer yes if you have a loan from FHA, VA, FmHA, or RHS. [This is

different from insurance on the home itself.] [IF D0=1: This can be found on your Settlement Statement in The Loan Terms box on about page 3 that has a line ‘Your initial monthly amount owed for principal, interest, and any mortgage insurance is.’ IF D0=2: This can be found around the middle of page one on the Closing Disclosure under the heading Projected Payments. If you have mortgage insurance, there will be a dollar amount listed for that heading.]

1. Yes [GO TO D15a]

2. No [GO TO LOGIC BEFORE D16a]

8. DON’T KNOW [GO TO LOGIC BEFORE D16a]

9. REFUSED [GO TO LOGIC BEFORE D16a]

D15a. What type of mortgage insurance do you have? Do you have mortgage insurance from…? (READ LIST)

1. A private insurance company, such as Mortgage Guarantee Insurance (MGIC)

(Conventional Insured)

2. Federal Housing Administration (FHA)

3. Farmers Home Administration (FmHA), or USDA/Rural Housing (RHS)

4. Veterans Administration (VA)

5. Mortgage insurance from a State agency for first-time homebuyers

6. Or some other type of mortgage insurance (Specify: _________________)

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 228

APPENDIX

[ASK D16a if D9a=1, 2, 3 AND D16b if D9a=2, 3; ALL ELSE SKIP TO E1]

D16a. IF D9A =1, 2, 3 ASK: Next, I’d like to focus on your second mortgage loan.

How much is the total loan amount on your second mortgage?

1. $___________________ [RANGE: 1-999,999+]

8. DON’T KNOW

9. REFUSED

D16a1. What was the initial interest rate on the second mortgage?

______________% Annual Interest Rate [RANGE: 0.00-15.00%]

8. DON’T KNOW

9. REFUSED

D16a2. Can this interest rate change over time?

1. Yes

2. No [SKIP TO D16b]

8. (VOL) DON’T KNOW [SKIP TO D16b]

9. (VOL) REFUSED [SKIP TO D16b]

D16a3. What is the current interest rate on this loan?

______________% Annual Interest Rate [RANGE: 0.00-15.00%]

8. DON’T KNOW

9. REFUSED

D16b. IF D9A =2, 3, ALSO ASK: Now let’s focus on your third mortgage loan.

How much is the total loan amount on your third mortgage?

1. $___________________ [RANGE: 1-999,999+]

8. DON’T KNOW

9. REFUSED

D16b1. What was the initial interest rate on the third mortgage?

______________% Annual Interest Rate [RANGE: 0.00-15.00%]

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 229

APPENDIX

D16b2. Can this interest rate change over time?

1. Yes

2. No [SKIP TO E1]

8. (VOL) DON’T KNOW [SKIP TO E1]

9. (VOL) REFUSED [SKIP TO E1]

D16b3. What is the current interest rate on this loan?

______________% Annual Interest Rate [RANGE: 0.00-15.00%]

Section E: Mortgage Performance

Note: This section is asked only of homeowners (A1=1). All others should skip to Section F.

Intro: The next set of questions asks about your experiences during the period since you purchased the home.

E1. Since you purchased the home, have you taken out a home equity line of credit (HELOC) or a home equity loan such as a second or third mortgage? Do not include any second or third mortgages that you used to purchase the home and have already told us about.

1. Home equity line of credit

2. Home equity loan [SKIP TO E3]

3. Both: home equity line of credit and second or third mortgage

4. No [Skip to E8]

8. DON’T KNOW [Skip to E8]

9. REFUSED [Skip to E8]

E2b. Have you ever used the home equity line of credit to borrow money?

1. Yes

2. No [SKIP TO E6]

8. (VOL) DON’T KNOW [SKIP TO E6]

9. (VOL) REFUSED [SKIP TO E6]

E3. How did you use the money you borrowed using the home equity loan or line of credit?

[Check all that apply]

1. Pay down credit cards or other debt

2. Make a home improvement or repair

3. Pay for appliances, furniture, or other home furnishings

4. Purchase or lease a vehicle

5. Pay for education for yourself or a child

6. Pay off medical costs

7. Other. Specify___________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 230

APPENDIX

[ASK IF E2b=1]

E4. What is the current balance of the home equity line of credit?

1. $____________________

8. REFUSED

9. DON’T KNOW

[ASK IF E2b=1]

E5. What is the current interest rate of the home equity line of credit?

____________________% [If E1=1, Skip to E8] [RANGE: 0.00-15.00%]

1.

8. DON’T KNOW [If E1=1, Skip to E8]

9. REFUSED [If E1=1, Skip to E8]

E1=1, SKIP TO E8

E6. When did you get your home equity loan? Please tell me the month and year.

_________Month _________Year [RANGE: 2013-2015]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E6a. What was the loan amount for your home equity loan(s)? [Prompt: If you have taken out more than one additional mortgage loan since you purchased the home, tell us the total amount of the new loans.]

1. $____________________ [RANGE: 0-999,999+]

8. DON’T KNOW

9. REFUSED

E7. What is the current interest rate of the home equity loan? [Prompt: If you have taken out more than one additional mortgage since you purchased the home, tell us the highest interest rate among the new loans.]

1. ____________________% [RANGE: 0.00-15.00]

8. DON’T KNOW

9. REFUSED

IF D9 = 6 paid cash skip to E16

Intro: The next set of questions asks about your experiences with all of the mortgage loans that we have talked about to this point, including loans you used to purchase the home and any home equity loans or lines of credit that you may have taken out since.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 231

APPENDIX

E8. [USE THIS VERSION IF CONDITION BELOW IS NOT MET] Since you purchased the

home, have you refinanced or made any modifications to your mortgage loan?

[If D9a=1, 2, 3 or E1=1, 2, 3] Since you purchased the home, have you refinanced or made any modifications to any of the mortgages on your home?

1. Yes

2. No [SKIP TO E9]

8. (VOL) DON’T KNOW [SKIP TO E9]

9. (VOL) REFUSED [SKIP TO E9]

E8a. How many mortgages or loans have you refinanced on your home since purchase?

1. _________ TOTAL NUMBER OF REFINANCED LOANS [RANGE 1-7]

8. DON’T KNOW

9. REFUSED

E8b. Did you refinance your primary mortgage loan? [Prompt: A primary mortgage loan is sometimes called a first mortgage. It is the loan with the largest principal balance.]

1. Yes

2. No

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8c. I’m going to read you several reasons why people refinance. For each, please tell me if it was a reason that you refinanced. Did you…? [READ LIST. CHECK ALL THAT

APPLY]

1. Lower the interest rate

2. Reduce your monthly payment

3. Borrow additional money from your home equity [Prompt: Answer yes if you

received any money from your home equity during the refinance.]

4. Other (SPECIFY)

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8d. How much additional money, besides the balance of the original loan, did you borrow during the refinance?

$_________ [RANGE: 0-999,999+]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[ASK IF E8d=1 GAVE RESPONSE AND VALUE>0]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 232

APPENDIX

E8e. Can you tell me how the money was used? [Check all that apply]

1. Pay down credit cards or other debt

2. Make a home improvement or repair

3. Pay for appliances, furniture, or other home furnishings

4. Purchase or lease a vehicle

5. Pay for education for yourself or a child

6. Pay off medical costs

7. Other. Specify___________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

IF E8a=1-7:

E8f. Please tell me about the refinanced loan.

[ALT WORDING IF E8a>1]: Please tell me about your refinanced loans. Let’s

start with the refinancing of your primary mortgage loan and then go by date.

E8f1: How much is the principal loan amount for this refinance loan?

$_________ [RANGE: 0-999,999+]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f2: What is the name of the bank or lender where you have this refinance loan?

Open-end: _________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f3: In what month and year did you obtain this refinance loan?

______/______ [RANGE: 2013-2015]

MM/DD

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f4. What type of loan is this refinance: a fixed-rate, line of credit, balloon/graduated payment, adjustable-rate, or some other type of loan?

1. Fixed (FRM)

2. Line of Credit

3. Balloon/graduated

4. Adjustable

5. Other, specify: __________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 233

APPENDIX

E8f5. What is the term of this loan, that is, what is the total number of months over which loan payments are to be made?

1. 30 years or 360 months

2. 25 years or 300 months

3. 20 years or 240 months

4. 15 years or 180 months

5. 10 years or 120 months

6. Or some other number of months (Specify: _____________MONTHS)

8. DON’T KNOW

9. REFUSED

E8f6. What is the initial interest rate on this refinance loan?

E8f7: [If E8f4=4] Has your interest rate changed for this refinance loan? If so, what is the current interest rate for the refinanced loan?

1. Yes, specify rate: ____% [RANGE: 0.00-15.00%]

2. Yes, don’t know new rate

3. No

8. DON’T KNOW

9. REFUSED

E8f8. [If E8f4=3] How many years before the final large payment of the principal for the balloon loan is made?

_________years [RANGE: 0-30]

8. DON’T KNOW

9. REFUSED

IF E8a=2-7:

E8f1_2: Now think about your second refinanced loan. How much is the principal loan amount for the second refinance loan?

$_________ [RANGE: 0-999,999+]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 234

APPENDIX

E8f2_2: What is the name of the bank or lender where you have your second refinance loan?

Open-end: _________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f3_2: In what month and year did you obtain the second refinance loan?

______/______ [RANGE: 2013-2015]

MM/YY

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f4_2. What type of loan is the second refinance: a fixed-rate, line of credit, balloon/graduated payment, adjustable-rate, or some other type of loan?

1. Fixed (FRM)

2. Line of Credit

3. Balloon/graduated

4. Adjustable

5. Other, specify: __________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f5_2. What is the term of the second loan, that is, what is the total number of months over which loan payments are to be made?

1. 30 years or 360 months

2. 25 years or 300 months

3. 20 years or 240 months

4. 15 years or 180 months

5. 10 years or 120 months

6. Or some other number of months (Specify: _____________MONTHS)

8. DON’T KNOW

9. REFUSED

E8f6_2. What is the initial interest rate on the second refinance loan?

$_________

1. DON’T KNOW

2. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 235

APPENDIX

E8f7_2: [If E8f4_2=4] Has your interest rate changed for the second refinance loan? If so, what is the current interest rate for the refinanced loan?

1. Yes, specify rate: ____ % [RANGE: 0.00-15.00]

2. Yes, don’t know new rate

3. No

8. DON’T KNOW

9. REFUSED

E8f8_2. [If E8f4_2=3] How many years before the final large payment of the principal for the balloon loan is made?

_________years [RANGE: 0-30]

8. DON’T KNOW

9. REFUSED

IF E8a=3-7:

E8f1_3: Now think about your third refinanced loan. How much is the principal loan amount for the third refinance loan?

$_________ [RANGE: 0-999,999+]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f2_3: What is the name of the bank or lender where you have your third refinance loan?

Open-end: _________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f3_3: In what month and year did you obtain the third refinance loan?

______/______ [RANGE: 2013-2015]

MM/YY

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

E8f4_3. What type of loan is the third refinance: a fixed-rate, line of credit, balloon/graduated payment, adjustable-rate, or some other type of loan?

1. Fixed (FRM)

2. Line of Credit

3. Balloon/graduated

4. Adjustable

5. Other, specify: __________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 236

APPENDIX

E8f5_3. What is the term of the third loan, that is, what is the total number of months over which loan payments are to be made?

1. 30 years or 360 months

2. 25 years or 300 months

3. 20 years or 240 months

4. 15 years or 180 months

5. 10 years or 120 months

6. Or some other number of months (Specify: _____________MONTHS)

8. DON’T KNOW

9. REFUSED

E8f6_3. What is the initial interest rate on the third refinance loan?

$_________

8. DON’T KNOW

9. REFUSED

E8f7_3: [If E8f4_3=4] Has your interest rate changed for the third refinance loan? If so, what is the current interest rate for the refinanced loan?

1. Yes, specify rate: ____% [RANGE: 0.00-15.00]

2. Yes, don’t know new rate

3. No

8. DON’T KNOW

9. REFUSED

E8f8_3. [IfE8f4_3=3] How many years before the final large payment of the principal for the balloon loan is made?

_________years [RANGE: 0-30]

8. DON’T KNOW

9. REFUSED

The next set of questions asks about the payments that you have made on the loans that are secured by your house. Please think about the payments you have made on all of the mortgage loans and home equity lines of credit that we have discussed for this home.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 237

APPENDIX

E9. Some people have had difficulty recently making their mortgage payments. Since purchasing the home, have you ever missed a monthly payment on a mortgage loan [or home equity line of credit]? 1. YES

2. NO [SKIP TO E14]

8. DON’T KNOW [SKIP TO E10]

9. REFUSED [SKIP TO E10]

E9a. What is the longest amount of time that you have been behind?

1. 0-30 days

2. 31-60 days

3. 61-90 days

4. 91 days or more

8. DON’T KNOW

9. REFUSED

E9b. Are you currently behind in your mortgage or loan payments?

1. YES

2. NO [SKIP TO E 10]

8. DON’T KNOW [SKIP TO E10]

9. REFUSED [SKIP TO E10]

E9c. Currently, how behind are you on mortgage or loan payments?

1. 0-30 days

2. 31-60 days

3. 61-90 days

4. 91 days or more

8. DON’T KNOW

9. REFUSED

E10. On your current home, have you received a notice of intent to foreclose from your bank or lender? 1. YES

2. NO [IF E9=8 OR 9; SKIP TO E14, OTHERWISE PROCEED TO E11]

8. DON’T KNOW [IF E9=8 OR 9; SKIP TO E14, OTHERWISE PROCEED TO

E11]

9. REFUSED [IF E9=8 OR 9; SKIP TO E14, OTHERWISE PROCEED TO E11]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 238

APPENDIX

E10a. Did you lose your home to foreclosure?

1. YES, when___________ [RANGE: 2013 – 2015]

2. NO

8. DON’T KNOW

9. REFUSED

E11. What caused you to get behind on your mortgage? [DO NOT READ LIST, SELECT ALL

THAT APPLY.]

1. I forgot to mail the check.

2. The check got lost in the mail.

3. My mortgage payments were always too high

4. My mortgage payments increased.

5. I had trouble paying homeowners insurance.

6. I had trouble paying property taxes.

7. I had trouble paying for home repairs or maintenance.

8. I had too much credit card debt or other debts.

9. My car expenses were too high.

10. I had a business venture that failed.

11. I lost my job.

12. I took a pay cut.

13. I or someone in my family got injured or had a medical emergency.

14. I have a chronic medical condition or disability.

15. I had a divorce or separation.

16. I had a death in my family.

17. Other (SPECIFY)

98. DON’T KNOW

99. REFUSED

E12. Did you contact your lender for assistance regarding your missed payment(s)?

1. YES

2. NO [SKIP TO E13]

8. DON’T KNOW [SKIP TO E13]

9. REFUSED [SKIP TO E13]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 239

APPENDIX

E12a. [IF RESPONDENT CONTACTED LENDER; E12=1] When did you first contact

your lender? [READ LIST]

1. After you received the foreclosure notice

2. 91 days or more after the missed payment

3. 61-90 days after the missed payment

4. 31-60 days after the missed payment

5. 0-30 days after the missed payment

6. Before you missed a payment

7. At another time

8. DON’T KNOW

9. REFUSED

E13. Did you contact a housing counseling agency, consumer credit counseling agency, or other nonprofit organization for assistance regarding your missed payment(s)?

1. YES

2. NO [GO TO E16]

8. DON’T KNOW [GO TO E16]

9. REFUSED [GO TO E16]

E13a. [IF RESPONDENT CONTACT COUNSELING AGENCY; E13=1], When did

you first contact the counseling agency? [READ LIST]

1. After you received the foreclosure notice [SKIP TO E16]

2. 91 days or more after the missed payment [SKIP TO E16]

3. 61-90 days after the missed payment [SKIP TO E16]

4. 31-60 days after the missed payment [SKIP TO E16]

5. 0-30 days after the missed payment [SKIP TO E16]

6. Before you missed a payment [SKIP TO E16]

7. At another time [SKIP TO E16]

8. DON’T KNOW [SKIP TO E16]

9. REFUSED [SKIP TO E16]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 240

APPENDIX

E14. Now imagine that you have encountered financial difficulty and are about to miss a loan payment. Would you contact your lender for assistance with your missed payment(s)?

1. YES

2. NO [SKIP TO E15]

8. DON’T KNOW [SKIP TO E15]

9. REFUSED [SKIP TO E15]

[ASK E14a IF E14=1]

E14a. When would you first contact your lender? [READ LIST]

1. After you received the foreclosure notice

2. 91 days or more after the missed payment

3. 61-90 days after the missed payment

4. 31-60 days after the missed payment

5. 0-30 days after the missed payment

6. Before you missed a payment

7. At another time

8. DON’T KNOW

9. REFUSED

E15. Would you contact a housing counseling agency, consumer credit counseling agency, or other nonprofit organization for assistance with your missed payment(s)?

1. YES

2. NO [SKIP TO E16]

8. DON’T KNOW [SKIP TO E16]

9. REFUSED [SKIP TO E16]

E15a. When would you first contact the counseling agency? [READ LIST]

1. After you received the foreclosure notice

2. 91 days or more after the missed payment

3. 61-90 days after the missed payment

4. 31-60 days after the missed payment

5. 0-30 days after the missed payment

6. Before you missed a payment

7. At another time

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 241

APPENDIX

E16. Imagine that you were faced with an unexpected home repair that costs $2,000. Would you be able to pay for the expense using your savings or money from family or friends?

[Prompt: Answer no if you would pay for the expense using credit or debt that is not a loan from a family member or friend.]

1. YES

2. NO

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

Section F: Monthly Housing Costs

INTRO: Now, I would like to ask you a few questions about your monthly housing costs.

[ASK F0 QUESTIONS IF RESPONDENT RENTS; A2=1. ALL OTHERS SHOULD SKIP TO

F1]

F0. How much do you spend each month on rent? (Interviewer Note: If R is in a housing situation where R is paying rent with someone, this question asks for the amount of money that only the respondent pays each month for rent.)

1. GAVE RESPONSE___________ [RANGE: 1-9,999+]

8. (VOL) DON’T KNOW (Skip to F0b)

9. (VOL) REFUSED (Skip to F0b)

F0a. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO F0]

F0b. Is your monthly rental payment automatically deducted from a bank

account?

1. Yes

2. No

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

F0c. Utilities include things like gas, electricity, water, and trash removal. They don’t include things like cable TV, Internet, or telephone. How much do you

spend each month on utilities? Do not include any utilities that are included in

your rent. 1. GAVE RESPONSE _________________[RANGE: 0-9,999+]

8. DON’T KNOW [SKIP TO F0e]

9. REFUSED [SKIP TO F0e]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 242

APPENDIX

F0d. To confirm, you said ___________________. Did I get that right?

1. YES

2. NO [GO BACK TO F0c]

F0e. Other than the costs for rent and utilities, do you pay any other monthly costs related to housing?

1. Yes. Specify expense______________.

2. No [SKIP TO F1 IF GROUP=1 OR GROUP=2; IF GROUP=3, SKIP

TO SECTION G]

8. REFUSED [SKIP TO F1 IF GROUP=1 OR GROUP=2; IF GROUP=3,

SKIP TO SECTION G]

9. DON’T KNOW [SKIP TO F1 IF GROUP=1 OR GROUP=2; IF

GROUP=3, SKIP TO SECTION G]

F0f. How much do you pay each month for that expense?

$______________

8. DON’T KNOW

9. REFUSED

F1. When you were searching for homes, did you use an electronic or written budget to determine how much you could afford to pay each month for your mortgage and other housing expenses?

1. Yes

2. No [SKIP TO F5]

8. (VOL) DON’T KNOW [SKIP TO F5]

9. (VOL) REFUSED [SKIP TO F5]

F1a. Did this electronic or written budget include?

1. Utilities

2. Transportation or Commuting Costs

3. Savings for home maintenance and unexpected repairs

ANSWER CHOICES

1. Yes

2. No

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 243

APPENDIX

F3. Did you use your electronic or written budget to determine the price of homes that you could afford?

1. Yes

2. No [Skip to F5]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

[GROUP=3 – SKIP TO SECTION G]

F4. Was your budgeted amount more, less, or the about the same as the actual purchase price of the home? Choose “about the same” if the budgeted amount was within $5,000 of the purchase price.

1. More

2. Less

3. The same

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

IF D9=6 (paid cash) skip to logic above F7.

Now, I’d like to start by asking about the monthly payments on your mortgage.

F5. How much is your regular required payment to the lender on your current/first mortgage?

Please include all payments to your lender for your mortgage, including any money towards an escrow account.

$__________

F5a. How often do you make these payments?

1. Monthly

2. Biweekly (every 2 weeks)

3. Quarterly

4. Other

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 244

APPENDIX

F5b. Is the amount that you pay your lender automatically deducted from a bank account?

1. Yes

2. No

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

F5c. Sometimes lenders require an escrow account be set up to pay for taxes, home insurance, mortgage insurance, or other payments. An escrow account is sometimes also referred to as a

‘reserve’ or ‘impound’ account. Does this payment include money towards an escrow account?

1. Yes [SKIP TO F6]

2. No [SKIP TO F5e]

3. DON’T KNOW [SKIP TO F5e]

4. REFUSED [SKIP TO F5e]

F5e. Do you make a separate escrow payment?

1. Yes

2. No [SKIP TO F7]

3. DON’T KNOW [SKIP TO F7]

4. REFUSED [SKIP TO F7]

F5f. How much is your regular escrow payment? [IF D0=1: This can be found on your Settlement Statement in the Loan Terms box on about page 3 that has a line ‘Total monthly amount owed including escrow account payments.’ IF D0=2: This can be found around the middle of page one of the Closing disclosure under the heading Projected Payments. The total is listed next to the label Estimated Escrow.]

1. $_______________

2. DON’T KNOW

3. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 245

APPENDIX

F6. Now think about your regular payments to your lender and escrow account. Please tell me whether your payments include the following: (READ LIST AND CHECK ALL THAT

APPLY) 1. Principal and interest

2. Real estate taxes

3. Property or homeowners insurance

4. Mortgage Insurance

5. Homeowners Association or Condo Fees

6. Home warranty costs

7. Other (specify)

8. Don’t know

9. Refused

[ASK F7 FOR ANY F6 CHOICE 2-7 THAT ARE NOT SELECTED, IF F6 WAS SKIPPED

ASK ALL]

F7. We would like to know more about some additional costs you may have that make up your housing payments each month. I’m going to read you a list of costs. For each, please tell me whether this is an expense you are responsible for paying, how often you make the payment, and how much the payment costs.

Are you responsible for paying……

[ASK IF CHOICE 4 NOT SELECTED

[ASK IF F7_C1=1]

[ASK IF F7_C1=1]

AT F6]

IF D9=6 skip to logic above F7_D1.

F7_C2. How often do you

F7_C3. What is the

F7_C1. Mortgage insurance?

typically make payments on amount of your

your mortgage insurance?

payment towards

1. Yes

this expense?

2. No

1. Monthly

8. DK

2. Biweekly (every 2

$_____________

9. REF

weeks)

8. DON’T KNOW

3. Quarterly

9. REFUSED

4. Other

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 246

APPENDIX

[ASK IF CHOICE 2 NOT SELECTED

[ASK IF F7_D1=1]

[ASK IF F7_D1=1]

AT F6]

F7_D2. How often do you

F7_D3. What is the

F7_D1. Real estate taxes?

typically make payments on amount of your

your real estate taxes?

payment towards

1. Yes

this expense?

2. No

1. Monthly

8. DK

2. Biweekly (every 2

$_____________

9. REF

weeks)

8. DON’T KNOW

3. Quarterly

9. REFUSED

4. Other

8. DON’T KNOW

9. REFUSED

[ASK IF CHOICE 3 NOT SELECTED

[ASK IF F7_E1=1]

[ASK IF F7_E1=1]

AT F6]

F7_E2. How often do you

F7_E3. What is the

F7_E1. Homeowner’s insurance?

typically make payments on amount of your

your homeowner’s

payment towards

1. Yes

insurance?

this expense?

2. No

8. DK

1. Monthly

$_____________

9. REF

2. Biweekly (every 2

8. DON’T KNOW

weeks)

9. REFUSED

3. Quarterly

4. Other

8. DON’T KNOW

9. REFUSED

[ASK IF CHOICE 5 NOT SELECTED

[ASK IF F7_F1=1]

[ASK IF F7_F1=1]

AT F6]

F7_F2. How often do you

F7_F3. What is the

F7_F1. Homeowner’s association or

typically make payments on amount of your

condo/coop fees?

your homeowner’s

payment toward this

association or condo/coop

expense?

1. Yes

fees?

2. No

$_____________

8. DK

1. Monthly

8. DON’T KNOW

9. REF

2. Biweekly (every 2

9. REFUSED

weeks)

3. Quarterly

4. Other

8. DON’T KNOW

9. REFUSED

[ASK IF CHOICE 6 NOT SELECTED

[ASK IF F7_G1=1]

[ASK IF F7_G1=1]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 247

APPENDIX

AT F6]

F7_G2. How often do you

F7_G3. What is the

F7_G1. Home warranty costs?

typically make payments on amount of your

your home warranty costs?

payment towards

1. Yes

this expense?

2. No

1. Monthly

8. DK

2. Biweekly (every 2

$_____________

9. REF

weeks)

8. DON’T KNOW

3. Quarterly

9. REFUSED

4. Other

8. DON’T KNOW

9. REFUSED

[ASK IF CHOICE 7 NOT SELECTED

[ASK IF F7_H1=1]

[ASK IF F7_G1=1]

AT F6]

F7_H2. How often do you

F7_H3. What is the

F7_H1. Any other costs? Please do not

typically make payments on amount of your

include principal and interest.

any other costs?

payment towards

this expense?

1. Yes

1. Monthly

2. No

2. Biweekly (every 2

$_____________

8. DK

weeks)

8. DON’T KNOW

9. REF

3. Quarterly

9. REFUSED

4. Other

8. DON’T KNOW

9. REFUSED

F11. Utilities include things like gas, electricity, water, and trash removal. They don’t include things like cable TV, Internet, or telephone. How much do you spend each month on utilities?

Do not include any utilities that are included in the monthly amount you pay your lender or for your escrow account. [If you make payments on a quarterly, bi-annual, or annual basis we can help convert that to a monthly amount.]

1. PER MONTH: $_______________________

2. PER QUARTER: $_____________________

3. BI-ANNUAL/TWICE A YEAR: $_______________________

4. ANNUAL/ONCE PER YEAR: $__________________________

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 248

APPENDIX

F12. Other than the costs we covered in this section, do you pay any other monthly costs related to housing?

1. Yes. Specify_________

2. No [SKIP TO F13]

8. DON’T KNOW [SKIP TO F13]

9. REFUSED [SKIP TO F13]

F12a. How much do you pay each month for that expense?

$______________

8. Don’t know

9. Refused

F13. [If F1=1] Think back to the budget you created when you were searching for homes. Now think about the total housing costs you pay each month, including your loan payments, utilities, and any other housing costs. Are your current monthly housing costs higher, lower, or about what you expected to pay each month based on your budget?

1. Higher

2. Lower

3. About what you expected

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

F14. Do you receive each month a homeownership voucher or other monthly subsidy to cover a portion of your monthly mortgage payments, such as HUD’s Section 8 Homeownership Voucher Program?

1. Yes

2. No [SKIP TO F15]

8. REFUSED [SKIP TO F15]

9. DON’T KNOW [SKIP TO F15]

F14a. How much do you receive each month?

1. $______________________ AMOUNT

8. DON’T KNOW

9. REFUSED

F15. Since moving into your new home, has any part of the home stopped working or otherwise needed unexpected repairs? Do not include any upgrades or home improvements that you expected to make when you moved into the home.

1.

Yes

2.

No [SKIP TO F16]

8. DON’T KNOW [SKIP TO F16]

9. REFUSED [SKIP TO F16]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 249

APPENDIX

F15a. What part of the home needed repair? [DO NOT READ LIST. CHECK ALL THAT

APPLY]

1. PLUMBING

2. ROOF, WINDOWS, OR WALLS

3. APPLIANCES SUCH AS STOVE, REFRIGERATOR OR WASHING

MACHINE

4. HEATING OR AIR CONDITIONING SYSTEMS

5. ELECTRICAL SYSTEMS SUCH AS FUSES, CIRCUIT BREAKERS

AND WIRING

6. OTHER (SPECIFY) ___________________________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

F15b. Did any unexpected repair cost more than $500 to fix?

1. Yes

2. No [SKIP TO F16]

8. DON’T KNOW [SKIP TO F16]

9. REFUSED [SKIP TO F16]

F15c. In total, how much did the unexpected expenses cost? If you have made more than one unexpected repair, include the total cost for all repairs.

1. $ _______________ [RANGE: 501 – 9999+]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

F15d. How did you pay for the expense(s)? Did you use… [READ LIST AND CHECK

ALL THAT APPLY]

1. Savings

2. Credit Cards

3. Loan from a lender

4. Money from Friends or Family

5. Did not pay – decided not to fix or could not afford to fix

6. Other; Please Specify: ______________________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 250

APPENDIX

F16. Since moving into your new home, have you experienced any other unexpected expenses of more than $500 that were not related to the house? For example, unexpected expenses might include medical bills, car repairs, and other bills that you did not expect.

1. Yes

2. No [SKIP TO SECTION F17]

8. (VOL) DON’T KNOW [SKIP TO SECTION G]

9. (VOL) REFUSED [SKIP TO SECTION G]

F16b. What types of expenses were they? [CHECK ALL THAT APPLY]

1. Medical bills

2. Car repairs

3. Other: Specify___________________

8. DON’T KNOW

9. REFUSED

F16c. In total, how much did the unexpected expenses cost? If you have had more than one unexpected expense, include the total cost for all expenses.

1. $ __________________ Amount

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

F16d. How did you pay for the expense(s)? Did you use [READ LIST]: [CHECK ALL

THAT APPLY]

1. Savings

2. Credit cards

3. Loan from a lender

4. Money from friends or family

5. Other; SPECIFY: ______________________

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 251

APPENDIX

Section G: Homebuyer Education and Counseling

[This section begins with asking questions to all study participants.]

INTRO: When buying a home, some people complete homebuyer education programs, also referred to as homebuyer counseling, or homebuyer training. These programs may include homebuyer classes, homebuyer education on the Internet and in-person or telephone counseling.

[PROMPT: If the respondent asks about whether to include the services offered through the study, say: “Please include any homebuyer education or counseling that you completed as part of the study.”]

The first set of questions asks about any one-on-one counseling that you received through a homebuyer program. Then I will ask you some questions on any homebuyer education programs you may have participated in.

G1. Since enrolling in the study, have you completed any one-on-one homebuyer counseling?

Homebuyer counseling usually involves a one-on-one session with a certified housing counselor to discuss your specific circumstances either in-person at a local agency or over the telephone.

1. YES

2. NO (SKIP TO G2)

8. (VOL) DON’T KNOW [SKIP TO G2]

9. (VOL) REFUSED [SKIP TO G2]

G1a. Did you complete the homebuyer counseling over the telephone, in-person at an agency, or through some other means?

1. Over the telephone

2. In-person

3. Other [Please specify: ______________]

8. DON’T KNOW

9. REFUSED

G1b. Thinking about the entire counseling session, how many total hours of one-on-one homebuyer counseling did you receive? Do not include any time spent in

homebuyer education classes, workshops or online courses.

1. ______________________ hours

8. DON’T KNOW

9. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 252

APPENDIX

G2. Now I’d like to ask you about any homebuyer education programs you may have

participated in. This includes educational instruction provided in a group workshop or through an online course and can take from one to ten hours. Since enrolling in the study about a year ago have you participated in any homebuyer education?

1. YES

2. NO [SKIP TO LOGIC BEFORE G4]

8. (VOL) DON’T KNOW [SKIP TO LOGIC BEFORE G4]

9. (VOL) REFUSED [SKIP TO LOGIC BEFORE G4]

G2a. Did you complete the homebuyer education online, in-person at a housing agency, or through some other means?

1. Online (using the Internet)

2. In-person

3. Other [Please specify: _________________]

8. DON’T KNOW

9. REFUSED

G2b. Thinking about the entire education course, how many total hours of instruction did you receive? Do not include any time spent in one-on-one homebuyer counseling

or information gathering you did on your own. [Prompt: If you started but did not complete a course, tell us how many hours you spent on homebuyer education

(and not the total number of hours required to complete the course).]

1. ______________________ hours

8. DON’T KNOW

9. REFUSED

G4. [IF YES TO G1 OR G2, OTHERWISE SKIP TO G7] Was homebuyer education or

counseling required by your lender?

1. YES

2. NO

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

G6. [IF YES TO G1 OR G2, OTHERWISE SKIP TO G7] Did you have to pay for the

homebuyer education program or counseling?

1. YES

2. NO [GO TO G7]

8. (VOL) DON’T KNOW [GO TO G7]

9. (VOL) REFUSED [GO TO G7]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 253

APPENDIX

G6a. How much did it cost?

1. $___________

8. DON’T KNOW

9. REFUSED

IF RA=1

G7. At any point, did your lender ever give you a list of organizations that offer homeownership education and counseling services?

1. YES [SKIP TO SECTION H]

2. NO [SKIP TO SECTION H]

8. (VOL) DON’T KNOW [SKIP TO SECTION H]

9. (VOL) REFUSED [SKIP TO SECTION H]

IF RA=2, 3

Intro: As part of the HUD First-Time Homebuyer Study, you were referred to complete homebuyer education and counseling either in-person at a local agency or over the Internet and telephone. Now, I want to ask you a few questions specifically about the homebuyer education and counseling services that you were referred to by the study.

SAMPLE VARIABLES: EDCOMP=1-3; COCOMP=1-3(CASES WHERE RA=1 WILL HAVE

NULL VALUES FOR EDCOMP AND COCOMP

G8. According to our records you [EDCOMP=3 did complete/ED COMP=1 did not complete/

EDCOMP=2 partially completed] the homebuyer education component of this study. This is where you had the opportunity to learn about different topics related to the home purchase process and homeownership [ insert phrase below for remote group, in-person group, or choice selection]

IF RA=2: through an online curriculum called eHome America.

IF RA=3: through an in-person group workshop at a local housing counseling agency.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 254

APPENDIX

G8a. [ASK IF EDCOMP=1, 2:] Which of the following reasons best explains why

you didn’t complete the homebuyer education offered through this study? [READ

LIST AND CHECK ALL THAT APPLY.]

1. The course was too long.

2. I was not interested in participating in the course.

3. The agency was too far away.

4. The homebuyer education did not work for my schedule.

5. The information covered in the homebuyer education did not appeal to me.

6. I did not have transportation to the agency.

7. My Internet connection was too slow.

8. I already bought a house and did not think homebuyer education applied to

me.

9. Other: Please specify [________________________]

98. DON’T KNOW

99. REFUSED

G8b. [ASK IF EDCOMP>1, OTHERWISE SKIP TO G9] Overall, how satisfied

were you with the homebuyer education you received through this study? Would you

say you were…?

1. Very satisfied

2. Somewhat satisfied

3. Neither satisfied nor dissatisfied

4. Somewhat dissatisfied

5. Very dissatisfied

6. DON’T KNOW

7. REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 255

APPENDIX

G8c. [ASK IF EDCOMP=3, OTHERWISE SKIP TO G9] Homebuyer education covers many topics. On a scale of 1 through 5, please tell us how helpful the homebuyer education was in each of the following areas: 1 meaning not useful at all and 5 meaning very useful. If the topic was not covered, then respond with not applicable.

Topic

A. Assessing Homeownership Readiness- [INTERVIEWER READ: Topics such as:

the pros and cons of homeownership, the overview of the home purchase process,

housing affordability, and how lenders determine mortgage readiness]

B. Budget and Credit- [INTERVIEWER READ: Topics such as: Tracking expenses,

creating a savings plan, budgeting and saving tips, importance of good credit ,

understanding credit, credit bureaus, reports and scores, and how to fix credit

problems]

C. Financing a Home: [INTERVIEWER READ: Topics such as: Housing

affordability and how to qualify for a loan, predatory loans and how to avoid them, types of mortgage loans, special financing products, steps in the mortgage loan

process, loan application and approval process, and the closing process]

D. Shopping for a Home: [INTERVIEWER READ: Topics such as: The home

buying team, real estate professionals, types of homes and ownership, how to select a home and neighborhood, how to make an offer, negotiating tips, the purchase

contract, inspections, escrow and closing process]

E. Maintaining a Home and Finances: [INTERVIEWER READ: Topics such as: How to maintain and protect a home, energy efficiency, preventive maintenance, home

repairs and improvements, taxes, insurance, what to do if you can’t make a mortgage payment ]

RANGE 1-5, 7=Not applicable, 8=DK, 9=REF

G9. According to our records, you [COCOMP=3: did complete/ COCOMP=1-2: did not

complete] the homebuyer counseling component of this study. This is where you had the opportunity to speak to a housing counselor one-on-one [ insert phrase below for remote group, in-person group, or choice preference] regarding your specific situation in purchasing a home

IF RA=2: over the telephone

IF RA=3: at a local housing counseling agency

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 256

APPENDIX

G9a. [ASK IF COCOMP=1-2] Which of the following best describes why you didn’t

complete the homebuyer counseling session offered through this study? [READ LIST

AND CHECK ALL THAT APPLY.]

1. I did not have the time.

2. I was not interested in this service.

3. The information that was covered in counseling did not appeal to me.

4. The agency was too far away.

5. Housing counseling was not offered on days or times that worked for my

schedule.

6. I already bought a house and did not think the homebuyer counseling applied to me.

7. I did not have transportation to the agency.

8. Other [Specify: _______________________ ]

98. DON’T KNOW

99. REFUSED

G9b. [ASK IF COCOMP>1, OTHERWISE SKIP TO G10] Overall, how satisfied were

you with the homebuyer counseling you received through this study? Would you say you were…?

1. Very satisfied

2. Somewhat satisfied

3. Neither satisfied nor dissatisfied

4. Somewhat dissatisfied

5. Very dissatisfied

8. DON’T KNOW

9. REFUSED

G9c. [ASK IF COCOMP=3] Homebuyer counseling is often tailored to the specific needs of the client. The counselor may have conducted a number of activities to assist you. On a scale of 1 through 5, please tell us how helpful the following counseling activities were to you, where 1 means not helpful at all and 5 means very helpful. If the activity was not covered, then respond with not applicable.

1. Reviewing information about you and your household’s income, expense, debt and savings?

2. Reviewing your credit report(s) with you?

3. Identifying your credit challenges?

4. Developing a household budget?

5. Analyzing your budget and recommending modifications?

6. Conducting various calculations including affordability based on income and debt?

7. Developing a written action plan for you?

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 257

APPENDIX

8. Following up with you after you completed counseling?

9. Making referrals for additional services?

10. Providing you information on delinquency and foreclosure services?

G10. [ASK IF EDCOMP>1 OR COCOMP>1] Would you recommend homeownership

education or counseling to another person in your situation?

1. Yes

2. Yes, but not from the agency in which I received services.

3. No

8. DON’T KNOW

9. REFUSED

Section H: Demographics

[This section applies to all study participants.]

INTRO: Finally, I would like to ask a few questions about your personal characteristics.

H1. Which of the following best describes your current marital status? [READ LIST]

1. Married

2. Living with an unmarried partner

3. Divorced

4. Separated

5. Widowed

6. Single never married

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

H2. How many people, not counting yourself, live with you?

1.________ (people) RANGE = 0-10 [If zero, skip to H5]

8. (VOL) DON’T KNOW [SKIP TO H5]

9. (VOL) REFUSED [SKIP TO H5]

H3. How many of these people are under the age of 18? [CATI: ANSWER CAN NOT BE

GREATER THAN H2]

1. ________ (people) = 0-10

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 258

APPENDIX

H5. Are you currently employed full time or part time? [IF NO, READ CODES 3-6. IF

SELF-EMPLOYED OR HOURS VARY, ASK WHETHER HE/SHE WOULD

TYPICALLY WORK MORE THAN 30 HOURS PER WEEK]

1. Full-time employment (30+ hours per week)

2. Part-time employment (1-29 hours per week)

3. Unemployed and looking for work

4. Not working/Homemaker/Retired

5. Student

6. Other [Specify_____]

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

H6. [SKIP IF LIVE ALONE/NO OTHER ADULTS]: Of the other adults in your household that live with you, how many are employed full time or part time?

1. _____________ # other adults in the household who are employed

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

H6a. In the past year, have you or any of the other employed adults in your household been laid off or lost at least one week of wages due to unemployment? [Prompt: Answer no if the break in unemployment was planned, such as a month break before starting a new job. This question is asking only about unemployment spells that are due to a layoff or some other factor outside of your control.]

1. Yes

2. No [SKIP TO H7]

8. DON’T KNOW [SKIP TO H7]

9. REFUSED [SKIP TO H7

H6b. [If yes] How many weeks of wages have been lost to unemployment in the past

year? 1. _____________ #

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

H7. Are you or any adult household members veterans of the U.S. Armed Services?

1. YES

2. NO

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 259

APPENDIX

H8. What is the highest level of education that you have completed?

1. Less than a high school diploma

2. High school diploma

3. Some college but no degree

4. 2 year degree

5. 4 year degree

6. Graduate/professional degree

8. (VOL) DON’T KNOW

9. (VOL) REFUSED

IF D9= 6 paid cash skip to CI1

H9.

GROUP=3. How many other people do you plan to buy your home with? As a reminder, here we are talking about people who will share ownership and financial responsibility for the home you may buy. In other words, these are individuals who will also be named on the home loan. They may or may not currently live with you.

GROUP=1, GROUP=2: How many other people did you buy your home with? As a reminder, here we are talking about people who share ownership and financial responsibility for the home you bought. In other words, these are individuals who are also to be named on the home loan.

They may or may not currently live with you.

1. Number of co-borrowers ______ [IF zero skip to CI1, else CONTINUE]

8. (VOL) DON’T KNOW [Skip to CI1]

9. (VOL) REFUSED [Skip to CI1]

[IF H9=1, QTY>0, SET QUALIFIED LEVEL=9]

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 260

Works Cited

Advisory Council for the National Industry Standards for Homeownership Education and Counseling. 2013. National Industry Standards for Homeownership Education and Counseling: Guidelines and Code of Ethics Reference Guide. Washington, DC: National Industry Standards for Homeownership Education and Counseling .

http://www.homeownershipstandards.com/Uploads/NISStdsGuidelines_LR_North%20Capi

tol_FINAL.pdf.

Alasuutari, Pertti, Leonard Bickman, and Julia Brennan, eds. 2008. The Sage Handbook of Social Research Methods. Thousand Oaks, CA: Sage.

An, Xudong, Raphael W. Bostic, and Vincent W. Yao. 2015. Financial Literacy and Mortgage Credit: Evidence from the Recent Mortgage Market Crisis. Social Science Research Network (SSRN). http://dx.doi.org/10.2139/ssrn.2631948.

Angrist, J.D., and G. Imbens. 1995. “Two-Stage Least Squares Estimation of Average Causal Effects in Models with Variable Treatment Intensity,” Journal of the American Statistical Association 90: 431–442.

Angrist, J., G.W. Imbens, and D. Rubin. 1996. “Identification of Causal Effects Using Instrumental Variables” (with discussion), Journal of the American Statistical Association 91: 444–472.

Barron, John, and Michael E. Staten. 2012. Is Technology-Enhanced Credit Counseling as Effective as In-Person Delivery? Working paper No. 2012-WP-05. Terre Haute, IN:

Networks Financial Institute, Indiana State University.

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2088917.

Bell, Stephen H., and Laura R. Peck. 2013. “Using Symmetric Predication of Endogenous Subgroups for Causal Inferences about Program Effects under Robust Assumptions: Part Two of a Method Note in Three Parts,” American Journal of Evaluation 34 (3): 413–426.

doi: 10.1177/1098214013489338.

Bloom, Howard S. 1984. “Accounting for No-Shows in Experimental Evaluation Designs,”

Evaluation Review 8 (2): 225–246.

Cameron, A. Colin, and Douglas L. Miller. 2015. “A Practitioner’s Guide to Cluster-Robust Inference,” Journal of Human Resources 50 (2): 317–372.

Collins, J. Michael, and Collin O’Rourke. 2011. Homeownership Education and Counseling: Do We Know What Works? Research Institute for Housing America Research Paper No. 1102.

http://ssrn.com/abstract=1823429.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 261

WORKS CITED

Cutts, Amy Crews, and William Merrill. 2008. “Interventions in Mortgage Default: Policies and Practices to Prevent Home Loss and Lower Costs.” In Borrowing to Live: Consumer and Mortgage Credit Revisited, edited by Nicolas P. Retsinas and Eric S. Belsky: 203–254.

Washington, DC: Brookings Institution Press.

DeMarco, Donna, Nichole Fiore, Debbie Gruenstein Bocian, Shawn Moulton, and Laura Peck.

2016. The First-Time Homebuyer Education and Counseling Demonstration: Early

Insights. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. https://www.huduser.gov/portal/publications/first-

homebuyer-early-insights.html.

DeMarco, Donna, Nichole Fiore, Shawn Moulton, Debbie Gruenstein Bocian, Laura Peck, Sarah Ballinger, Karen Cuenca, Louise Rothschild, and Stephen Whitlow. 2017. The First-Time Homebuyer Education and Counseling Demonstration Baseline Report: Study Design and Implementation. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

https://www.huduser.gov/portal/publications/first-homebuyer-counseling.html.

Fannie Mae. 2018. “2017 Credit Supplement.”

http://www.fanniemae.com/resources/file/ir/pdf/quarterly-annual-

results/2017/q42017_credit_summary.pdf.

Federal Financial Institutions Examination Council (FFIEC). 2013. “FFIEC Median Family Income Report.” https://www.ffiec.gov/pdf/msa13inc.pdf.

———. 2015. “FFIEC Median Family Income Report.”

https://www.ffiec.gov/pdf/msa15inc.pdf.

Federal Housing Finance Agency (FHFA). 2018. “House Price Index Datasets.”

https://www.fhfa.gov/DataTools/Downloads/Pages/House-Price-Index-Datasets.aspx#qpo.

Freddie Mac. 2018. “Monthly Average Commitment Rate And Points On 30-Year Fixed-Rate Mortgages Since 1971.” http://www.freddiemac.com/pmms/pmms30.html.

Harvill, Eleanor L., Laura R. Peck, and Stephen H. Bell. 2013. “On Overfitting in Analysis of Symmetrically Predicted Endogenous Subgroups from Randomized Experimental

Samples: Part Three of a Method Note in Three Parts,” American Journal of Evaluation 34 (4): 545–566. doi: 10.1177/1098214013503201.

Hsueh, JoAnn, Desiree Principe Alderson, Erika Lundquist, Charles Michalopoulos, Daniel Gubits, and David Fein. 2012. The Supporting Healthy Marriage Evaluation: Early Impacts on Low-Income Families: Technical Supplement. OPRE Report 2012-27. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 262

WORKS CITED

Izrael, David, Michael P. Battaglia, and Martin R. Frankel. 2009. “Extreme Survey Weight Adjustment as a Component of Sample Balancing (aka Raking).” SAS Global Forum Paper No. 247-2009. http://support.sas.com/resources/papers/proceedings09/247-2009.pdf.

Joint Center for Housing Studies of Harvard University. 2015. The State of the Nation’s Housing, 2015. Cambridge, MA: Joint Center for Housing Studies of Harvard University.

http://www.jchs.harvard.edu/sites/jchs.harvard.edu/files/jchs-sonhr-2015-full.pdf.

Litwok, Daniel, and Laura R. Peck. 2018. “Variance Estimation in Evaluations with No-Shows: A Comparison of Methods,” American Journal of Evaluation 39 (4). doi:

1098214017749318.

Moulton, Shawn, Laura R. Peck, and Stephen H. Bell. 2014. Social Policy Impact Pathfinder (SPI-Path) Analytic Suite: SPI-Path|Individual User Guide. Bethesda, MD: Abt Associates Inc.

Moulton, Shawn, Laura R. Peck, and Keri-Nicole Dillman. 2014. “Moving to Opportunity’s Impact on Health and Well-Being among High Dosage Participants,” Housing Policy Debate 24 (2): 415–446. doi: 10.1080/10511482.2013.875051.

Moulton, Shawn, Laura Peck, Nichole Fiore, Debbie Gruenstein Bocian, and Donna DeMarco.

2018. Who Participates in Homebuyer Education and Counseling Services and Why?

Insights from HUD’s First-Time Homebuyer Education and Counseling Demonstration.

Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. https://www.huduser.gov/portal/publications/First-Time-

Homebuyer-Education-and-Counseling-Services.html.

Moulton, Shawn, Laura R. Peck, and Adam Greeney. 2017. “Analyzing the Influence of Dosage in Social Experiment, with Application to the Supporting Health Marriage Program.”

American Journal of Evaluation.

http://journals.sagepub.com/doi/10.1177/1098214017698566.

National Association of REALTORS®. 2018a. “Housing Affordability Index.”

https://www.nar.realtor/research-and-statistics/housing-statistics/housing-affordability-

index.

———. 2018b. “Median Sales Price of Existing Single-Family Homes for Metropolitan Areas.”

https://www.nar.realtor/sites/default/files/documents/metro-home-prices-q4-2017-ranked-

median-single-family-2018-02-13.pdf.

Peck, Laura R. 2002. Subgroup Analysis in Social Experiments. Unpublished Ph.D. dissertation.

Robert F. Wagner Graduate School of Public Service, New York University.

———. 2003. “Subgroup Analysis in Social Experiments: Measuring Program Impacts Based on Post Treatment Choice,” American Journal of Evaluation 24 (2): 157–187.

doi: 10.1016/S1098-2140(03)00031-6.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 263

WORKS CITED

———. 2013. “On Analysis of Symmetrically-Predicted Endogenous Subgroups: Part One of a Method Note in Three Parts.” American Journal of Evaluation 34 (2): 225–236. doi: 10.1177/1098214013481666.

Peck, Laura R., and Stephen H. Bell. 2014. The Role of Program Quality in Determining Head Start’s Impact on Child Development. OPRE Report #2014-10. Washington, DC: Office of Planning, Research, and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.

Puma, Michael J., Robert B. Olsen, Stephen H. Bell, and Cristofer Price. 2009. What to Do When Data Are Missing in Group Randomized Controlled Trials. NCEE Report No.

2009-0049. Washington, DC: National Center for Education Evaluation and Regional

Assistance, Institute of Education Sciences, U.S. Department of Education.

https://files.eric.ed.gov/fulltext/ED511781.pdf.

Schochet, Peter Z., and Hanley Chiang. 2009. Estimation and Identification of the Complier Average Causal Effect Parameter in Education RCTs. NCEE Report No. 2009-4040.

Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Spader, Jonathan, and Christopher Herbert. 2017. “Waiting for Homeownership: Assessing the Future of Homeownership, 2015-2035,” Boston College Journal of Law & Social Justice 37: 267–294.

Spader, Jonathan, Daniel McCue, and Christopher Herbert. 2016. Homeowner Households and the U.S. Homeownership Rate: Tenure Projections for 2015-2035. Working paper. Joint Center for Housing Studies of Harvard University. http://www.jchs.harvard.edu/research-

areas/working-papers/homeowner-households-and-us-homeownership-rate-tenure-

projections-2015.

Stein, Roger M., Ashish Das, Yufeng Ding, and Shirish Chinchalker. 2010. “Moody’s Mortgage Metric Prime: A Quasi-Structural Model of Prime Mortgage Portfolio Losses.” Technical Document, Moody’s Research Labs.

https://www.moodys.com/sites/products/ProductAttachments/Moody%27s%20Mortgage%2

0Metrics%20Prime%20White%20Paper.pdf.

Urban Institute. 2018. “Housing Finance at a Glance: A Monthly Chartbook, February 2018.”

https://www.urban.org/sites/default/files/publication/96796/housing_finance_at_a_glance_a

_monthly_chartbook_february_2018_0.pdf.

U.S. Department of Housing and Urban Development (HUD). 2017. Worst Case Housing Needs: 2017 Report to Congress. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

https://www.huduser.gov/portal/sites/default/files/pdf/Worst-Case-Housing-Needs.pdf.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 264

WORKS CITED

U.S. Department of Labor (DOL), Bureau of Labor Statistics. n.d. “Labor Force Statistics from the Current Population Survey,”

https://data.bls.gov/pdq/SurveyOutputServlet?request_action=wh&graph_name=LN_cpsbre

f3.   U.S. Census Bureau, American FactFinder. 2013. “American Community Survey 1-Year

Estimates.” https://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml.

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 265

WORKS CITED

The HUD First-Time Homebuyer Education and Counseling Demonstration—Preliminary Findings ▌pg. 266

Document Outline

- Table of Contents

- List of Exhibits

- List of Boxes

- Foreword

- Executive Summary

- 1. Introduction

- 1.1 Research Question and Outcome Domains

- 1.2 Two Perspectives on Impacts

- 1.3 Report Objectives and Study Timeline

- 1.4 Organization of This Report

- 2. Evaluation Design and Analysis

- 2.1 Enrollment and Characteristics of Study Participants

- 2.2 The Evaluation Design

- 2.2.1 Initial Study Design: Control Group Plus Remote and In-Person Treatment Groups

- 2.2.2 Modified Study Design: Control Group Plus Remote and Choice Treatment Groups

- 2.3 Service Take-up and Completion Rates

- 2.4 Impact Evaluation Questions and Contrasts

- 2.4.1 Primary Research Questions

- 2.4.2 Impacts for Subgroups

- 2.4.3 Impacts by Delivery Mode

- 2.5 The Analytic Process for Computing Impacts

- 2.6 Data Sources

- 2.7 Outcomes for the Evaluation’s Impact Analyses

- 3. The Intervention: In-Person and Remote Homebuyer Education and Counseling

- 3.1 Content and Quality of Study’s Homebuyer Education and Counseling

- 3.2 In-Person Service Delivery

- 3.2.1 In-Person Education

- 3.2.2 In-Person One-on-One Counseling

- 3.2.3 Advantages/Disadvantages of In-Person Service Delivery

- 3.3 Remote Service Delivery

- 3.3.1 Online Education Course

- 3.3.2 Telephone One-on-One Counseling

- 3.3.3 Advantages/Disadvantages of Remote Service Delivery

- 3.4 Comparison of In-Person and Remote Services

- 3.5 Participant Experiences With Services

- 3.6 Key Take-Aways About the Intervention

- 4. Prospective Homebuyers’ Experiences

- 4.1 Homeowner Status and Housing Arrangements

- 4.2 Homebuying Experience of Purchasers

- 5. Impacts on Preparedness and Search

- 6. Impacts on Financial Capability

- 6.1 Impacts on Financial Knowledge

- 6.2 Impacts on Financial Behavior

- 6.3 Impacts on Financial Indicators

- 7. Impacts on Sustainable Homeownership

- 8. Conclusion and Implications

- 8.1 Key Findings

- 8.2 Implications of Findings

- 8.2.1 Impact on Access to Homeownership

- 8.2.2 Impact on Sustainable Homeownership

- 8.3 Insights into Service Delivery Modes

- 8.4 Generalizability of Findings

- 8.4.1 Housing and Credit Environment

- 8.4.2 Study Sample

- 8.5 Next Steps

- Appendix A: Analytic Methods

- A.1 Estimating the Impact of Being Offered Services

- A.2 Estimating the Impact of Taking Up Services

- A.3 Samples Used to Answer Evaluation Questions

- A.4 Methods for Handling Missing Baseline and Outcome Data

- A.5 Balance Testing on Baseline Characteristics of the Analytic Sample

- A.6 Survey Nonresponse Weighting Methods

- A.7 Ability to Detect Impacts: Minimum Detectable Effects

- Appendix B: Data Sources and Measures

- B.1 Data Sources

- B.2 Combining Data Sources for Impact Analysis

- B.3 Measure Construction

- Appendix C: Expanded Results for the Overall Impact of Services

- C.1 Overall Impact (and How to Read the Impact Exhibits in This Appendix)

- C.2 Alternative Estimates of Overall Impact

- Appendix D: Expanded Results for Impacts by Service Delivery Mode

- D.1 Expanded Results by Service Delivery Mode: In-Person, Remote, and Choice

- D.2 Comparison of Impact of Taking Up In-Person Services with Impact of Taking Up Remote Services

- D.3 Comparison of Impact of Choice of Service Modes with Impact of Remote Services

- D.4 Sensitivity of Impacts of In-Person and Remote Services to Using Alternative Samples

- Appendix E: Impacts on Subgroups Defined by Baseline Characteristics

- E.1 Differences in Subgroup Impacts: Gender

- E.2 Differences in Subgroup Impacts: Age

- E.3 Differences in Subgroup Impacts: Area Housing Affordability

- E.4 Additional Subgroup Results

- Appendix F: Impacts on Subgroups Defined by Likelihood of Service Participation

- F.1 Method for Analyzing the Role of Service Participation

- F.2 What Characteristics Predict Participation in Services?

- F.3 What Is the Impact on Those Most Likely to Take Up Any Services?

- F.4 What Is the Impact on Those Most Likely to Complete the Education Curriculum?

- F.5 What Is the Impact on Those Most Likely to Complete One-on-One Counseling?

- F.6 What Is the Impact on Those Most Likely to Complete All Services?

- F.7 Discussion

- Appendix G: Impacts on Subgroups Defined by Likelihood of Home Purchase

- G.1 Method for Analyzing the Role of Home Purchase

- G.2 What Characteristics Predict Home Purchase?

- G.3 What Is the Impact on Those Most Likely to Purchase a Home?

- G.4 Discussion

- Appendix H: Short-Term Follow-Up Survey

- Works Cited

Table of Contents

Table of Contents

List of Exhibits

List of Boxes

Foreword

Executive Summary

1. Introduction

1.1 Research Question and Outcome Domains

1.2 Two Perspectives on Impacts

1.3 Report Objectives and Study Timeline

1.4 Organization of This Report

2. Evaluation Design and Analysis

2.1 Enrollment and Characteristics of Study Participants

2.2 The Evaluation Design

2.2.1 Initial Study Design: Control Group Plus Remote and In-Person Treatment Groups

2.2.2 Modified Study Design: Control Group Plus Remote and Choice Treatment Groups

2.3 Service Take-up and Completion Rates

2.4 Impact Evaluation Questions and Contrasts

2.4.1 Primary Research Questions

2.4.2 Impacts for Subgroups

2.4.3 Impacts by Delivery Mode

2.5 The Analytic Process for Computing Impacts

2.6 Data Sources

2.7 Outcomes for the Evaluation’s Impact Analyses

3. The Intervention: In-Person and Remote Homebuyer Education and Counseling

3.1 Content and Quality of Study’s Homebuyer Education and Counseling

3.2 In-Person Service Delivery

3.2.1 In-Person Education

3.2.2 In-Person One-on-One Counseling

3.2.3 Advantages/Disadvantages of In-Person Service Delivery

3.3 Remote Service Delivery

3.3.1 Online Education Course

3.3.2 Telephone One-on-One Counseling

3.3.3 Advantages/Disadvantages of Remote Service Delivery

3.4 Comparison of In-Person and Remote Services

3.5 Participant Experiences With Services

3.6 Key Take-Aways About the Intervention

4. Prospective Homebuyers’ Experiences

4.1 Homeowner Status and Housing Arrangements

4.2 Homebuying Experience of Purchasers

5. Impacts on Preparedness and Search

6. Impacts on Financial Capability

6.1 Impacts on Financial Knowledge

6.2 Impacts on Financial Behavior

6.3 Impacts on Financial Indicators

7. Impacts on Sustainable Homeownership

8. Conclusion and Implications

8.1 Key Findings

8.2 Implications of Findings

8.2.1 Impact on Access to Homeownership

8.2.2 Impact on Sustainable Homeownership

8.3 Insights into Service Delivery Modes

8.4 Generalizability of Findings

8.4.1 Housing and Credit Environment

8.4.2 Study Sample

8.5 Next Steps

Appendix A: Analytic Methods

A.1 Estimating the Impact of Being Offered Services

A.2 Estimating the Impact of Taking Up Services

A.3 Samples Used to Answer Evaluation Questions

A.4 Methods for Handling Missing Baseline and Outcome Data

A.5 Balance Testing on Baseline Characteristics of the Analytic Sample

A.6 Survey Nonresponse Weighting Methods

A.7 Ability to Detect Impacts: Minimum Detectable Effects

Appendix B: Data Sources and Measures

B.1 Data Sources

B.2 Combining Data Sources for Impact Analysis

B.3 Measure Construction

Appendix C: Expanded Results for the Overall Impact of Services

C.1 Overall Impact (and How to Read the Impact Exhibits in This Appendix)

C.2 Alternative Estimates of Overall Impact

Appendix D: Expanded Results for Impacts by Service Delivery Mode

D.1 Expanded Results by Service Delivery Mode: In-Person, Remote, and Choice

D.2 Comparison of Impact of Taking Up In-Person Services with Impact of Taking Up Remote Services

D.3 Comparison of Impact of Choice of Service Modes with Impact of Remote Services

D.4 Sensitivity of Impacts of In-Person and Remote Services to Using Alternative Samples

Appendix E: Impacts on Subgroups Defined by Baseline Characteristics

E.1 Differences in Subgroup Impacts: Gender

E.2 Differences in Subgroup Impacts: Age

E.3 Differences in Subgroup Impacts: Area Housing Affordability

E.4 Additional Subgroup Results

Appendix F: Impacts on Subgroups Defined by Likelihood of Service Participation

F.1 Method for Analyzing the Role of Service Participation

F.2 What Characteristics Predict Participation in Services?

F.3 What Is the Impact on Those Most Likely to Take Up Any Services?

F.4 What Is the Impact on Those Most Likely to Complete the Education Curriculum?

F.5 What Is the Impact on Those Most Likely to Complete One-on-One Counseling?

F.6 What Is the Impact on Those Most Likely to Complete All Services?

F.7 Discussion

Appendix G: Impacts on Subgroups Defined by Likelihood of Home Purchase

G.1 Method for Analyzing the Role of Home Purchase

G.2 What Characteristics Predict Home Purchase?

G.3 What Is the Impact on Those Most Likely to Purchase a Home?

G.4 Discussion

Appendix H: Short-Term Follow-Up Survey

Works Cited