A Journal of Policy

Development and Research

Selected OutcOmeS Of HOuSing ASSiStAnce

VOlume 20, number 1 • 2018

U.S. Department of Housing and Urban Development  |  Office of Policy Development and Research

Managing Editor: Mark D. Shroder

Associate Editor: Michelle P. Matuga

Advisory Board

Dolores Acevedo-Garcia

Brandeis University 

Ira Goldstein 

The Reinvestment Fund 

Richard K. Green 

University of Southern California 

Mark Joseph 

Case Western Reserve University 

Matthew E. Kahn 

University of California, Los Angeles 

C. Theodore Koebel 

Virginia Tech 

Jens Ludwig 

University of Chicago 

Mary Pattillo 

Northwestern University 

Carolina Reid 

University of California 

Patrick Sharkey 

New York University

Cityscape

A Journal of Policy 

Development and Research

Selected OutcOmeS Of HOuSing ASSiStAnce

VOlume 20, number 1 • 2018

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

The goal of  Cityscape is to bring high-quality original research on housing and community development issues to scholars, government officials, and practitioners. Cityscape is open to all relevant disciplines, including architecture, consumer research, demography, economics, engineering, 

ethnography, finance, geography, law, planning, political science, public policy, regional science, sociology, statistics, and urban studies.

Cityscape is published three times a year by the Office of Policy Development and Research (PD&R) of the U.S. Department of Housing and Urban Development (HUD). Subscriptions are 

available at no charge and single copies at a nominal fee. The journal is also available on line at 

huduser.gov/periodicals/cityscape.html.

PD&R welcomes submissions to the Refereed Papers section of the journal. Our referee process is double blind and timely, and our referees are highly qualified. The managing editor will also respond to authors who submit outlines of proposed papers regarding the suitability of those 

proposals for inclusion in  Cityscape. Send manuscripts or outlines to cityscape@hud.gov.

Opinions expressed in the articles are those of the authors and do not necessarily reflect the views and policies of HUD or the U.S. government.

Visit PD&R’s website, huduser.gov, to find this report and others sponsored by PD&R. Other services of HUD USER, PD&R’s Research and Information Service, include listservs, special interest and bimonthly publications (best practices, significant studies from other sources), access to public use databases, and a hotline (1–800–245–2691) for help with accessing the information you need.

Contents

Symposium

Selected Outcomes of Housing Assistance .................................................................................... 1 

Guest Editors: Meena Bavan and David Hardiman

Guest Editors’ Introduction 

Findings From PD&R’s Multidisciplinary Research Team ........................................................ 3

Length of Stay in Assisted Housing .......................................................................................... 11 

by Kirk McClure

Housing Cost Burden in the Housing Choice Voucher Program: The Impact of  

HUD Program Rules .................................................................................................................. 39 

by Casey Dawkins and Jae Sik Jeon

Opting In, Opting Out: A Decade Later ................................................................................... 63 

by Anne Ray, Jeongseob Kim, Diep Nguyen, Jongwon Choi, Kelly McElwain,  

and Keely Jones Stater

The Quality of Assisted Housing in the United States ............................................................ 89 

by Sandra Newman and Scott Holupka

An International Perspective: Reflection on the Symposium ................................................ 113 

by Kwan Ok Lee

Refereed Papers .................................................................................................................... 115

Prioritizing Homeless Assistance Using Predictive Algorithms:  

An Evidence-Based Approach ................................................................................................. 117 

by Halil Toros and Daniel Flaming

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents ...... 147 

by Matthew Palm

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles ................. 167 

by Jenny Schuetz, Genevieve Giuliano, and Eun Jin Shin

Departments .......................................................................................................................... 191

Data Shop 

First-Time Homebuyers: Toward a New Measure .................................................................. 193 

by Arthur Acolin, Paul Calem, Julapa Jagtiani, and Susan Wachter

Do It Yourself: Obtaining Updated Transit Stop and Route Shapefiles in Urban and 

Nonurban Areas ....................................................................................................................... 205 

by Seva Rodnyansky

Cityscape

iii

Contents

SpAM 

Calculating Varying Scales of Clustering Among Locations ................................................. 215 

by Ron Wilson and Alexander Din

Evaluation Tradecraft 

Household Survey on Tribal Lands: Frame Building Through Rural Address-Based  

Sampling and Traditional Enumeration ................................................................................. 233 

by Carol Hafford, Steven Pedlow, and Nancy Pindus

Correction 

What Have We Learned From Paired Testing in Housing Markets?  ..................................... 241 

by Sun Jung Oh and John Yinger

iv Volume 20 Number 1

Symposium

Selected Outcomes of Housing Assistance 

Guest Editors: Meena Bavan and David Hardiman

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape

1

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

2

Selected Outcomes of Housing Assistance

Guest Editors’ Introduction

Findings From PD&R’s Multi-

Disciplinary Research Team

Meena Bavan

David Hardiman

U.S. Department of Housing and Urban Development

The views expressed in this article are those of the guest editors and do not represent the official positions or policies of the Office of Policy Development and Research, the U.S. Department of Housing and Urban Development, or the U.S. government.

This symposium of  Cityscape presents new information on the primary affordable rental assistance programs of the U.S. Department of Housing and Urban Development (HUD) from research 

produced under the Multi-Disciplinary Research Team (MDRT).

This introduction provides some basic policy context for how these important new additions to the positive evidence on federal housing programs fit with other major recent studies. A brief description of the key findings of the articles follows.

The symposium brings together four studies that are the product of an innovative public-private research delivery vehicle created by HUD’s Office of Policy Development and Research (PD&R). 

This vehicle partners academic and research experts with a rapid procurement method, MDRT.

The studies—both on their own and when taken together as larger body of work—provide a 

valuable addition to the growing body of research on the importance and effectiveness of federal housing programs.

The symposium also includes an international perspective from Kwan Ok Lee of the National 

University of Singapore. Lee (2018) connects the symposium articles to related research in Asian countries with assisted housing programs, including Singapore, China, and Hong Kong.

Affordable housing advocates have described the need for affordable housing, especially among low-income families and persons, as a national crisis. The latest available HUD estimates show 8.3 

million very low-income renter households with “worst case needs” for affordable housing. Watson 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape

3

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Bavan and Hardiman

et al. (2017) found that, although the overall supply of rental stock grew substantially from 2013 

and 2015, the supply of stock affordable to very low-income renters actually decreased.1 The HUD 

estimates on housing needs are consistent with the findings from major academic institutions.2

The articles in this issue bolster the case for the importance and effectiveness of federal rental assistance programs, building on evidence from recent national studies.

HUD recently issued the long-term findings of a landmark study on the effectiveness of different programs in reducing and eliminating homelessness for families with children (Gubits et al., 2016). 

The Family Options Study gathered evidence through the scientific method of random assignment. The study compared the effectiveness of providing HUD rental assistance—Section 8 hous-

ing choice vouchers (HCVs)—with alternative approaches including  rapid re-housing (shorter-term assistance with services),  transitional housing, and more  usual care that often includes emergency shelter assistance. The results were dramatic.

Although all four approaches helped families experiencing homelessness, families provided with HCVs had far better and longer-term outcomes than families provided with the shorter-term 

assistance options. Families receiving HUD rental assistance were far less likely to experience homelessness again, with less than one-half as many episodes of subsequent homelessness as 

families receiving the shorter-term alternatives. Families provided with HCV rental assistance also had positive outcomes in areas other than their housing situation, including many crucial for child development, such as fewer family-child separations, less psychological distress (usually the mother), less economic stress, less domestic violence, better school mobility, fewer behavior and sleep problems, and less food insecurity (Gubits et al., 2016).3 

The findings from the Family Options Study built on HUD’s previous large-scale evidence-

gathering effort in the Welfare to Work Demonstration Program. This major demonstration was also based on random assignment. Although the primary goal of the demonstration was to measure outcomes for employment and self-sufficiency goals, it also provided key findings on the effectiveness of HUD rental assistance programs in achieving their core goals of affordability.

The results from the demonstration showed that HCV rental assistance significantly reduced the likelihood of homelessness, overcrowding, and doubling up among all types of low-income families (Mills et al., 2006). Homelessness was nearly eliminated for families offered a voucher. After 4 years of study, 45 percent of families in the control group (not offered vouchers) reported one of the following situations in the past year: homeless at some point, stayed in an emergency shelter, or doubled up with a relative or friend. This prevalence rate was cut to only 9 percent for families 1  Worst case needs are defined as unassisted very low-income renters who either pay more than one-half their incomes for rent, or live in substandard physical conditions, or both. The estimate of 8.3 million households also does not include other key housing needs—those facing actual homelessness who aren’t counted in the Census Bureau data used for the report, or very low-income homeowners who may face similar cost and conditions problems.

2 See, for example, JCHS (2017).

3 Note also that shorter-term options, such as rapid re-housing, played an important role and did involve less cost. Longer-term assistance benefits did not add significantly to additional overall costs, however ($3,800 total over 3 years). For additional discussion on the Family Options Study, see HUD PD&R (2016).

4

Selected Outcomes of Housing Assistance

Findings From PD&R’s Multi-Disciplinary Research Team

who were offered a voucher. In terms of the two groups’ reported rates of actual homelessness (on the street or in a shelter), the decrease was from 13 to only 3 percent (a 74-percent drop; Wood, Turnham, and Mills, 2008).4

Further recent evidence on the effectiveness of federal rental housing assistance comes from a major and innovative data-matching study: Childhood Housing and Adult Earnings: A Between-

Siblings Analysis of Housing Vouchers and Public Housing. This landmark effort combined and analyzed long-term, large-scale datasets for millions of households that received HUD housing assistance with U.S. Census Bureau data on household employment, earnings, and other major life outcomes (Andersson et al., 2006; HUD PD&R 2017a).

The study found that children whose families receive HUD rental assistance while the children are teenagers grow up to have higher earnings and lower incarceration rates in their early twenties. 

Public housing and housing vouchers were both found to have positive and significant effects.

The researchers analyzed the results for different groups (race or ethnicity and gender) and found positive and statistically significant benefits from childhood residence in assisted housing on young adult earnings for nearly all demographic groups. Specific results in terms of long-term earnings from employment found that, for females, each additional year with public housing assistance as a teenager generated a $488 annual increase in earnings as a young adult. The increase in earnings for females with HCV assistance was a roughly similar $468 per year of assistance. For males, the corresponding estimates are $508 (public housing) and $256 (vouchers) per year in additional 

earnings as a young adult.

Thus, contrary to some speculation or stereotypes, the study found positive effects on later earnings for housing assistance. Both types of affordable housing assistance had positive outcomes relative to not receiving any assistance. Perhaps this finding should not be surprising, as a higher likelihood of such factors as homelessness, housing instability, or reduced family budgets for other necessities would seem likely to have a negative effective on family and life outcomes. Furthermore, the positive outcomes for public housing—including some that were superior to housing 

vouchers for some groups—are encouraging and may show the need for a variety of affordable 

housing options and delivery mechanisms.

The study also found additional important positive effects on incarceration rates. Childhood 

participation in assisted housing was found to reduce the likelihood of incarceration across all household race and ethnicity groups.

This large-scale dataset produced by the Childhood Housing and Adult Earnings study will con-

tinue to be a source for additional research findings and thus has the potential to increase its return on federal investment. That is because of a PD&R request, consistent with other HUD-funded 4 Of the 9 percent of voucher users who experienced housing insecurity, most had left the program willingly or unwillingly due to personal crisis, stints in residential drug treatment or jail, or misunderstandings or noncompliance with program rules.

Cityscape

5

Bavan and Hardiman

studies, to make the resulting matched dataset available to other researchers through the U.S. Census Bureau’s Center for Administrative Records Research and Applications (CARRA).5 The privacy and access controls employed by the Census Bureau made such an arrangement possible.

The Census Bureau’s own data, through the Supplemental Poverty Measure (SPM), provide 

further evidence for the importance and effectiveness of federal rental assistance programs. Briefly, SPM provides a powerful analytic tool for an alternative means of estimating the magnitude, extent, and character of poverty in America. It measures the rate and the demographics of poverty when other key factors, such as taxes and benefits programs, are taken into account. Under this measure of poverty, housing programs lift more than 3 million people, at least one-third of them children, above the poverty line. Put another way, if federal housing assistance were eliminated altogether, the national poverty rate would increase by a full percentage point, from 15.5 to 16.5 percent (as of 2013) with an even greater increase for children in poverty—a 1.4-percent increase from 16.4 to 17.8 percent (Short, 2014).6 

PD&R’s Multi-Disciplinary Research Team

One way that PD&R has sought to more quickly and cost-effectively add to the body of HUD 

evidence-based research and to create information on which to improve policies and programs is through MDRT. PD&R developed the MDRT vehicle to assemble a team of qualified researchers that could be on call to deliver sound, objective research on high-impact policy issues. Researchers are selected for their expertise to produce an array of high-quality, short-turnaround research. 

MDRT researchers use a variety of HUD and external data sources to answer research questions 

relating to HUD’s priority policies and strategic goals.

Reports produced by MDRT are intended to have a high impact. They provide sound, data-based 

research and analysis to answer highly relevant policy questions in a timely manner and produce results that can be applied in practical ways to federal programs for affordable housing and economic development.7

5 For additional background on CARRA and PD&R’s participation and encouragement of its research opportunities, see HUD PD&R (2017b, 2017c). For PD&R’s encouragement of the use of CARRA for cooperative agreements issued under the Research Partnerships vehicle, see huduser.gov/portal/oup/research_partnerships.html. For a list of working papers 

produced through the Census Bureau’s Center for Administrative Records Research and Applications, see https://www.

census.gov/library/working-papers/series/carra-wp.html.

6 For percentage increases in poverty rate, see Short (2014: 12), Table 5a. Although the SPM measure includes housing subsidies from federal, state, and local governments, the vast majority of assistance is from the federal government. For additional findings on the effectiveness of housing assistance using the SPM, see GAO (2015). For additional private research findings on the effectiveness of housing assistance, including through use of SPM, see Fischer (2015) and Sherman, Trisi, and Parrott (2013).

7 Reports from the MDRT are all available in a single location on PD&R’s HUDUSER website at huduser.gov/portal/

publications/mdrt_reports.html. In addition to MDRT, PD&R implemented another vehicle for relatively rapid research results with a high return on investment of federal taxpayer funds, through cooperative agreements with colleges, universities, and other outside nonprofit research organizations—the Research Partnerships program. For more information, see HUD PD&R (2017d).

6

Selected Outcomes of Housing Assistance

Findings From PD&R’s Multi-Disciplinary Research Team

Discussion of Symposium Articles

The article by Kirk McClure not only builds on previous research on the length of stay in assisted housing, but adds a critical piece that has been missing in past attempts (largely due to the limited scope and lack of complex methodology in previous attempts). Using MDRT resources, McClure 

(2018) is able to analyze HUD administrative data over a 20-year period, from 1995 through 2015. 

He applies a critical survival function analysis that analyzes the proportions of a specific cohort (by year of entry) of assisted households that remain in assisted housing (that is, “survive”) after any specified length of stay over a 13-year period.

McClure finds that, although a substantial number of households stay 13 or more years in assisted housing, the typical household in assisted housing stays an average of 6 years. Length of stay also varies by household type. Elderly households stay about 9 years, and nonelderly households with children stay approximately 4 years. The article also finds that the average length of stay in assisted housing has been generally increasing over time for most cohorts of assisted households, influenced by factors such as household characteristics and market conditions.

Casey Dawkins and Jae Sik Jeon examine trends in housing cost burden for HCV households 

between the years 2003 and 2015. They use HUD administrative data for a cohort analysis of those households that initially leased up in 2003 and 2008. The research aims to identify household, housing, and geographic factors associated with housing cost burden in the HCV program.

Dawkins and Jeon (2018) find that housing cost burdens have risen among HCV households since 

2003; the year-to year changes in housing cost burden roughly follow trends in the recent housing market cycle. Housing cost burdens have been particularly high for households with the lowest incomes. Households headed by females, nonelderly persons, non-Hispanic Black persons, and 

persons without a disability were more likely than other households to exhibit severe housing cost burdens. 

Anne Ray, Jeongseob Kim, Diep Nguyen, Jongwon Choi, Kelly McElwain, and Keely Jones Stater 

address the continuing loss of the assisted housing inventory and raises the question on the 

long-term sustainability of affordable housing, particularly for families with children. This article updates Econometrica (2006), a study of the risk of loss of affordable housing from HUD’s multifamily portfolio between 1998 and 2004. Ray et al. (2018) update the 2006 study by replicating the cross-tabulation and multivariate analyses for HUD’s multifamily portfolio, of 18,000 developments and 1.5 million housing units, for 2005 through 2014.

This updated analysis shows a continuing transition from HUD’s older mortgage programs toward greater reliance on Section 8 rental assistance to provide affordable units. More owners made active decisions to opt in to Section 8 assistance in the latter period, and HUD reduced enforcement and foreclosure actions. Factors such as for-profit ownership and low rent-to-FMR (Fair Market Rent) ratios continued to be associated with higher risk of affordability loss, but these factors were less influential in 2005-to-2014 than in the original study.

Ray et al. (2018) also assess the stability of housing for elderly residents and persons with disabilities, funded by HUD’s Section 202 program. They also explore the use of low-income housing Cityscape

7

Bavan and Hardiman

tax credits and HUD refinancing to preserve affordability in Section 8 developments. The analysis finds that these preservation tools are associated with extended affordability for thousands of HUD-assisted properties. Additional preservation initiatives and improved targeting may be needed to preserve other HUD-assisted properties, particularly smaller developments in strong real estate markets.

Finally, Sandra Newman and Scott Holupka focus on the quality of assisted housing and find that the government inspection and quality control systems play a role in providing physically adequate housing to assisted housing residents.

The authors use two separate and interesting measures of housing quality developed using data from the 2011 and 2013 American Housing Survey. Both indices indicate that the quality of assisted housing is comparable with that of unassisted housing (Newman and Holupka, 2018). The 

findings demonstrate that the current inspection and quality control systems appear to achieve the goal of providing physically adequate housing to assisted housing residents. Housing quality varied by the type of assisted housing; for example, disabled households had better housing quality using a voucher compared with living in multifamily housing. For large households, living in the South and living in public housing were associated with considerably worse housing quality.

Acknowledgments 

The guest editors thank all the authors of articles, blind peer reviewers of the symposium, and the editorial staff of  Cityscape, especially John Wehmueller for his invaluable editing and technical expertise.

Guest Editors

Meena Bavan is a social science analyst in the U.S. Department of Housing and Urban Develop-

ment, Office of Policy Development and Research.

David Hardiman is a program analyst in the U.S. Department of Housing and Urban Development, 

Office of Policy Development and Research.

References

Andersson, Fredrik, John C. Haltiwanger, Mark J. Kutzbach, Giordano Palloni, Henry O. Pol-

lakowski, and Daniel H. Weinberg. 2016.  Childhood Housing and Adult Earnings: A Between-Siblings Analysis of Housing Vouchers and Public Housing. PD&R Research Partnerships.  huduser.gov/portal/

publications/Childhood-Housing-Adult-Earnings.html.

Dawkins, Casey, and Jae Sik Jeon. 2018. “Housing Cost Burden in the Housing Choice Voucher 

Program: The Impact of HUD Program Rules,”  Cityscape 20 (1): 39–62.

8

Selected Outcomes of Housing Assistance

Findings From PD&R’s Multi-Disciplinary Research Team

Econometrica, with Meryl Finkel, Charles Hanson, Richard Hilton, Ken Lam, and Melissa Van-

dawalker. 2006.  Multifamily Properties: Opting In, Opting Out and Remaining Affordable. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and 

Research.

Fischer, Will. 2015. “Research Shows Housing Vouchers Reduce Hardship and Provide Platform for Long-Term Gains Among Children.” Center on Budget and Policy Priorities. https://www.cbpp.org/

research/housing/research-shows-housing-vouchers-reduce-hardship-and-provide-platform-for-

long-term.

Government Accountability Office (GAO). 2015.  Federal Low-Income Programs: Multiple Programs Target Diverse Populations and Needs. GAO-15-516. Washington, DC: Government Accountability Office. http://www.gao.gov/products/GAO-15-516.

Gubits, Daniel, Marybeth Shinn, Michelle Wood, Stephen Bell, Samuel Dastrup, Claudia D. Solari, Debi McInnis, Tom McCall, and Utsav Kattel. 2016.  Family Options Study: 3-Year Impacts of Housing and Services Interventions for Homeless Families. Washington, DC: Government Publishing Office; 

U.S. Department of Housing and Urban Development. huduser.gov/portal/publications/Family-

Options-Study.html.

Joint Center for Housing Studies of Harvard University (JCHS). 2017.  The State of the Nation’s Housing 2017. Cambridge, MA: Joint Center for Housing Studies of Harvard University. http://www.jchs.

harvard.edu/research/state_nations_housing.

Lee, Kwan Ok. 2018. “An International Perspective: Reflection on the Symposium,”  Cityscape 20 

(1): 113–114.

McClure, Kirk. 2018. “Length of Stay in Assisted Housing,”  Cityscape 20 (1): 11–38.

Mills, Gregory, Daniel Gubits, Larry Orr, David Long, Judie Feins, Bulbul Kaul, and Michelle 

Wood. 2006.  Effects of Housing Vouchers on Welfare Families. Report prepared by Abt Associates, Amy Jones & Associates, Cloudburst Consulting, and The QED Group. Washington, DC: U.S. 

Department of Housing and Urban Development, Office of Policy Development and Research. 

huduser.gov/portal/publications/pdf/hsgvouchers_1_2011.pdf.

Newman, Sandra, and Scott Holupka. 2018. “The Quality of Assisted Housing in the United 

States,”  Cityscape 20 (1): 89–112.

Ray, Anne, Jeongseob Kim, Diep Nguyen, Jongwon Choi, Kelly McElwain, and Keely Jones Stater. 

2018. “Opting In, Opting Out: A Decade Later,”  Cityscape 20 (1): 63–88.

Sherman, Arloc, Danilo Trisi, and Sharon Parrott. 2013. “Various Supports for Low-Income 

Families Reduce Poverty and Have Long-Term Positive Effects on Families and Children.” Center on Budget and Policy Priorities. https://www.cbpp.org/research/various-supports-for-low-income-

families-reduce-poverty-and-have-long-term-positive-effects.

Cityscape

9

Bavan and Hardiman

Short, Kathleen. 2014. “The Supplemental Poverty Measure: 2013.” Current Population Reports. 

Washington, DC: U.S. Census Bureau. http://www.census.gov/content/dam/Census/library/

publications/2014/demo/p60-251.pdf. 

U.S. Department of Housing and Urban Development, Office of Policy Development and Research 

(HUD PD&R). 2017a. “Childhood Housing and Adult Earnings: A Between-Siblings Analysis of 

Housing Vouchers and Public Housing,”  The Edge, April. huduser.gov/portal/pdredge/pdr-edge-

research-041717.html.

———. 2017b. “Message From PD&R Senior Leadership: Data Democratization and Evidence-

Based Policy,”  The Edge, January. huduser.gov/portal/pdredge/pdr-edge-frm-asst-sec-010917.html. 

———. 2017c. “New Access to Experimental Data Announced,”  The Edge, March. huduser.gov/

portal/pdredge/pdr-edge-frm-asst-sec-030617.html.

———. 2017d. “Message From PD&R Senior Leadership: Research Partnerships: Locally Driven, 

Jointly Supported,”  The Edge, February. huduser.gov/portal/pdredge/pdr-edge-frm-asst-sec-022117.

html.

———. 2016. “Message From PD&R Senior Leadership: Ending Family Homelessness Is Ending 

Child Homelessness,”  The Edge, November. huduser.gov/portal/pdredge/pdr-edge-frm-asst-

sec-110716.html.

Watson, Nicole Elsasser, Barry L. Steffen, Marge Martin, and David A. Vandenbroucke. 2017.  Worst Case Housing Needs: 2017 Report to Congress. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. huduser.gov/portal/publications/

Worst-Case-Housing-Needs.html.

Wood, Michelle, Jennifer Turnham, and Gregory Mills. 2008. “Housing Affordability and Family 

Well-Being: Results From the Housing Voucher Evaluation,”  Housing Policy Debate 19: 381–384. 

http://www.abtassociates.com/reports/Woods_Turnham_Mills_%5B11%5D_HPD.pdf.

10 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Kirk McClure 

University of Kansas

Abstract

This research addresses the length of time that households remain in the various assisted housing programs administered by the U.S. Department of Housing and Urban Development. The research finds that the typical household in assisted housing now stays for about 6 years, and this figure is increasing for all groups of assisted households. The elderly stay for about 9 years, while nonelderly families with children stay for about 4 years. Racial and ethnic minorities seem to stay for longer in the Housing Choice Voucher program, but the 

influence of race and ethnicity is less within the public housing and the Section 8 project-based housing programs. Market conditions influence length of stay in assisted housing 

in a manner suggesting substitution effects. Where the rents on housing in the private 

marketplace are comparatively high or the availability of rental housing is comparatively low, households in assisted housing tend to stay longer.

Introduction

A household enters an assisted housing program by demonstrating income eligibility, usu-

ally after a lengthy waiting period often measured in years (Smith et al., 2015). Once in an 

assisted housing program, the household may remain indefinitely, but a household may 

choose to leave assisted housing for any number of reasons. These reasons include change of 

household needs, eviction for noncompliance with program or landlord rules, loss of income, 

or graduating out of need as income rises.

To inform budget and policy decisions concerning the various rental assistance programs, 

the U.S. Department of Housing and Urban Development (HUD) needs accurate and reliable 

length-of-stay estimates for households in the subsidized housing programs it administers 

(Thompson, 2007). The objective of this research is to use the HUD administrative data to 

analyze specific cohorts of assisted households over time to obtain as complete a picture 

as possible of the influences of actual household experiences on length of stay in assisted 

housing programs. These programs include the public housing program, the Housing Choice 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 11

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

McClure

Voucher (HCV) program, and the Section 8 New Construction/Substantial Rehabilitation 

project-based housing program.1 The analysis examines these programs over time with the 

time spans varying as a function of the available data.

The analysis looks at income level, source of income (for example, income from wages and 

income from public assistance) and various household characteristics (for example, elderly, 

disabled, and nonelderly and nondisabled households), as well as housing market conditions 

(for example, tight versus soft markets) that influence the length of stay of various cohorts of assisted households. In addition to basic length of stay, the analysis replicates and extends the research literature by including survival analysis and other alternative methods that provide a realistic picture of how long households remain in assisted housing.

The research questions addressed are—

1. How long do HUD-assisted households stay in the public housing, HCV, and project-based 

Section 8 programs (examined separately)?

2. Has length of stay changed over time and for various cohorts of households (from 1995 to the present)?

3. What factors (for example, household composition, income, and housing market conditions) 

influence length of stay?

4. Does the distribution of stays reflect a threshold that separates households that exit early from households that stay for an extended period?

The analysis employs a survival analysis approach that includes examination of how house-

hold characteristics as well as external market factors influence length of stay. Significant program changes such as flat rents, tenant contributions, and market rent levels can affect 

the average length of stay observed at a given point in time (Olsen et al., 2005). Changing 

composition in the assisted population can alter the level of turnover because some types of 

households, such as the elderly, may remain in the program longer than others (Ambrose, 

2005; Cortes, Lam, and Fein, 2008; Freeman, 1998). Changes in HUD’s data systems and the 

completeness and quality of reporting from local public housing authorities (PHAs) can affect estimates of length of stay. These issues are examined to the extent that the completeness and quality of reporting from local PHA permits.

Point-in-time length of stay estimates could be affected by changes in the composition of 

the assisted population or changes in the data collection procedures that do not represent 

an actual change in the pattern of decisions by households to exit the program. Much of the 

prior research used a point-in-time methodology to estimate the average lengths of stay for all households in assisted housing programs at the time of the study. The methodology used here 

examines all assisted household over a very long time period allowing identification of how 

lengths of stay for various cohorts have changed over time. Point-in-time studies examine av-

erage lengths of stay without firm knowledge of when a household will leave assisted housing. 

1 Section 8 project-based housing includes only housing with the voucher attached to the unit. Tenant-based vouchers and Section 8 certificates are included in the HCV program.

12 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

The longitudinal approach used here permits estimating the length of stay of all households 

that entered and exited assisted housing from 2000 to 2015 and even longer in some cases.

The research measures different lengths of stay in the assisted housing programs (public housing, HCV, and Section 8 project-based housing) for each of the types of participating households. 

The household descriptors found to be influential on length of stay in prior studies include 

(1) presence and ages of children; (2) race and ethnicity of head of household; (3) elderly and disability status; and (4) income (for example, level or poverty status) and sources (wages, public assistance, and so on).

Where possible, the research generates these estimates over time spans that cover variations in programmatic and market factors that influence lengths of stay. Programmatic factors include 

(1) changes in data reporting system and (2) PHA participation in Moving To Work (MTW) 

and other special initiatives.2

Housing market factors have also been shown to influence length of stay (Freeman, 2005). 

These market factors include (1) vacancy rate, population size, and median rents; (2) region 

of the country; and (3) incidence of poverty.

Prior Research

Several pieces of research have been published addressing the factors that influence the length of stay of a household in assisted housing and the timing of the decision to leave. Most of 

these studies use HUD administrative data to investigate these issues. Collectively, the research demonstrates that the length of stay in assisted housing varies by program, by household type, and by the housing market conditions in which the household resides.

Hungerford (1996) was the first to venture into explaining variation in the length of time that an assisted household remains in a housing program. He employed a hazard model, which estimates 

the probability that a household will leave at any given time. He drew his data from the Survey of Income and Program Participation, a household level longitudinal panel study carried out by the U.S. Census Bureau. The results indicated that elderly households and female-headed 

households tend to remain longer in assisted housing. He found that households with greater educational attainment remain for a shorter period. Households with children also have shorter stays.

Bahchieva and Hosier (2001) examined administrative data from the New York City Housing 

Authority. The lengths of stay in public housing were found to be very long. Half of all spells lasted 42 years or more, and a quarter lasted 55 years or more. New York City is an exceptionally tight, high-priced housing market, and its public housing developments are generally viewed as high-quality. These factors may contribute to long spells in public housing, which may not be the case in other housing markets. The authors found that shorter lengths of stay in public housing were associated with being young, very old, single, White, non-Latino recent immigrant, 

nonuser of public assistance, having a higher income, and living in a smaller apartment.

2 The MTW program permits high-performing PHAs greater flexibility in the administration of their project-based and tenant-based funds so as to test innovative, locally designed strategies to use funds more efficiently. See HUD (2017).

Cityscape 13

McClure

These two studies did not make use of HUD administrative data. With HUD data, the research 

can cover a much wider study area and can capture specific variations between programs. A 

variety of research projects have used HUD administrative data for this purpose.

Lubell, Shroder, and Steffen (2003) used data from HUD’s Multifamily Tenant Characteristics 

System (MTCS) and Tenant Rental Assistance Certification System (TRACS). The MTCS data 

cover public housing and vouchers. The TRACS data cover the Section 8 project-based hous-

ing developments as well as a variety of other project-based subsidy programs. The authors 

focused on both length of stay in assisted housing as well as whether assisted households 

worked. They found that five of every nine nonelderly nondisabled assisted tenants are em-

ployed. They found that the median length of stay was 4.69 years in public housing and 3.08 

years in the HCV program. The shortest stays were found among households with children 

and the longest among elderly households and households with disabilities.

Olsen, Davis, and Carrillo (2005) looked at HUD data from 1995 to 2002 to estimate dif-

ferences in attrition rates among households in the HCV program, but not the various HUD 

project-based programs. The authors found that elderly or disabled households are less likely to leave the program and that the prevailing vacancy rate in the market influenced decisions 

to leave, with greater vacancy rates associated with a lower probability of leaving assisted 

housing. The authors argued that vacancy rates not only describe market softness but also 

moving costs leading to ambiguous expectations for this relationship. A significant contribu-

tion of their research is the analysis of administrative decisions by PHAs. The authors found that large decreases in the HCV program’s payment standard, which sets a ceiling on the maximum amount of subsidy that can be given to any one household, have a very small effect on 

program attrition. The same is true for increases in the tenant contribution, that is, the share of the income of each household that must be contribute toward payment of rent and utilities.

Ambrose (2005) examined households in both the tenant-based HCV program and the 

project-based public housing and Section 8 programs. Rather than looking at the length of 

stay in assisted housing, he employed a hazard rate approach which models the influences 

upon a household’s decision to leave assisted housing at any given point in time. He found 

that both characteristics of households and housing markets influence that decision. Among 

the household characteristics, the likelihood of leaving a program increases with the presence of children and with larger households and decreases among households that are elderly or 

disabled and also decreases among households that are Black or Hispanic. Among employed 

households, he finds limited support for the idea that increased wages increase the likelihood of leaving public housing but not for the other programs. The same is true with income level 

generally; higher income households are more likely to leave. He found mixed results on the 

influence of housing market characteristics. Greater poverty in the neighborhood decreases 

the probability of leaving assisted housing, but higher educational attainment among the 

neighborhood population increases the probability of leaving. Finally, the greater the level of housing price appreciation in the market, the lower the level of leaving housing assistance. 

Ambrose noted the similarities of his findings with Hungerford (1996) a decade earlier.

14 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Cortes, Lam, and Fein (2008) found that the demographic profiles and household composi-

tion of assisted tenants changed, and such changes influence the length of stay. Their study 

particularly focused on how the presence of children influenced length of stay in the HCV 

program, and they found that the presence of an infant or a toddler increases a household’s 

length of stay in the HCV program. The presence of other children, however, reduced the 

effect. The presence of teenagers, especially male teenagers, reduced the length of stay.

Climaco et al. (2008) again examined only households in the HCV program, focusing on the 

use of the portability feature of the program. They examined households that received vouch-

er assistance from 1998 to 2005, finding that 8.9 percent made a portability move. The rate of portability movers was highest among Black households (10.3 percent) compared with White 

households (8.1 percent) and Hispanic households (8.6 percent). Households with young 

children or with a younger head of household were more likely to make a portability move 

than is true for all HCV households. The length of stay in the HCV program is influenced by 

portability moves as these moves are most likely to occur between the fourth and fifth years of participation. The authors found that HCV households that made portability moves relocated 

to census tracts with lower poverty rates.

Haley and Gray (2008) looked at just those households in Section 202 supportive housing for 

the elderly. Their study period was limited to a single year, 2006. They found that residents of Section 202 housing developments had a median stay of 4 years with 18 percent of all households residing in the housing for more than 10 years. Typically, elderly persons admitted to 

Section 202 projects reside for longer periods of time in this kind of housing than do the elderly households admitted to public housing, other multifamily assisted housing, or using vouchers. 

Smith et al. (2015) make an important contribution to the research on length of stay in 

assisted housing. They used data from the Urban Institute’s HOPE VI Panel study to look at 

what happens to housing assistance leavers. This panel followed 887 households from five 

housing developments from 2001 to 2005. During that period, 103 households left housing 

assistance. The authors found that households leave housing assistance for both positive 

and negative reasons. Positive reasons include marriage or a wage increase; negative reasons 

include breaking program rules, being evicted, or being relocated. The housing assistance 

leavers were found to be doing better than those still in public housing or receiving rent subsidies; they had higher incomes, were more likely to be married, and lived in lower poverty, 

safer communities. Not surprisingly, households that left for negative reasons were found to 

be worse off than those who left for positive reasons.

The prior research confirms that multiple factors influence that amount of time that a house-

hold resides in assisted housing. 

These factors include demographic factors such as—

•  Age: Elderly households generally stay longer.

•  Disability: Households with disabled individuals generally stay longer.

•  Children: The presence of children in a household tends to shorten the stay.

Cityscape 15

McClure

•  Gender: Female-headed households tend to stay longer.

•  Race: Minority households, especially Black households, tend to stay longer but also tend to make greater use of portability moves within the HCV program.

•  Income: Higher income is associated with shorter stay.

•  Welfare: The lower income with welfare usage is associated with longer stays.

•  Education: Higher levels of educational attainment are associated with shorter stays.

These factors also include market conditions such as—

•  Vacancy: Researchers disagree on the influence of vacancy rates. Some research suggests that tights markets (low vacancy rates) inhibit moving thus lengthening stays in assisted housing. 

Other research suggests that soft markets (high vacancy rates) contributed to longer stays.

•  Prices: A high level of rent and rent inflation is associated with longer stays.

These factors include administrative decisions within the HCV program such as—

•  Payment standards: Decreases in payment standards are not associated with households leaving the program.

•  Tenant contribution: Increases in the tenant’s contribution toward rent causes greater program attrition.

This information guides the current analysis of length of stay in assisted housing.

Methods and Data

This research assesses the length of stay in assisted housing by households. The research uses methods that calculate the period of assistance for different types of households, in different types of markets, confronting different sets of administrative procedures.

It is important to note at the outset that changes in length of stay vary by household type, by program, and by housing market conditions. All of these variations are examined. The reasons 

that a household chooses to leave assisted housing are not recorded in the administrative data. 

Thus, it is not known if a household left because their income rose so that the household 

graduated out of assisted housing, or if the household had to leave due to noncompliance 

or breakup of a household. Whatever the cause, variations in the length of stay in assisted 

housing can be seen across different household groups. The programs are increasingly serv-

ing older populations. The lengths of stay in these programs will become longer as elderly 

households are prone to longer stays. Variations in length of stay can be seen across different housing markets. As rents continue to rise faster than inflation and faster than the incomes of renter households, extremely low-income renter households will have fewer and fewer private 

market alternatives, preventing them from leaving assisted housing. HUD administrative data 

were explored to parse out these variations.

16 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Administrative data are not always complete or accurate. As was true with prior research, 

many household records contain suspect information requiring decisions on treatment of 

these troublesome data records. Households with missing data, miscoded data, and otherwise 

suspect data were omitted.

The research examined individual households to assess whether tenant behavior changes over 

time through a cohort-based method. This method can provide accurate, readily understand-

able, longitudinal descriptions of the assisted population’s lengths of stay in assisted housing. 

These lengths of stay are analyzed across different points in time, across households of different types, and across different housing markets, all which may affect households’ decisions to exit the various public assistance programs.

The household data were drawn from HUD’s administrative data. These household level data 

were merged with American Community Survey data describing demographic, housing, and 

economic conditions of the markets where the assisted households reside.

The primary database for this study is the recently created Longitudinal Occupancy, Demog-

raphy, and Income (LODI) file that combines data from MTCS and TRACS for 1995 through 

2015. This 21-year timeframe offers the opportunity to better examine any changes over time 

in the length of stay of households in any of the three major HUD rental assistance programs.

The data were collected from three types of files. For the years 2003 through 2015, the data 

came from the combined LODI reporting system, which contained data from the three major 

programs. For the years 1995 through 2002, the data came from two separate systems. The 

MTCS data cover public housing as well as tenant-based vouchers combined with the earlier 

Section 8 Certificate program. The TRACS data cover the various Section 8 project-based 

programs as well as a variety of other multifamily programs. The files were merged to form a 

single dataset covering all reported households in the following programs—

1. The HCV program: This program includes all voucher households reported by PHAs plus 

Section 8 Certificates reported in 2002 or earlier but not including households in the MTW 

program.

2. The public housing program: This program includes all reported public housing households 

from PHAs that were not in the MTW program.

3. MTW PHAs: This program includes all households reported by MTW PHAs whether the 

household is using a tenant-based voucher or project-based assistance.

4. Section 8 project-based program: This program includes all households in units assisted by the regular Section 8 New Construction/Substantial Rehabilitation program.

5. Section 202/8: This project-based program serves the very low-income elderly through 

nonprofit sponsors.

6. Section 202/811 project rental assistance contracts and Section 202/162 project assistance contracts: this group of project-based programs covers housing for the elderly and disabled that received capital advances to nonprofit sponsors.

Cityscape 17

McClure

Note that the following programs were not included in the analysis: (1) the Section 8 Moder-

ate Rehabilitation program, (2) the Rent Supplement program, (3) the Rental Assistance 

Program, (4) the Section 236 program, and (5) the Below Market Interest Rate program.

Analysis

The data were examined to identify their scale and reliability. Survival functions were generated to illustrate the pace of exits from assisted housing by program and the changes in the functions over time. Next, separate analysis of the length of stay in assisted housing was performed by program for different types of households and for different racial and ethnic groups. Last, the analysis examined the possible drivers of the decision of assisted households to exit assisted housing.

Scale of the Data in the Study

Exhibit 1 describes the scale of the data brought to this study. Over 80 million records were included in the study. Note that this exhibit counts all households that entered, left, or remained Exhibit 1

Count of Assisted Household and Percent Ending Participation by Program and 

Year of Reporting

Section 

Housing 

Section 

Year of 

Public 

Moving 

Section 

202/ 

Choice 

8 Project 

Report-

%

Housing  % to Work  %

%

202/8 

% 811/162  %

Voucher 

Based 

ing

Countb

Countc

Count

PRAC 

Counta

Countd

Count

1995

176,152 13

495,737 10

1996

238,845 14

480,771 12

1997

320,084 14

520,614 13

1998

307,090

9

342,646 10

860,593

9

193,981

7

35,554

8

1999

865,491 11

660,252 13

801,643 10

188,825

8

42,556

7

2000

1,178,121 12

883,790 13

845,762 17

198,221 13

52,230 13

2001

1,090,084 15

659,160 13

915,548 13

203,606 11

61,514 11

2002

1,023,810 15

519,920 19

852,434 14

192,555 12

65,276 11

2003

2,018,606 13

1,067,758 18

1,147,450 16

246,791 14

22,158 14

2004

2,033,948 14

1,131,311 20

1,223,538 18

253,507 15

25,655 15

2005

2,079,755 16

1,159,520 21

1,227,866 18

250,573 15

27,824 16

2006

2,231,601 17

1,277,773 23

4,067

0

1,208,650 18

247,528 15

29,558 16

2007

2,236,668 15

1,274,534 20

56,367

5

1,254,894 19

246,232 15

32,393 16

2008

2,266,021 14

1,290,500 19

106,875 12

1,238,125 19

242,021 15

33,391 16

2009

2,262,709 14

1,266,540 18

148,896 14

1,229,023 18

238,475 15

34,463 16

2010

2,239,551 14

1,282,782 18

208,619 15

1,225,216 18

236,434 15

35,948 15

2011

2,211,323 13

1,315,687 18

270,762 17

1,225,002 17

235,064 14

36,842 15

2012

2,164,736 12

1,280,553 16

248,552

5

1,219,145 18

232,829 15

37,477 16

2013

2,158,019 12

1,285,272 17

256,459

8

1,202,938 18

229,948 15

37,827 16

2014

2,159,297 12

1,328,168 20

354,135 25

1,198,642 17

229,105 15

37,954 15

2015

2,206,597 12

1,318,363 21

389,193 27

1,205,568 18

229,409 15

38,730 15

All years  33,468,508 14 20,841,651 18 2,043,925 16 20,082,037 17 4,095,104 14 687,350 14

a  Includes Section 8 tenant-based certificates.

b  Includes only units administered by non-Moving to Work public housing authorities.

c  Includes both project-based and tenant-based units.

d  Does not include Section 202/8 units.

PRAC = project rental assistance contract.

18 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

in each program in each year. Thus, to the extent that a household left, and a new household 

moved in, a unit can be counted multiple times in a single reporting year. The data are less 

comprehensive in the early years of the automated MTCS and TRACS data entry systems. Thus, 

the numbers of households included in the data from 1995 through 1998 are smaller than in 

the later years, 2003 through 2015, when the data collections systems were up to speed.

Exhibit 1 indicates—

•  Data are available and reliable for the public housing program from 1995, although the level of reporting is lower from 1995 through 2002. 

•  The adoption of the MTW program changed the reporting requirements for the PHAs 

participating in that program. Thus, from 2006 forward, the MTW public housing and voucher 

households are reported separately.

•  Data are also available for the HCV program from 1995, but the data are considered to be more reliable from 1999 forward.

•  Data are available for the Section 8 project-based programs from 1998 forward. 

Exhibit 1 lists the percentage of reporting households in each year that ended participation 

in the housing assistance program. The rates of program exiting do vary from program to 

program and over the decades of the study period. However, the general finding is that rates 

of exiting rental assistance do not vary by much. Over the entire study period, the percentage of households that ended participation averaged 14 to 18 percent each year for all programs. 

Thus, 1 in 5 to 1 in 7 assisted households leave each program in each year.

The programs differ in terms of the rates of exit—

•  An average of 14 percent of participating households in the HCV program exit each year. This annual average rate of exit ranged from a low of 9 percent to a high of 17 percent with no clear pattern over the study period.

•  Households in the public housing program ranged from 10 percent to 23 percent with an 

average of 18 percent exiting each year. 

•  Households in the Section 8 project-based housing developments exited at annual rates from 9 percent to 19 percent with an average of 17 percent.

•  The Section 202/8 households and Section 202/811/162 households reported very low rates of exit in the early years of 1998 and 1999. These reports may be unreliable given that they rose very sharply in 2000 and have remained relatively steady since that time. Since 2000, both sets of the Section 202 developments have experienced exit rates ranging from 11 percent to 16 percent per year.

In terms of the percent of assisted households leaving the housing assistance programs in any one year, all programs peaked in the mid-2000s. Exit rates for the HCV program peaked in 

2006, public housing peaked in 2006, and Section 8 project-based housing peaked in 2008. 

The mid-2000s were a period of turmoil in housing markets, but those problems were more 

keenly felt in the markets of owner-occupied housing and not in rental markets.

Cityscape 19

McClure

Survival Functions

Survival functions indicate the proportion of a selected group of assisted households that 

remain in assisted housing (that is, “survive”) after any specified length of stay. Exhibit 2 

presents survival functions for four cohorts of tenants for each of the three major housing 

assistance programs. The cohorts are based on households that lived in or exited from assisted housing during 4 different years: 2015, 2010, 2005, and 2000. The changes in these functions 

display the changing trends in the patterns of staying (or surviving) in assisted housing. The charts illustrate survival from program entry through 13 years, although a small proportion of households may stay substantially longer. Survival function illustrations have the advantage that they identify any thresholds beyond which there are either rapid withdrawals from the programs or stabilization of the stays (Thompson, 2007). The charts answer questions such as—

•  What is the general shape of the survival function?

•  Is there a point where there is rapid exiting from the programs? 

•  Is there a point where the pace of exiting stabilizes?

•  How do the programs compare to each other?

Exhibit 2 illustrates that, typically, 90 percent of households remain in assisted housing 

through the first year. The losses are minimal during the first year, probably because of a year-long lease on the assisted unit. This has the effect of keeping households in the unit. After the first year, the survival function reflects the successive loss of households from the programs for each length of stay in years. In all cases, the figures illustrate a very standard form of survival function, with survival always decreasing but at a decreasing rate over time.

For the HCV program, the shape of the survival functions has not changed dramatically. 

However, they have shifted toward a horizontal axis more so than either the public housing 

program or the Section 8 project-based program, as HCV tenants are choosing to stay longer. 

Perhaps this shift reflects a response to worsening rental housing market conditions. After the first year, the HCV survival functions do not show any dramatic thresholds where the rates 

of exits change substantially. There is no point at which the pace of exits stabilizes. Rather, the most recent functions for 2010 and 2015 show a steady loss of households at a gradually 

slowing pace.

For the public housing and project-based Section 8 programs, the survival functions have 

changed very little over time. Both programs, in all four periods, show a steady decline in 

the percentages of assisted households that remain in the program. Neither program shows 

evidence of any thresholds where the pace of change shifts dramatically or lengths of stay after which the pace of change stabilizes.

20 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Exhibit 2

Survival Functions by Household Cohort and Program Type

(a) Housing Choice Voucher Program

(b) Public Housing Program

(c) Section 8 Project-Based Housing Program

Length of Stay by Program

Exhibit 3 indicates the average length of stay of households that exited the various programs by the year of exit. This exhibit provides the means to compare the typical length of stay 

across the programs and over the entire study period.

Cityscape 21

McClure

Exhibit 3

Average Length of Stay of Households in Assisted Housing by Program by Year of 

Exit

Housing 

Section 

Section 

Public 

Moving to 

Section 

All  

Year of Exit

Choice 

8 Project 

202/811/162 

Housingb

Workc

202/8

Programs

Vouchera

Basedd

PRAC

1995

0.9

4.6

3.5

1996

1.3

4.8

3.5

1997

1.6

4.6

3.4

1998

1.7

4.2

5.3

6.2

2.0

4.5

1999

2.6

3.9

5.0

6.2

2.2

3.8

2000

3.6

4.3

5.0

6.2

2.5

4.4

2001

3.8

5.0

4.5

6.0

2.6

4.4

2002

3.6

5.3

4.5

6.1

2.8

4.4

2003

3.6

5.1

4.5

6.0

2.4

4.4

2004

4.0

5.7

4.7

6.1

2.5

4.8

2005

4.5

6.0

4.7

6.2

2.6

5.0

2006

4.9

6.8

5.0

4.7

6.2

2.8

5.5

2007

4.9

6.1

5.5

4.7

6.2

2.9

5.3

2008

5.1

5.6

6.2

4.8

6.2

2.9

5.2

2009

5.4

5.5

5.5

4.9

6.3

3.1

5.3

2010

5.8

5.9

6.6

5.0

6.4

3.3

5.6

2011

5.8

5.5

6.2

5.0

6.4

3.4

5.5

2012

5.7

5.4

6.1

5.0

6.5

3.5

5.4

2013

6.0

5.5

5.4

5.1

6.5

3.8

5.6

2014

6.5

5.8

6.2

5.1

6.7

4.1

5.9

2015

6.6

5.9

5.7

5.3

6.7

4.3

6.0

Median 2015

4.8

3.0

4.0

3.0

4.4

2.4

3.6

Average all years

4.9

5.6

5.9

4.9

6.3

3.0

5.1

Median all years

3.1

2.6

4.4

2.6

4.1

1.9

2.9

a  Includes Section 8 tenant-based certificates.

b  Includes only units administered by non-Moving to Work public housing authorities.

c  Includes both project-based and tenant-based units.

d  Does not include Section 202/8 units.

PRAC = project rental assistance contract.

The reliability of the data becomes an issue with the examination of this exhibit. During 

the early years of HUD’s automated tenant data systems, the reporting of household stays 

may have been biased. Households that lived in assisted housing for a long time prior to the 

automated systems often did not have had their date of admission recorded. As a result, the 

households with complete records, including both date of admission and date of exit, may 

be biased toward those households that entered assisted housing only a short time before 

the year of exit. Thus, the length of stay figures become more trustworthy after 1998, as the automated systems matured. 

With this caveat, it is apparent that the length of stay in assisted housing has grown longer in all programs over the study period. In 2000, the typical household that ended participation 

in assisted housing lived in that housing for 4.4 years. By 2015, the typical household that 

ended participation had lived in assisted housing for 6.0 years, an increase of 1.6 years. The increase in average length of stay among households that left assisted housing was greatest in 22 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

the HCV program, growing by 3.0 years from 3.6 in 2000 to 6.6 in 2015. The increase was 

smaller in public housing. The average length of stay by households leaving public housing 

was 4.6 years in 1995, falling slightly to 4.3 years in 2000 but growing to 5.9 years in 2015, an increase of 1.6 years. The Section 8 project-based housing program is more stable in terms of the length of stay than the other programs. The Section 8 program had an average length 

of stay of 5.0 years in 2000, rising less than one-third of a year to 5.3 years in 2015, after experiencing small increases and decreases in the intervening years.

The MTW PHAs generally followed the same trend, with increasing length of stay over time. 

Unlike the other programs, households in MTW programs showed a slight drop in average 

length of stay from 2014 to 2015. Because HUD gives discretion to MTW PHAs to alter their 

approach to delivering assisted housing and the mixing of tenant-based households with 

project-based households, this volatility could be expected.

The Section 202/811/162 developments experienced longer than average stays that mirrored 

the regular Section 8 project-based developments. The special needs households served by Sec-

tion 202/811 and 202/162 developments had the shortest average length of stay, but they also 

experienced a large proportional growth in length of stay from 2.5 years in 2000 to 4.3 years in 2015.

Length of Stay by Household Type

Prior research indicates that the length of stay in assisted housing varies with the type of 

household. Elderly households and households with disabilities tend to stay longer than 

nonelderly, nondisabled households. Households without children tend to stay longer than 

households with children.

By 2015, the housing assistance programs helped about 5.1 million households, up from 4.0 

million in 2000. HUD categorized these households by three characteristics of household 

type: (1) elderly or nonelderly, (2) disabled or nondisabled, and (3) with or without children. 

Exhibit 4 indicates that the largest group is elderly households with no children, at 33 percent of the total. In size, the nonelderly with children group follows at 32 percent of the total. 

This is a reversal in the rankings of these two categories. In 2000, nonelderly households 

with children were the largest household type comprising 43 percent of the total. Growth 

in the population of assisted households is almost entirely among elderly households and 

households with disabled members. These two groups grew collectively by about 1 million 

households from 2000 to 2015. Nonelderly households with children grew by fewer than 

20,000 households during the same time period.

The message to take from exhibit 4 is that changes in the composition of the assisted house-

holds very likely drive changes in the length of stays in assisted housing. Elderly households and households with disabled people are known to remain in assisted housing longer than 

nonelderly, nondisabled households. This shift toward more elderly and disabled households 

will generate longer stays in assisted housing independent of changes in other factors.

Cityscape 23

McClure

Exhibit 4

Households in Assisted Housing by Household Type for Years 2000, 2005, 2010, 

and 2015

Year of Reporting

Household Category 

2000

2005

2010

2015

and Subtotals

House-

House-

House-

House-

Percent

Percent

Percent

Percent

holds

holds

holds

holds

Household category

Elderly, no children,   1,215,988 

30

 1,401,791 

30

 1,560,527 

30

 1,667,674 

33

nondisabled

Nonelderly, no 

 503,972 

12

 745,588 

16

 900,883 

17

 917,370 

18

children, disabled

Nonelderly, 

 473,984 

12

 478,776 

10

 569,279 

11

 603,364 

12

no children, 

nondisabled

Elderly, with children, 

 38,030 

1

 45,512 

1

 49,934 

1

 49,953 

1

nondisabled

Nonelderly, with 

 73,280 

2

 237,703 

5

 276,289 

5

 259,398 

5

children, disabled

Nonelderly, 

 1,742,621 

43

 1,823,221 

39

 1,819,037 

35

 1,630,997 

32

with children, 

nondisabled

All households

 4,047,875  100

 4,732,591  100

 5,175,949  100

 5,128,756  100

Household subtotals

Elderly

 1,254,018 

31

 1,447,303 

31

 1,610,461 

31

 1,717,627 

33

Nonelderly

 2,793,857 

69

 3,285,288 

69

 3,565,488 

69

 3,411,129 

67

Disabled

 577,252 

14

 983,291 

21

 1,177,172 

23

 1,176,768 

23

Able bodied

 3,470,623 

86

 3,749,300 

79

 3,998,777 

77

 3,951,988 

77

With children

 1,853,931 

46

 2,106,436 

45

 2,145,260 

41

 1,940,348 

38

No children

 2,193,944 

54

 2,626,155 

55

 3,030,689 

59

 3,188,408 

62

Elderly or disabled

 1,831,270 

45

 2,430,594 

51

 2,787,633 

54

 2,894,395 

56

Nonelderly able 

 2,216,605 

55

 2,301,997 

49

 2,388,316 

46

 2,234,361 

44

bodied

Exhibit 5 examines average length of stay of different households by year of exit from assisted housing, combining all households from all programs. The exhibit is organized by a household type designation used by HUD. This household type designation divides all assisted 

households into six categories based upon three variables indicating if the household is  

(1) elderly or nonelderly, (2) disabled or nondisabled, and (3) membered with children or not.

Exhibit 5 presents lengths of stay of households existing from 1995 through 2015 for these 

household types, extending the period of time studied over the prior research with the caveat that some of the counts are quite small in the early years of 1995 to 2000 for some programs. 

Despite the longer study period, the results are generally similar to previous studies. Elderly households tend to have longer stays at 8 to 9 years compared to less than 5 years for 

nonelderly households. Households with children, which often consist of single mothers 

with children, tend to have shorter lengths of stay. The group of assisted households that has expanded in size the most in recent years is households with disabled members. This group 

tends to have lengths of stay comparable to those of the nonelderly, well short of the stays of the elderly households.

24 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Exhibit 5

Average Length of Stay of Households in Assisted Housing by Household Type by 

Year of Exit

Elderly  

Nonelderly 

Nonelderly 

Elderly  

Nonelderly 

Nonelderly 

Year of Exit Nondisabled 

Disabled  

Nondisabled  Nondisabled 

Disabled  

Nondisabled 

No Children No Children No Children With Children WIth Children WIth Children

1995

7.4

2.9

2.4

6.3

2.3

1.9

1996

7.9

2.3

2.9

6.7

2.0

2.0

1997

7.8

2.4

2.9

7.4

2.5

2.1

1998

7.7

3.0

3.0

8.0

3.4

3.0

1999

7.3

2.8

2.8

7.2

3.1

2.7

2000

7.6

3.4

3.8

8.1

3.7

3.1

2001

7.6

3.5

3.8

7.6

3.5

3.0

2002

7.7

3.6

3.7

8.0

3.6

3.1

2003

7.8

3.6

3.7

8.2

3.6

3.0

2004

8.3

3.9

4.2

9.5

3.8

3.3

2005

8.6

4.2

4.6

10.2

4.2

3.5

2006

9.1

4.7

5.3

12.5

4.7

3.9

2007

8.7

4.5

4.9

10.2

4.4

3.7

2008

8.5

4.5

4.8

9.6

4.4

3.6

2009

8.4

4.6

4.8

9.2

4.5

3.7

2010

8.7

4.8

5.3

9.5

4.9

3.8

2011

8.7

4.8

4.9

9.0

4.9

3.8

2012

8.6

4.7

4.8

8.8

4.5

3.7

2013

8.8

4.9

5.1

9.1

4.6

3.9

2014

9.1

5.0

5.4

9.4

4.9

4.1

2015

9.1

5.1

5.5

9.5

4.9

4.2

Median 2015

6.7

3.1

2.7

6.5

3.3

2.8

 All years 

8.5

4.4

4.5

9.5

4.4

3.5

Median all 

5.9

2.5

2.0

5.9

2.8

2.2

years

 Growth in 

years 

  2000 to 

 1.5 

 1.7 

 1.7 

 1.4 

 1.2 

 1.1 

2015

All six household types listed in exhibit 5 experienced longer stays in assisted housing over the study period. With only a few very small exceptions, each household type’s average length of stay increased with the passage of each year. However, changes from one year to the next 

were not dramatic, and the increases were incremental but unequal. The rank ordering of 

the household types by average length of stay in the late 1990s remained the same in 2015, 

with elderly households staying longest and the nonelderly with children staying the shortest. 

However, the amount of growth was only slightly different. The average length of stay for 

elderly households grew 1.4 to 1.5 years from 2000 to 2015. The stays for the nondisabled, 

nonelderly households grew by 1.1 to 1.7 years, and the stays for households with disabilities grew 1.2 to 1.7 years. 

Average length of stays can be misleading because survival functions of many shapes can have 

the same average. To prevent being misled by this issue but to make the analysis manageable, 

Cityscape 25

McClure

exhibits 6 through 11 follow the procedures used by Cortes, Lam, and Fein (2008). These 

authors examined length of stay of cohorts at three points along the survival function: the 

25th, the 50th (median), and the 75th percentile for each of the three major programs. This 

technique discloses any dramatic shifts in the survival functions, such as a large increase or decrease in the length of stay. Any shift in the survival function will be identified as a significant change in the average length of stay at any one or all of the percentiles.

Exhibit 2 shows that survival functions shifted for households in the HCV program with lon-

ger stays in more recent years. Exhibit 6 helps to identify the patterns of change by household type for the HCV program, providing insights into which household types experienced the 

greatest shifts and in what direction. The simple answer to this issue is that all household 

types experienced some level of increased length of stay over time. All household types, at 

all three percentiles, experienced increases in the length of stay during all three periods from 2000 to 2005, 2005 to 2010, and 2010 to 2015. The scales of the shifts were generally comparable. The 50th percentile, or median length of stay, increased from 0.5 years to 3.3 years over the various time periods, with most increasing at the lower end of this range, from 0.5 to 1.1 years, over any 5-year period. The largest 5-year increases in median stays were for elderly households with increases at the median of over 2 years for elderly households with no 

children and over 3 years for elderly households with children. There was only one exception 

to the pattern of growth in length of stay for the HCV households; the nondisabled elderly 

households with children experienced a slight reduction in the median length of stay from 

2010 to 2015. It is worth noting that this cohort had one of the longest lengths of stay among all the assisted households at more than 7 years at the median. Thus, a slight downward shift is unremarkable for this already long-tenured population.

The survival functions for the public housing households and the Section 8 project-based 

households did not shift by much. Thus, the value of exhibits 7 and 8 is to determine whether Exhibit 6

Length of Stay of Households in the Housing Choice Voucher Program by 

Household Type for Year of Exit

Year 

Elderly  

Nonelderly 

Nonelderly 

Elderly  

Nonelderly 

Nonelderly 

Per-

of 

Nondisabled 

Disabled  

Nondisabled  Nondisabled 

Disabled  

Nondisabled 

centile

Exit

No Children No Children No Children With Children WIth Children WIth Children

2000

25th

1.7

0.9

0.9

1.3

0.9

0.9

50th

4.6

1.8

2.1

3.3

2.0

1.9

75th

10.3

4.2

5.7

8.2

4.6

4.0

2005

25th

2.6

1.4

1.2

2.3

1.5

1.3

50th

5.1

2.9

2.8

4.4

2.9

2.7

75th

10.4

5.3

6.0

8.5

5.0

4.6

2010

25th

3.1

1.7

1.4

3.4

1.9

1.6

50th

7.2

3.9

3.9

7.7

3.9

3.3

75th

11.8

8.2

8.9

12.1

8.0

7.0

2015

25th

3.6

1.7

1.4

3.4

1.9

1.8

50th

8.0

4.2

4.3

7.3

4.4

4.1

75th

13.7

8.8

10.2

13.1

8.3

7.8

26 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Exhibit 7

Length of Stay of Households in the Public Housing Program by Household Type by 

Year of Exit

Year 

Elderly  

Nonelderly 

Nonelderly 

Elderly  

Nonelderly 

Nonelderly 

Per-

of 

Nondisabled 

Disabled  

Nondisabled  Nondisabled 

Disabled  

Nondisabled 

centile

Exit

No Children No Children No Children With Children WIth Children WIth Children

2000

25th

2.1

0.6

0.5

1.5

0.8

0.7

50th

5.8

1.7

1.2

4.8

1.8

1.6

75th

12.9

4.0

3.4

16.9

4.4

3.5

2005

25th

2.9

0.9

0.7

3.0

0.9

0.9

50th

7.3

2.3

2.0

9.5

2.4

2.0

75th

15.9

5.3

5.7

25.8

5.5

4.3

2010

25th

2.9

1.0

0.9

2.4

1.1

0.9

50th

7.1

2.5

2.1

6.3

2.4

2.0

75th

14.6

5.6

5.8

16.6

5.3

4.1

2015

25th

3.0

1.1

0.9

2.4

1.1

1.0

50th

7.6

2.7

2.4

6.5

2.5

2.3

75th

15.6

6.1

6.5

15.4

5.4

4.7

Exhibit 8

Length of Stay of Households in the Section 8 Project-Based Housing Program by 

Household Type by Year of Exit

Year 

Elderly  

Nonelderly 

Nonelderly 

Elderly  

Nonelderly 

Nonelderly 

Per-

of 

Nondisabled 

Disabled  

Nondisabled  Nondisabled 

Disabled  

Nondisabled 

centile

Exit

No Children No Children No Children With Children WIth Children WIth Children

2000

25th

2.4

1.0

0.7

2.4

1.0

0.9

50th

6.2

2.2

1.8

6.1

2.2

1.9

75th

13.1

5.4

4.8

15.0

5.4

4.1

2005

25th

2.3

0.9

0.7

1.8

1.0

0.9

50th

6.0

2.1

1.6

4.5

2.1

1.9

75th

12.7

4.7

3.6

12.1

4.7

3.7

2010

25th

2.4

0.9

0.8

2.0

1.1

0.9

50th

6.1

2.2

1.8

5.4

2.3

1.9

75th

12.4

4.9

4.5

12.8

4.7

3.8

2015

25th

2.6

1.1

0.9

2.2

1.1

1.0

50th

6.5

2.6

2.0

5.4

2.7

2.3

75th

13.1

5.7

4.9

12.0

5.8

4.5

the lengths of stay among exiting household remained stable for all household types. It is possible that the stable overall survival functions masked significant shifts in different directions between household types that washed out when combined. The message from these two tables 

is that both public housing and Section 8 project-based housing experienced, with only minor 

exceptions, very small shifts in lengths of stay across all household types across all time periods.

The shifts from one time period to the next in lengths of stay were nearly all small fractions of a year. The same is true for shifts at the 25th and 75th percentiles. The only exception to this shift of any scale is the population of elderly households that are caring for children. This Cityscape 27

McClure

population is largely the caretakers for their grandchildren because the children’s parents were no longer able to care for them leaving the elderly to care for grandchildren (Pebley and Rudkin, 1999). This particular cohort experienced some volatility, both up and down, in terms of length of stay. It is important to realize this cohort has the longest length of stay at the upper reaches of the survival function in all three of the major programs. The 75th percentile length of stay is 12 to 15 years, compared with 4 to 10 years for the nonelderly cohorts.

Length of Stay by Race and Ethnicity 

Exhibits 9 through 11 perform the same analysis of shifts in survival functions across the 

rental assistance programs, but with these tables the comparison is across racial and ethnic 

groups. These tables look for significant shifts in the survival functions for various racially or ethnically defined groups of households across the three rental assistance programs. To keep 

the tables of a manageable scale, all households have been placed into one of four racial or 

ethnic groups based on the race and ethnicity of the head of household. The first three groups are households that are non-Hispanic, with separate groups for White, Black, and other non-Hispanic households. The fourth group contains all Hispanics households of any race.

Exhibit 9 lists the lengths of stay along the survival functions for HCV households. All households increased their median lengths of stay over all time periods, but the increases were all small, ranging from 0.5 to 1.5 years. In all cases, the length of stay at the median was greater for minority households compared to White households. Comparing the 2015 with the 2000 

cohort of exiters, median stays increased more for Black and Hispanic households than for 

White and other non-Hispanic households. Over the longer timespan, the 75th percentile 

length of stay increased substantially among exiters for all demographic groups. 

The lengths of stay used for this analysis provide limited information about factors that could lead to disparities in length of stay. Studies by DeLuca, Garboden, and Rosenblatt (2013) and Exhibit 9

Length of Stay of Households in the Housing Choice Voucher Program by 

Household Race and Ethnicity for Year of Exit 2000, 2005, 2010, and 2015

Year of 

White  

Black  

Other  

Percentile

Hispanic

Exit

Non-Hispanic

Non-Hispanic

Non-Hispanic

2000

25th

0.9

1.1

1.0

1.0

50th

1.8

2.5

2.5

2.3

75th

4.3

5.1

5.9

5.7

2005

25th

1.3

1.9

1.9

1.8

50th

2.8

3.5

3.5

3.4

75th

5.0

6.1

6.1

6.3

2010

25th

1.5

2.1

2.0

2.0

50th

3.3

4.9

4.4

4.9

75th

7.4

8.9

8.5

9.1

2015

25th

1.4

2.4

1.8

2.3

50th

3.9

5.6

5.2

5.8

75th

8.7

10.2

10.5

10.9

28 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Krysan and Bader (2007) that examined panels of voucher households in individual cities 

found that minority households confront greater challenges in their search for rental housing. 

These challenges may influence the perceived desirability of integrated neighborhoods. Dis-

crimination has been found to increase the difficulty of minority households with a voucher 

(Basolo and Nguyen, 2010).

Both overall increases in length of stay for all demographic groups as well as differences for minority HCV households could be driven by the shrinking availability of units with rents 

below the Fair Market Rent levels that govern the program. HUD’s Worst Case Housing Needs 

study for 2015 (Watson et al., 2017) and the Affordable Housing Needs study for 2005 (HUD 

PD&R, 2007) indicate that the availability of rental units below the Fair Market Rent levels fell by about 6 percent over the period from 2005 to 2015. The number of affordable units 

per 100 income-eligible households fell from 86.6 in 2005 to 81.6 in 2015. If reductions in 

affordable units were greater in minority-dominated rental markets, it could increase market 

pressure causing minority HCV households to stay longer in the assisted housing program.

Exhibit 10 finds very small differences between non-Hispanic White households and the three 

minority groups in terms of increases in median lengths of stay in public housing. This is 

very different than the increases in lengths of stay in the HCV program, in which all minority groups increased their lengths of stay relative to White households. 

Exhibit 11 extends the comparison to the Section 8 project-based housing program. The 

results are very similar to those for the public housing program. Changes in lengths of stay 

are generally small from one time period to the next, and the changes contain a mix of both 

positive changes (longer stays) and negative changes (shorter stays). 

Further research using household level survey data is needed to explore the many possible 

reasons behind the longer lengths of stay in assisted housing among minority households. 

Exhibit 10

Length of Stay of Households in the Public Housing Program by Household Race 

and Ethnicity for Year of Exit 2000, 2005, 2010, and 2015

Year of 

White  

Black  

Other  

Percentile

Hispanic

Exit

Non-Hispanic

Non-Hispanic

Non-Hispanic

2000

25th

0.6

1.0

0.8

0.8

50th

1.6

2.3

2.3

2.1

75th

4.4

5.6

5.9

5.4

2005

25th

0.8

1.3

1.1

1.5

50th

2.1

3.1

2.9

4.1

75th

5.5

7.4

7.5

10.3

2010

25th

0.9

1.4

1.4

1.5

50th

2.3

3.1

3.3

3.6

75th

5.7

7.1

8.1

9.0

2015

25th

0.9

1.4

1.2

1.7

50th

2.4

3.2

3.3

4.3

75th

6.0

7.1

8.0

10.3

Cityscape 29

McClure

Exhibit 11

Length of Stay of Households in the Section 8 Project-Based Housing Program by 

Household Race and Ethnicity for Year of Exit 2000, 2005, 2010, and 2015

Year of 

White  

Black  

Other  

Percentile

Hispanic

Exit

Non-Hispanic

Non-Hispanic

Non-Hispanic

2000

25th

1.0

1.2

1.2

1.1

50th

2.5

2.9

3.1

2.9

75th

7.1

6.5

7.2

7.1

2005

25th

0.5

1.3

–

1.4

50th

0.8

1.8

–

2.0

75th

3.7

5.1

–

10.5

2010

25th

1.0

1.2

1.0

1.2

50th

2.5

2.7

2.6

2.7

75th

6.4

5.7

7.3

6.5

2015

25th

1.1

1.4

1.2

1.4

50th

2.8

3.0

3.0

3.4

75th

7.0

6.3

7.6

7.5

Note: Insufficient data are available for non-Hispanic other race households in 2005.

Drivers of Length of Stay

Having looked at household the characteristics of race, ethnicity, age, disability and the 

presence of children, the analysis now turns to identification of what factors, if any, may drive the decision to exit assisted housing. These drivers may include household income level, its 

source, as well as market conditions, such as vacancy rates and rent levels. The analysis examines individually the relationship between these factors and lengths of stay in assisted housing to establish expectations on the scale and direction of the relationships. 

Length of Stay by Income Level and Source of Income

Differences in lengths of stay and changes in lengths of stay from one time period to the next could result from household factors other than race or ethnicity. Households differ by income levels and the sources of that income, both of which could influence decisions to remain in 

assisted housing or leave to enter the unsubsidized market. Exhibit 12 addresses this issue.

All households in the HCV, public housing, and Section 8 project-based programs benefit from 

very similar subsidy calculations. For public housing and project-based Section 8, each household generally pays about 30 percent of their adjusted gross income toward the gross rent on the rental unit in which the household lives.3 In the HCV program, the maximum subsidy provided is tied 

to a percentage of the local Fair Market Rent, with tenants allowed to pay more than 30 percent of their income for units renting above that level. The program does permit a household to pay up to 40 percent of income on housing if the household chooses to consume more housing.4

3 There are some exceptions from this rule. In some settings, households can be subject to flat rents that can alter the tenant’s required contribution as a percentage of income.

4 McClure (2005) found that, in 2002, 62 percent of HCV households paid 31 percent of income toward rent. Another 21 

percent spent more than 31 percent but not more than 40 percent of income. About 17 percent of HCV households spent more than 40 percent of income on rent, a housing cost hardship that the HCV program was designed to prevent.

30 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Exhibit 12

Correlation Between Length of Stay and Income by Source All Programs, 2015

Number of  

Correlation  

Income Source

Mean Value ($)

Significance

Households

Coefficient

Total income

 454,677 

 13,747 

0.124

**

Adjusted total income

 454,677 

 12,690 

0.126

**

Total wage income

 163,824 

 19,598 

0.225

**

Total welfare income

 109,544 

 4,016 

– 0.120

**

Length of stay

5.2 years

**   p  < .01.

The rental assistance programs pay the difference between the tenant’s contribution and 

the rent charged for the unit. All participating households are subject to similar eligibility rules limiting their participation in any of the programs. Thus, nearly all households in 

the programs have extremely low income, generally placing them below 30 percent of the 

Area Median Family Income of the metropolitan area or the nonmetropolitan county where 

they live. In 2015, the average household income for the assisted households was between 

$13,000 and $14,000. This means that the typical household receiving housing assistance 

lives below poverty. For that year, the U.S. Department of Health and Human Services (2017) 

set the poverty guidelines at about $12,000 for a family of one, $16,000 for a family of two, and $20,000 for a family of three.

The 2015 households that lived in assisted housing through the HCV, public housing, or 

Section 8 project-based programs had a mean income of only $13,456. The 2015 households 

that left assisted housing had a comparable mean income at $13,747. Thus, in terms of in-

come, the households that exited assisted housing were approximately the same as the larger 

population of households that remained in assisted housing. Higher or lower income does not 

seem to influence the decision to remain in assisted housing or to exit. About 36 percent of 

the households exiting assisted housing had income from employment. Not surprisingly, these 

employed households had higher incomes than their unemployed counterparts at $21,200 

with $19,598 of that income from wages. A smaller portion of the 2015 exiting households, 

24 percent, had income from public assistance. The income from public assistance is much 

lower for these households, at only $4,016, bringing them to a total income that averaged 

only $11,114. Interestingly, about one-third of these public assistance recipient households 

also had income from employment and that employment income was much larger, at 

$16,856, than the income from public assistance. 

Consistent with the research literature, it was expected that household income for households in assisted housing would be negatively correlated with length of stay. Those with the least 

income would have been more inclined to remain in assisted housing longer. Those with the 

greatest income would have a higher capacity to navigate the private, unsubsidized hous-

ing market and thus be more likely to end their time in assisted housing. Similarly, it was 

expected that source of income would matter. If a household had income from employment, 

Cityscape 31

McClure

it would seem more likely that this household could gain the extra income needed to enter 

the private market. Thus, income from wages was expected to be negatively correlated with 

length of stay. Finally, income from the various public assistance programs was expected to 

create the opposite effect. Income from assistance programs was expected to be positively as-

sociated with length of stay as greater public assistance usage would be associated with greater dependency on rental assistance to meet housing needs.

None of these expectations were supported (see exhibit 12). The correlations are all statistically significant at better than the .01 level. However, statistical significance is not policy-relevant significance. The scale of the dataset for this analysis is very large, with hundreds of thousands of households in the three major rental assistance programs in 2015. Almost 

any analysis generating statistics from a dataset this large will produce statistically significant results. While being statistically significant, the correlation coefficients are small with absolute values ranging from .12 to .22. Coefficients of this scale indicate that the variables explain only about 1.4 to 4.8 percent of the variation in the dependent variable—length of stay. At the very minimum, other factors must explain much more of the variation.

It remains something of an unanswered question why greater income among the eligible poor 

in assisted housing would be associated with longer stays in that housing rather than shorter stays, and greater public assistance usage would be associated with shorter stays. While perhaps counter-intuitive at first, it may be that families receiving income from job earnings are less resilient, for instance more at risk to loss of that income due to short-term emergencies (for example, becoming sick while in a low-wage job without paid sick leave), or having to move more frequently in order to find alternative employment (DeLuca, Garboden, and Rosenblatt, 2013).

Length of Stay by Rent Levels

Differences in length of stay and changes in length of stay could result from rent levels either charged to the households through the rental assistance program or from the market within 

which the household lives.

All of the households in the HCV, public housing, and Section 8 programs pay about 30 percent of income on housing. As a result, the burden of rent on the income of the eligible poor is roughly the same across all households. With this equivalent burden, it would be expected that higher or lower rents paid by the tenants would have no effect on length of stay other than the fact that the tenant rent reflects the incomes of the households. Exhibit 11 indicates that tenant income is weakly but positively associated with length of stay. Exhibit 13 suggests that the same holds true for tenant rent at almost exactly the same strength of correlation. In this regard, it can be speculated that tenant rent may not be related to length of stay except through the income effect.

Some households in public housing are subject to flat rents. The expectation was that flat 

rents, especially if set at a high level, might create pressure on households to leave public housing and enter the private market. This expectation suggests a negative and significant correlation with length of stay, but the opposite was found. There is a significant but small positive relationship between flat rent amount and length of stay for public housing households 

that are subject to flat rents.

32 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Exhibit 13

Correlation Between Length of Stay and Program Rents All Programs, 2015

Number of  

Correlation  

Income Source

Mean Value ($)

Significance

Households

Coefficient

Tenant rent amount

 429,215 

 327 

0.123

**

Flat rent amounta

 62,774 

 538 

0.130

**

Gross rent amountb

 300,495 

 854 

0.244

**

Length of stay

5.2 years

**   p  < .01.

a Public housing only.

b Housing Choice Voucher program and Section 8 only.

Gross rents are limited in the HCV and Section 8 project-based housing programs. They are 

limited through the Fair Market Rents published by HUD and, for the HCV program, the pay-

ment standards associated with the Fair Market Rents. These Fair Market Rents vary consider-

ably across the county reflecting the rents found in each individual marketplace. However, 

the great benefit of a household receiving housing assistance through the Section 8 program is that their housing cost burden is the generally the same, 30 percent of income, independent 

of the surrounding housing market conditions. Given the immunity from market pressures, it 

would be expected that there would be no correlation between gross rents and length of stay, 

but the relationship is positive, statistically significant, but small. Such a relationship could be a response to a market substitution effect. If the surrounding market has higher rents, they would be expected to discourage household from leaving assisted housing and moving into 

the private marketplace which would generate a positive correlation between gross rents and 

length of stay.

Length of Stay by Housing Market Conditions 

The softness of the surrounding rental housing market may influence the probability that a 

household stays in or exits out of assisted housing. If the household has many options, espe-

cially affordable options, then it seems likely that the pace of exiting assisted housing would increase. To determine if this is the case, the households that exited assisted housing in 2015 

were examined for correlations between their length of stay and measures of housing market 

conditions both in the immediate census tract and, if the household resided within a Core 

Based Statistical Area (CBSA), in the surrounding CBSA. These correlations test whether or 

not the length of stay is shorter for households living in census tracts with higher rental housing vacancy rates. A second test is whether or not the length of stay is shorter for households living in census tracts with lower gross rent levels. Exhibit 14 examines these issues.

At both the tract and the CBSA level, length of stay is positively associated with population. 

Locations with larger populations tend to correlate with households staying longer in assisted housing. The incidence of poverty is found to have an inverse effect; the greater the level of poverty in the surrounding tract and CBSA, the shorter the length of stay, suggesting that assisted households are more willing to leave assisted housing if they are subjected to living in a Cityscape 33

McClure

Exhibit 14

Correlation Between Length of Stay and Program Rents All Programs, 2015

Number of  

Correlation  

Population

Mean Value 

Significance

Households

Coefficient

Length of stay

430,768

5.94

Tract population

422,650

4,456

0.05

**

Tract percent poverty

422,640

27.68

– 0.03

**

Tract percent minority

422,640

46.39

0.15

**

Tract median gross rent

422,640

732

0.21

**

Tract rental vacancy rate

422,640

8.14

– 0.08

**

CBSA population

381,009

2,618,041

0.30

**

CBSA percent below poverty

381,009

16.18

– 0.11

**

CBSA percent minority

381,009

32.67

0.19

**

CBSA median gross rent

381,009

858

0.31

**

CBSA rental vacany rate

381,009

8.12

– 0.14

**

**   p  < .01.

CBSA = Core Based Statstical Area.

high-poverty setting. The incidence of the share of racial or ethnic minorities in the surrounding populations is found to have the opposite effect; the greater the incidence of minorities in the population, the longer the stay in assisted housing.

The price of housing and the availability of alternative rental housing in the marketplace are expected to influence the assisted household’s decision to stay in or leave assisted housing. 

Both price and availability effects were found to exist. Length of stay is positively associated with rent levels, again at both the tract and CBSA levels, as would be expected. If the assisted household confronts higher rents in the surrounding neighborhood and the surrounding 

metropolitan area, it is more likely that the household will remain in assisted housing. Length of stay is negatively associated with rental vacancy rates at both the tract and CBSA levels, as would be expected. If the assisted household confronts tighter rental housing markets offering fewer alternative units for rent, it is more likely that the household will remain in assisted housing. None of the correlations coefficients are compellingly strong. The strongest are the coefficients for median gross rents in the tracts and the CBSAs; these are .26 and .30 respectively. Coefficients at this level explain only 7 percent and 9 percent of the total variation length of stay, leaving the vast majority of the variation to be explained by other factors.

Conclusion 

HUD plays a very large role in helping extremely low-income renter households afford the cost of housing. HUD supports housing developments typically occupied by extremely low-income 

households through the public housing program as well as the Section 8 project-based housing 

program. HUD also funds the HCV program, which supports extremely low-income households 

that enter the private marketplace to rent a unit. Once a household begins to receive assistance through one of these three programs, how long will the household stay in the program?

34 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

This research finds that the typical household that left assisted housing recently stayed for about 6 years. Differences between types of households are stark; elderly households stayed 

longer at 9 years, and disabled households stayed for about 5 years, while nonelderly families with children stayed for about 4 years.

The length of stay has increased somewhat over time for all groups. The average length of 

stay in assisted housing grew for elderly households by 1.5 to 1.7 years from 2000 to 2015. 

Households with disabilities saw their average stay grow by 1.2 to 1.7 years during the same 

period. Nonelderly families with children experienced the smallest change; their average 

length of stay grew by 1.1 years.

Racial and ethnic minorities seem to stay for longer periods of time within the HCV program, 

but the influence of race and ethnicity is less within the public housing and the Section 8 

project-based housing programs. 

Among the eligible renter households, all of whom have very low incomes, those with more 

income seem to stay longer in assisted housing as do those with income from wages. Those 

households with income from public assistance seem to stay for shorter periods.

Market conditions influence length of stay in assisted housing in a manner suggesting substi-

tution effects. Where the rents on housing in the private marketplace are comparatively high 

or the availability of rental housing is comparatively low, households in assisted housing stay longer. Where alternative housing is the private market is expensive and scarce, households 

will stay longer in assisted housing.

The research finds that households that remain in assisted housing tend to follow a common 

pattern of stays. Once admitted into one of the assisted housing programs, over 90 percent of all assisted household remain in that housing through the first year. From 70 to 80 percent of households remain through the second year. The pace of leaving assisted housing continues 

but at a decreasing rate over time. About one-half of all assisted households leave by 4 to  

6 years after entry, and about 80 percent leave by years 9 to 11.

It is not surprising that the length of stay in assisted housing is increasing. Prior research suggests that this pattern has been seen in the recent past (Ambrose, 2005) as well as the more 

distant past (Hungerford, 1996).

The prior research, as well as the research presented here, cannot identify definitive reasons for the changes in lengths of stay in assisted housing. The research can only identify relationships that exist between lengths of stay and various forces that might influence a household’s decision to leave or remain in assisted housing. The prior research, as well as this research project, confirms that length of stay is related to the household’s age, presence of children as well as the ability of the household to find alternative housing in the private marketplace. 

While definitive causation is beyond the scope of this study, it is likely that fundamental 

market forces including increasing housing costs and inadequate incomes play the greatest 

role. The economic forces in the U.S. rental markets are moving in a manner that probably 

contributed to the longer stays in assisted housing. From 2000 to 2015, the United States 

Cityscape 35

McClure

saw median gross rent grow by 54 percent (U.S. Census Bureau, 2017). This growth in rents 

outpaced inflation by 16 percentage points as the Consumer Price Index grew by 38 percent 

during the same time period (U.S. Department of Labor, 2017). The rapid growth in rents 

contributes to the loss of affordable housing in the nation because the incomes of renters 

are not keeping up with inflation, much less with the growth of rents. From 2000 to 2015, 

median renter incomes grew by only 31 percent (U.S. Census Bureau, 2017). This trend 

has continued for a very long time. Despite the rise and fall of prices of homes for owner-

occupancy during the housing bubble, its collapse, and the recovery that followed, rents have been on a steady upward path, outpacing both inflation and renter income. As long as this 

pattern continues, it can be expected that the lengths of stay in assisted housing will continue to increase.

Acknowledgments

The author acknowledges the assistance of Meena Bavan and Barry Steffen at the U.S. Depart-

ment of Housing and Urban Development, Office of Policy Development and Research.

Author

Kirk McClure is a professor in the Urban Planning Program, University of Kansas in Lawrence, 

Kansas.

References

Ambrose, Brent W. 2005. “A Hazard Rate Analysis of Leavers and Stayers in Assisted Housing 

Programs,”  Cityscape 8 (2): 69–93.

Bahchieva, Raisa, and Amy Hosier. 2001. “Determinants of Tenure Duration in Public Hous-

ing: The Case of New York City,”  Journal of Housing Research 12 (2): 307–348.

Basolo, Victoria, and Mai Thi Nguyen. 2010. “Does Mobility Matter? The Neighborhood 

Conditions of Housing Voucher Holders by Race and Ethnicity,”  Housing Policy Debate 16 

(3–4): 297–324.

Climaco, Carissa G., Christopher N. Rodger, Judith D. Feins, and Ken Lam. 2008. “Portability 

Moves in the Housing Choice Voucher Program, 1998–2005,”  Cityscape 10 (1): 5–40.

Cortes, Alvaro, Ken Lam, and David Fein. 2008. “Household Life Cycle and Length of Stay in 

Housing Assistance Programs,”  Cityscape 10 (1): 117–156.

DeLuca, Stephanie, Phillip M.E. Garboden, and Peter A. Rosenblatt. 2013. “Segregating Shel-

ter: How Housing Policies Shape the Residential Locations of Low-Income Minority Families,” 

The ANNALS of the American Academy of Political and Social Science 647 (1): 268–299.

Freeman, Lance. 1998. “Interpreting the Dynamics of Public Housing: Cultural and Rational 

Choice Explanations,”  Housing Policy Debate 9 (2): 323–353.

36 Selected Outcomes of Housing Assistance

Length of Stay in Assisted Housing

Freeman, Lance. 2005. “Does Housing Assistance Lead to Dependency? Evidence From HUD 

Administrative Data,”  Cityscape 8 (2): 115–133.

Haley, Barbara A., and Robert W. Gray. 2008. Section 202 Supportive Housing for the Elderly: 

Program Status and Performance Measurement. U.S. Department of Housing and Urban 

Development, Office of Policy Development and Research.

Hungerford, Thomas L. 1996. “The Dynamics of Housing Assistance Spells,”  Journal of Urban Economics 39 (2): 193–208.

Krysan, Maria, and Michael Bader. 2007. “Perceiving the Metropolis: Seeing the City Through 

a Prism of Race,”  Social Forces 86 (2): 699–733.

Lubell, Jeffrey M., Mark Shroder, and Barry Steffen. 2003. “Work Participation and Length of 

Stay in HUD-Assisted Housing,”  Cityscape 6 (2): 207–223.

McClure, Kirk. 2005. “Rent Burden in the Housing Choice Voucher Program,”  Cityscape 8 (2): 5–20.

Olsen, Edgar O., Scott E. Davis, and Paul E. Carrillo. 2005. “Explaining Attrition in the Housing Voucher Program,”  Cityscape 8 (2): 95–113.

Olsen, Edgar O., Catherine A. Tyler, Jonathan W. King, and Paul E. Carrillo. 2005. “The Ef-

fects of Different Types of Housing Assistance on Earnings and Employment,”  Cityscape 8 (2): 163–187.

Pebley, Anne R., and Laura L. Rudkin. 1999. “Grandparents Caring for Grandchildren: What 

Do We Know?”  Journal of Family Issues 20 (2): 218–242.

Smith, Robin E., Susan J. Popkin, Taz George, and Jennifer Comey. 2015. “What Happens to 

Housing Assistance Leavers?”  Cityscape 17 (3): 161–192.

Thompson, Diane. 2007. “Evaluating Length of Stay in Assisted Housing Programs: A Meth-

odological Note,”  Cityscape 9 (1): 217–238.

U.S. Census Bureau. 2017. “American Community Survey 2015 1-Year; Census 2000.” 

https://factfinder.census.gov/faces/nav/jsf/pages/searchresults.xhtml?refresh=t.

U.S. Department of Health and Human Services. 2017. “2015 Poverty Guidelines.” https://

aspe.hhs.gov/2015-poverty-guidelines.

U.S. Department of Housing and Urban Development (HUD). 2017. “Moving to Work 

(MTW).” https://portal.hud.gov/hudportal/HUD?src=/program_offices/public_indian_hous-

ing/programs/ph/mtw.

U.S. Department of Housing and Urban Development, Office of Policy Development and 

Research (HUD PD&R). 2007.  Affordable Housing Needs 2005: Report to Congress. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and 

Research. huduser.gov/portal/publications/affhsg/affHsgNeeds.html.

Cityscape 37

McClure

U.S. Department of Labor. 2017. “Consumer Price Index.” https://www.bls.gov/cpi/.

Watson, Nicole Elsasser, Barry L. Steffen, Marge Martin, and David A. Vandenbroucke. 2017. 

Worst Case Housing Needs: 2017 Report to Congress. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. huduser.gov/portal/

publications/Worst-Case-Housing-Needs.html.

38 Selected Outcomes of Housing Assistance

Housing Cost Burden in the 

Housing Choice Voucher Program: 

The Impact of HUD Program Rules

Casey Dawkins

Jae Sik Jeon

University of Maryland

Abstract

U.S. renters’ housing cost burdens have reached historic highs, and these burdens fall 

most heavily on renters earning the lowest incomes. The Housing Choice Voucher 

(HCV) program, the nation’s largest tenant-based rental assistance program, is designed 

to alleviate high housing cost burdens for qualifying low-income households. In theory, 

voucher recipients should not spend more than 40 percent of their income on rent while 

participating in the program, yet research finds that many HCV program participants 

still experience housing cost burdens in excess of this threshold. 

This article examines recent trends and determinants of housing cost burdens for 

voucher recipients. We rely on cross-sectional and longitudinal data constructed from 

U.S. Department of Housing and Urban Development administrative files to character-

ize the prevalence of housing cost burden over the 2003-to-2015 period, explore longitu-

dinal trajectories of voucher recipients who initially leased a unit in 2003, and examine the marginal impact of various factors on the odds of an HCV household experiencing 

a housing cost burden in 2015. The findings suggest that certain provisions of the HCV 

program, particularly local payment standards and the restriction on housing cost bur-

den monitoring to those recently admitted or recently moved, play an important role in 

shaping housing cost burdens.

Introduction

U.S. renters’ housing cost burdens have reached historic highs. According to JCHS (2017), the number of renters spending more than 30 percent of their income on rent rose from 14.8 million to 21.3 million between 2001 and 2014, and the number of renters spending more than 50 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 39

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Dawkins and Jeon

percent of their income on rent rose from 7.5 million to a record high of 11.4 million. According to the U.S. Department of Housing and Urban Development’s (HUD’s)  2017 Worst Case Housing Needs report, these high housing cost burdens fall most heavily on renters earning the lowest incomes. Of those earning less than 30 percent of the Area Median Income (AMI), 65 percent are severely cost burdened, spending more than 50 percent of their income on rent (Watson et al., 2017: table A-1A). 

These high housing cost burdens reduce income available to meet other important household 

needs. For example, JCHS (2017) found that severely cost burdened low-income households 

spend 53 percent less on food, healthcare, and transportation than similar households without housing cost burdens.

The Housing Choice Voucher (HCV) program, the nation’s largest tenant-based rental assistance program, is designed to alleviate these high housing cost burdens for qualifying low-income 

households, while also expanding housing choices in a wider variety of neighborhoods that offer beneficial economic and social opportunities. Rather than limit households to government-owned or subsidized housing options, the HCV program expands the range of potential housing options to include all privately owned rental housing units that are managed by landlords willing to 

participate in the program. Also, because the HCV subsidy moves with the tenant, the program 

allows households to flexibly adjust their housing in response to changing household needs and preferences.

Eligibility for the HCV program is limited to low-income renters whose income is less than or equal to 50 percent of AMI, and local public housing agencies (PHAs) are required to allocate at least 75 percent of vouchers to those earning no more than 30 percent of the AMI. For those participating in the HCV program, HUD awards Housing Assistance Payments (HAPs) through 

local PHAs that cover the difference between 30 percent of a household’s adjusted gross income and a payment standard that reflects the cost of renting a unit that meets HUD’s Housing Quality Standards (McClure, 2005).

Although voucher recipients are required by HUD program rules to spend no more than 40 per-

cent of their income when entering the program or moving to a new unit, research finds that more than 16 percent of HCV participants experienced housing cost burdens in excess of 40 percent 

in 2002 (McClure, 2005). This article relies on cross-sectional and longitudinal data constructed from HUD administrative files to characterize the prevalence of housing cost burden over the 

2003-to-2015 period, explore longitudinal trajectories of voucher recipients who initially leased a unit in 2003, and examine the marginal impact of various factors on the odds of a HCV household experiencing a housing cost burden in 2015. These analyses are designed to address different 

aspects of the following research question: how do HUD program rules influence HCV housing 

cost burdens? The findings suggest that certain provisions of the HCV program play an important role in shaping housing cost burdens, particularly local payment standards and the restriction on housing cost burden monitoring to those recently admitted or recently moved.

The article is organized as follows. The next section examines relevant literature addressing recent trends in housing cost burden for U.S. households and HCV renters. Following the literature 

review, we discuss the data and methods, present the research findings, and conclude with a summary of the most important findings and their policy implications.

40 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

Background

Rising rents and falling renter incomes have contributed to a rental housing crisis in American cities. Rental prices peaked in 2007, steadily declined through 2007 and 2010, and have risen since (JCHS, 2017). DiPasquale and Murray (2017) found that between 1970 and 2010, incomes fell for renters in all but the highest income quintile. Between 2000 and 2010, incomes for renters in the lowest income quintile fluctuated, falling by 12 percent between 2000 and 2005, rising by 7 percent between 2005 and 2008, and falling again by 6 percent between 2008 and 2010. During the 

housing recession, changes in income and household composition played particularly important 

roles in shaping renters’ cost burden trajectories (Colburn and Allen, 2018).

We know little about how voucher-assisted renters fared during the recent housing market 

boom-bust cycle. The most recent detailed analysis of housing cost burden in the HCV program 

conducted prior to the housing market recession found that, although HCV-assisted households’ 

cost burdens were on the decline, 38 percent of HCV program participants in 2002 spent more 

than 31 percent of their income on housing costs, and 17 percent spent more than 40 percent of their income on housing costs (McClure, 2005). Williamson (2011) examined data from a sample 

of about 38,000 households residing in Florida’s Low-Income Housing Tax Credit (LIHTC) properties and found that about 35 percent of LIHTC residents receiving vouchers spent more than 30 

percent of household income on rent. Leopold et al. (2015) conducted a more recent analysis of HUD administrative data (from 2013) and found that 42 percent of voucher recipients earning 

extremely low incomes spend more than 30 percent of income on rent.

Because the HCV program is designed to reduce housing cost burdens for qualifying low-income 

households, why do so many voucher recipients incur high housing cost burdens? Households 

may choose to spend a larger proportion of their income on rent to obtain housing that is higher quality, larger, or located in more desirable neighborhoods. If higher cost burdens are associated with improved neighborhood quality, then a voucher recipient’s realization of these benefits may be a positive policy outcome. Even short-term gains in access to certain local public goods, such as high-quality schools, may yield long-term gains in a child’s future economic opportunities and well-being. However, if these initially higher cost burdens persist or rise over time as rents rise relative to household incomes, households may be unable to remain in their chosen housing unit to take advantage of beneficial neighborhood amenities and services.

Certain types of households may be more likely to incur higher housing cost burdens than others. 

McClure (2005) found that among all voucher recipients, housing cost burden is particularly high for single-parent female-headed households, larger families with children (who need larger units), and those with extremely low incomes. It is possible that low-income families with children are more strongly “tied” to location, due to reliance on local social networks for social support and financial assistance (Dawkins, 2006). Likewise, non-White households may experience housing 

market discrimination, limiting their ability to move to adjust housing costs. This latter explanation is consistent with McClure’s (2005) finding that households headed by African-Americans are more likely than other households to spend more than 40 percent of their income on rent.

Cityscape 41

Dawkins and Jeon

Supply-side conditions may also influence households’ ability to reduce housing cost burdens 

upon residential mobility. Pendall (2000) found that households receiving tenant-based rental assistance are more concentrated in distressed neighborhoods when those neighborhoods have a 

higher concentration of rental housing, despite such households’ tendency to avoid neighborhoods with very low rents. Another factor is landlords’ reluctance to participate in the HCV program. 

Unless states or localities have adopted legislation prohibiting the discrimination against those receiving tenant-based assistance, landlords’ participation in the HCV program is purely voluntary, and many landlords choose not to participate due to perceived administrative barriers or other considerations (Freeman, 2011).

HUD program rules and administrative procedures may also play a role in shaping HCV housing 

cost burdens. Local PHAs may prioritize admissions to households that are more or less likely to incur higher housing cost burdens over time. The Quality Housing and Work Responsibility 

Act of 19981 (QHWRA) expanded the discretionary authority of local PHAs and set threshold 

requirements for the incomes of those newly admitted to HUD programs. Since 1998, PHAs have 

been required to ensure that 75 percent of new voucher holders have incomes no greater than 30 

percent of the AMI and that all assisted households spend no more than 40 percent of income on housing costs at the time of lease up. Beyond these requirements, PHAs have substantial discretion to prioritize assistance to different types of households. Some PHAs place priority on housing those in greatest need, whereas others place emphasis on housing those most able to move to achieve greater self-sufficiency (Devine et al., 2000). Dawkins (2007) found that since the enactment of QHWRA, PHAs increasingly have been admitting smaller families headed by older adults and 

fewer extremely low-income female-headed households with children, thus signaling a trend away from the types of households identified by McClure (2005) who are most likely to incur high 

housing cost burdens.

For those households admitted to the HCV program, HUD policies determine the subsidy payment 

to renters and the range of housing units from which households may choose. Prior to selecting a unit, local PHAs determine the minimum tenant payment for an HCV household, or total tenant 

payment (TTP), which is equal to the greater of: (1) 30 percent of monthly adjusted income,  

(2) 10 percent of monthly gross income, (3) the welfare rent (in as-paid states only), or (4) a minimum rent payment as determined by the local PHA. Households may contribute more than 30 percent 

of their income toward rent but not more than 40 percent of monthly adjusted income upon entering the HCV program or signing a new lease (HUD, 2001). Thus, at the time of initial admission or lease up, HCV-assisted renters’ cost burdens are effectively constrained to be between 30 and 40 

percent of household income.

After a household’s minimum and maximum contribution is determined, the household selects 

a unit, and the rent subsidy contributed by the local PHA hinges crucially on whether the rent for the selected unit is higher or lower than the payment standard established by the local PHA (generally equal to 90 to 110 percent of the metropolitan area Fair Market Rent [FMR]). If gross rent (contract rent plus any utility allowance) is equal to or lower than the payment standard, then housing cost burden is equal to (TTP/income). In this case, since TTP is usually equal to 30 

1 “Quality Housing and Work Responsibility Act of 1998; Initial Guidance,” FR–4434–N–01.  Federal Register 64 (32) February 18, 1999. 

42 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

percent of a household’s adjusted monthly income, the cost burden would always be 30 percent 

or lower, regardless of the level of the payment standard. However, if gross rent is higher than the payment standard, cost burden is equal to ([gross rent – payment standard + TTP] / income). 

If rent is initially above the payment standard and increases over time, housing cost burden will always increase unless the payment standard is adjusted or the PHA determines that rent increases are unreasonable, based on a rent-reasonableness evaluation.

Existing research points to two unresolved questions pertaining to the influence of these HUD 

program rules on HCV cost burdens. First, to what extent are households’ higher housing cost 

burdens driven by the selection of housing units priced above the local payment standard? Mc-

Clure (2005) found that a large percentage of HCV households incurring cost burdens are served by PHAs that establish local payment standards below 90 percent of FMR, but his analysis does not identify whether households served by these PHAs are actually more likely to choose housing units above the payment standard. Second, do households newly admitted to the HCV program and 

those moving to a new unit incur lower cost burdens than other HCV program participants? Per 

HUD program rules, households are not required to comply with the 40-percent cap on housing 

cost burdens after their initial lease period. Although PHAs monitor annual adjustments to income and rents after a household signs a new lease, PHAs have limited ability to influence the rent charged by local landlords beyond the “rent reasonableness” evaluation. In hot housing markets, high percentage increases in rent may be consistent with prevailing market rents. HUD does 

not specify the methodology that local PHAs must follow when conducting rent-reasonableness 

evaluations, and some local PHAs do not conduct rent reasonableness evaluations on a regular 

basis (Turnham and Khadduri, 2001; Varady, Wang, and Mittal, 2007). Furthermore, a local PHA’s ability to increase rent subsidies is budget-constrained, and any increase in rent subsidies reduces the number of additional vouchers that can be awarded. Although HUD adjusts tenant payments 

in response to income changes upon annual reexaminations, these adjustments may not keep pace with changes in income if income streams vary from month to month.

In the analyses that follow, we address this gap in existing research by addressing the following research question: how do HUD program rules influence HCV housing cost burdens? We pay particular attention to the two features of HUD program rules discussed previously. First, we examine how renting units above the payment standard contributes to housing cost burden. Second, we 

examine whether those newly admitted to the HCV program or those recently moving to a new 

unit are less likely to incur cost burdens in excess of 40 percent of income.

Data and Methods

This research relies on administrative data from HUD’s Public and Indian Housing Information 

Center system to examine trends in HCV housing cost burden between 2003 and 2015. The data 

are assembled from tenant-level databases collected from the HUD-50058 Family Report form 

completed by local PHAs.

We rely on two primary databases for the analyses. The first database is a set of cross-sectional household-level files for each year between 2003 and 2015. These files (one for each year) 

include the last household record available for each household in each year for all households that Cityscape 43

Dawkins and Jeon

successfully leased up during or prior to the year in question. Households with zero income, those that receive project-based vouchers, and those that receive vouchers from Moving to Work (MTW) PHAs are excluded from the analyses. Using these databases, we examine trends in housing cost burden over time for all HCV-assisted households. For those participating in the HCV program 

in the most recent period (2015), we estimate logistic regression models to examine the marginal impact of various household, housing unit, and geographic characteristics on the odds of a HCV 

household experiencing a housing cost burden.

We also construct a longitudinal file of those who leased up in 2003. We follow these households over time, appending observations on rental spells for each year after initial lease up until either 2015 or the year in which the household exits from the HCV program. Using this database, we 

examine the duration of housing cost burden, emphasizing factors associated with different housing cost burden trajectories.

In all analyses, we define housing cost burden as the ratio of the family’s total contribution to housing payments (gross rent minus the household’s HAP) to the household’s total annual adjusted gross income. Gross rent is equal to the contract rent plus a utility allowance. HAP is defined as the lower of gross rent or the payment standard minus the TTP. We use the terms  rent burden and housing cost burden interchangeably throughout the article to reflect the percentage of income spent on housing costs. We categorize housing cost burdens into the following cost burden categories: no cost burden (spending 30 percent or less of income on housing costs), moderate cost burden (spending 31 to 40 percent of income on housing costs), high cost burden (spending 41 to 50 

percent of income on housing costs), and severe cost burden (spending 51 percent of income or higher on housing costs). The so-called “30 percent rule” is a standard threshold level of housing cost burden that can be traced to the Brooke Amendment to the 1968 Housing and Community 

Development Act.2 Because voucher recipients are required to spend no more than 40 percent of income on housing upon lease up, we use the 40-percent threshold to define the second housing cost burden threshold. The 50-percent threshold corresponds to HUD’s definition of severe cost burden in its Worst Case Housing Needs reports. 

Findings

The discussion of research findings is organized according to three different types of analyses. The first section below examines information from the cross-sectional household-level files for each year between 2003 and 2015. The second section examines information from the longitudinal 

database of those households that leased up in 2003 to examine housing cost burden trajectories. 

The third section relies on information from the cross-sectional file of households that leased up during or prior to 2015 to estimate logistic regression models that explain the marginal impact of various household, housing unit, and geographic characteristics on the odds of an HCV household experiencing a housing cost burden. Each of these analyses addresses different aspects of the following overarching research question: how do HUD program rules influence HCV housing cost 

burdens? 

2 Housing and Community Development Act of 1969, Section 213(a). Pub. L. 91–152. December 24, 1969.

44 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

Prevalence of HCV Housing Cost Burden Over Time

Exhibit 1 displays, for each year, the total number of HCV households spending 30 percent or 

less of their income on rent, 31 percent or more of income on rent, between 31 and 40 percent of income on rent, between 41 and 50 percent of income on rent, 51 percent or more of income 

on rent, and the total number of households. Exhibit 2 displays the percentage of households 

falling into each of these housing cost burden categories by year (excluding the “any cost burden” 

category). The total sample sizes (after excluding those with project-based vouchers, those with zero income, and those served by MTW PHAs) range between 1.6 and 1.7 million households, 

depending on the year.

Exhibit 1

HCV Households by Extent of Housing Cost Burden, 2003–2015

No Cost  

Any Cost   Moderate Cost 

High Cost 

Severe Cost 

Burden  

Burden  

Burden  

Burden  

Burden  

Total

(30% or Less) (31% or More)

(31–40%)

(41–50%)

(51% or More)

2003

 1,095,683 

 517,665 

 360,794 

 78,500 

 78,371 

 1,613,348 

2004

 1,011,929 

 590,988 

 400,695 

 95,364 

 94,929 

 1,602,917 

2005

 904,844 

 684,472 

 447,381 

 117,583 

 119,508 

 1,589,316 

2006

 909,791 

 765,417 

 496,044 

 134,108 

 135,265 

 1,675,208 

2007

 999,470 

 710,429 

 473,754 

 116,054 

 120,621 

 1,709,899 

2008

 1,095,772 

 644,484 

 435,053 

 101,988 

 107,443 

 1,740,256 

2009

 1,066,702 

 666,378 

 440,855 

 110,858 

 114,665 

 1,733,080 

2010

 1,070,330 

 671,092 

 444,345 

 109,283 

 117,464 

 1,741,422 

2011

 1,033,761 

 716,661 

 460,935 

 123,229 

 132,497 

 1,750,422 

2012

 986,171 

 760,437 

 490,866 

 130,585 

 138,986 

 1,746,608 

2013

 952,359 

 755,471 

 472,499 

 136,247 

 146,725 

 1,707,830 

2014

 893,875 

 811,315 

 499,697 

 149,189 

 162,429 

 1,705,190 

2015

 941,798 

 786,958 

 505,710 

 136,846 

 144,402 

 1,728,756 

HCV = Housing Choice Voucher.

Exhibit 2

Proportion of HCV Households With Specified Housing Cost Burden, 2003–2015

HCV = Housing Choice Voucher.

Cityscape 45

Dawkins and Jeon

Although the total number of HCV-assisted households has remained essentially flat over the 2003-to-2015 period, the total number of cost-burdened HCV households has increased by 52 percent 

from 517,665 in 2003 to 786,958 in 2015 (the number of cost-burdened households reached a high of 811,315 in 2014 but fell slightly in 2015). As a share of total HCV households in each year, those experiencing any level of cost burden increased by 13 percentage points, those experiencing moderate cost burdens increased by 7 percentage points, those experiencing high cost burdens increased by 3 percentage points, and those experiencing severe cost burdens increased by 4 percentage points.

The year-to-year change in housing cost burden roughly corresponds to the recent housing market boom-bust cycle. The share of HCV households experiencing housing cost burdens rose to a peak of 46 percent of households in 2006, followed by a steady decline during the housing bust. As the housing market began to recover, the share of cost-burdened HCV households rose again to 

a higher peak of 48 percent in 2014. These trends suggest that the increase in rental affordability during the initial years of the housing recession temporarily reduced housing cost burdens, but by 2015, housing cost burdens had risen to prerecession levels.

To provide additional context for the drop in housing cost burden during the housing bust, exhibit 3 

compares the percentage of HCV renters that experienced a cost burden or a severe cost burden with the percentage of all U.S. renters experiencing the same levels of cost burden over the 2003-to-2015 period as reported in the  2017 Worst Case Housing Needs Report. As this figure illustrates, the gap in housing cost burden between HCV renters and all U.S. renters widened between 2005 

and 2013. One possible explanation for the cost burden gap between HCV households and all 

U.S. renters is that the trends in exhibits 2 and 3 were driven by newly admitted HCV households that were more likely to incur lower cost burdens. Although there was an influx of newly admitted voucher recipients during the housing bust, new admissions never comprised more than about 10 

percent of all HCV program participants in any given year.

Exhibit 3

Housing Cost Burdens for HCV Households and U.S. Renters, 2003–2015

HCV = Housing Choice Voucher.

46 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

Another possible explanation for the trends in exhibits 2 and 3 is that vouchers provided a stable source of rent payment for voucher recipients that insulated these households from job loss or income shocks. This explains why housing cost burdens did not rise during the housing bust, but it does not fully explain why HCV cost burdens declined. For housing cost burdens to fall for HCV 

renters, rents must also have declined. In response to job losses and rising unemployment rates especially among young workers, vacancy rates rose, and real rents fell in most housing markets during the housing bust (DiPasquale, 2011). Since moving is generally less costly for renters than for owners, renters could more easily respond to changes in employment and housing market conditions, leading to downward pressure on rents in areas hit hardest by the housing bust. Collinson (2011) found that between 2007 and 2010, rents fell by 6 to 8 percentage points in the housing markets that were hit hardest by foreclosures.

How does housing cost burden vary with income? HUD annually establishes income limits by 

family size for its assisted housing programs that are based on AMI of the surrounding FMR area, which is typically coincident with the U.S. Census Bureau-defined metropolitan area for that year. 

As of 2015, the majority of HCV households in the sample (61 percent) had incomes between 11 

and 30 percent of AMI, 11 percent had incomes less than 10 percent of AMI, and the remainder 

had incomes more than 30 percent of AMI. Among those with incomes less than 10 percent of 

AMI, 61 percent experienced a housing cost burden and 36 percent experienced a severe housing cost burden in 2015. Also, among those earning $5,000 or less, 65 percent experienced a housing cost burden, and 40 percent experienced a severe housing cost burden.

Exhibit 4 displays the percentage of households within different income ranges that experienced severe housing cost burdens (51 percent or more) in each year. The gap in severe housing cost burdens between those with incomes of 10 percent or less of AMI and those with incomes of more Exhibit 4

Prevalence of Severe Housing Cost Burden by Income as a Percent of AMI, 2003–2015

40 

35 

, .............

,.... 

--_,, , ...... 

.... 

... ...

,

--

V, 

....... ...

---

--

""C  30 

0 

, 

, 

,_ .. --------

.c 

, 

Q) 

V, 

, 

::J  25 

0 

I 

,, 

> 

,.... , 

u 20 

I 

0 

+,J  15 

Q) 

Q) 

c..  10 

-------

--------- -- -

5 

-------

-----------

0 

2003 

2004 

2005 

2006 

2007 

2008 

2009 

2010 

2011 

2012 

2013 

2014 

2015 

Year 

Income 10% or less of AMI 

Income 11-30% of AMI 

--- Income 31-50% of AMI 

• • • • • •  Income 51-80% of AMI 

- • Income more than 80% of AMI 

AMI = Area Median Income. HCV = Housing Choice Voucher.

Cityscape 47

Dawkins and Jeon

than 80 percent of AMI has grown over time from 35 percentage points in 2003 to 47 percentage points in 2015. It is notable that in all years but 2003, more than 20 percent of those with incomes of 10 percent or less of AMI experienced cost burdens of 51 percent or higher, a level of cost burden that is inconsistent with HCV program goals.

The discussion in the previous section suggests that those newly admitted to the HCV program 

may incur lower housing cost burdens, due to HUD’s 40-percent cap on housing cost burdens. 

Exhibit 5 displays the percentage of households that incur severe housing cost burdens by the number of years of program participation. After 2005, those newly admitted to the HCV program experienced consistently lower housing cost burdens than those who had been in the program for a longer period of time, and the gap between those newly admitted and those with longer HCV 

program durations generally grew over time.

Exhibit 5

Prevalence of Severe Housing Cost Burden by Length of Program Participation, 

2003–2015

HCV = Housing Choice Voucher.

HCV Housing Cost Burden Trajectories

For how long do voucher recipients remain in a cost-burdened state, and how does residential 

mobility and the selection of a unit above the payment standard contribute to the duration of cost burden? To address this question, we rely on information from a longitudinal file of those who leased up in 2003. We follow these households over time, appending observations on rental spells for each year after initial lease up until either 2015 or the year in which the household exits from the HCV program. Using this database, we examine several descriptive statistics related to housing cost burden trajectories, including—

•  Average total duration in the HCV program (years).

•  Average duration in cost burden (years), separately by level of cost burden experienced.

•  Percentage of households cost burdened at the beginning of the voucher contract by severity of cost burden.

•  Percentage of households cost burdened at least 1 year by severity of cost burden.

48 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

•  Percentage of households cost burdened consecutively throughout the voucher contract, by 

severity of cost burden, separately for those who never moved and those who moved at least 

once since initial lease up.

•  For those who moved at least once, the average number of moves and percentage of households that increased (decreased) cost burden upon mobility.

Exhibit 6 reports each of these longitudinal statistics for those that initially leased a unit in 2003. 

The statistics are further disaggregated by the household’s initial rent payment vis-à-vis the payment standard. On average, households that initially leased a unit in 2003 participated in the HCV 

program for about 6 years. Approximately 2 of those years were spent in a cost-burdened state, but for those renting a unit priced above the payment standard (73,045 or about 36 percent of all households), nearly 4 years are spent in a cost-burdened state. 

Exhibit 6

Longitudinal Statistics for HCV Households That Leased a Unit in 2003

Rent ≤  

Rent >  

All HCV  

Payment  

Payment  

Households

Standard

Standard

All HCV households

204,213

131,168

73,045

Average total duration (years)

6.3

6.4

6.1

Average duration in cost burden (years)

2.4

1.7

3.8

Average duration in high cost burden (years)

0.8

0.6

1.1

Average duration in severe cost burden (years)

0.4

0.4

0.5

Cost burdened at beginning of the voucher contract (%)

37.3

5.3

94.7

High cost burdened at beginning of the voucher contract (%)

5.7

3.5

9.6

Severely cost burdened at beginning of the voucher contract 

3.1

2.6

4.1

(%)

Cost burdened at least 1 year (%)

64.3

46.1

97.0

High cost burdened at least 1 year (%)

30.3

24.0

41.7

Severely cost burdened at least 1 year (%)

18.0

15.4

22.6

Cost burdened throughout the voucher contract (%)

16.4

1.7

42.7

High cost burdened throughout the voucher contract (%)

1.8

0.9

3.5

Severely cost burdened throughout the voucher contract (%)

0.9

0.6

1.4

HCV households that never moved

165,121

105,315

59,806

Cost burdened throughout the voucher contract (%)

19.1

2.0

49.1

High cost burdened throughout the voucher contract (%)

2.2

1.1

4.2

Severely cost burdened throughout the voucher contract (%)

1.1

0.7

1.7

All HCV households that moved since initial lease up

39,092

25,853

13,239

Average number of moves

1.4

1.4

1.4

Reduced cost burden upon mobility (%)

42.5

37.7

51.8

Increased cost burden upon mobility (%)

45.5

47.5

41.5

Cost burdened throughout the voucher contract (%)

4.9

0.6

13.5

High cost burdened throughout the voucher contract (%)

0.3

0.2

0.4

Severely cost burdened throughout the voucher contract (%)

0.1

0.1

0.1

HCV = Housing Choice Voucher.

Cityscape 49

Dawkins and Jeon

A substantial share (37 percent) of HCV households experienced some level of cost burden, 

and about 3 percent experienced severe cost burdens. Since program rules preclude households 

from spending more than 40 percent of their income on rent at the time of lease up, this latter percentage suggests that some PHAs are not adequately monitoring and enforcing compliance with the HUD program rules. Among those renting a unit above the payment standard, nearly all (95 

percent) are cost burdened upon initial lease up.

For all HCV households, 64 percent experienced a cost burden of at least 1 year or longer, and this percentage is particularly high among those initially renting a unit priced above the payment standard. Of households, 18 percent experienced a severe housing cost burden for at least 1 year. 

Because even a single year of high housing cost burden may spur housing instability, due to the financial pressures of moving and finding a lower cost unit, this high level of exposure to temporary cost burdens is particularly troublesome.

Our primary metric of the impact of mobility on cost-burden trajectories is the percentage of households that are cost burdened consecutively throughout their participation in the HCV 

program. For all households, approximately 16 percent of households fell into this category, but among those renting a unit priced above the payment standard, 43 percent were consecutively 

cost burdened. Mobility plays an important role in reducing exposure to consecutive cost burdens, as revealed by the statistics shown in the bottom of exhibit 6. Whereas 19 percent of households that never moved were consecutively cost burdened, less than 5 percent of those who moved since lease up were consecutively cost burdened. Among those living in a unit priced above the payment standard, the difference between movers and stayers is even larger (14 versus 49 percent). Given that only about 20 percent of households entering the program in 2003 subsequently moved to 

another unit, the long duration of cost burden among stayers is particularly troublesome. Although a slightly higher percentage of those who moved increased rather than reduced housing cost 

burdens upon mobility (46 versus 43 percent), those initially leasing a unit above the payment standard were more likely to reduce their cost burden upon mobility, whereas those initially leasing a unit below the payment standard were more likely to increase their cost burden upon mobility.

Determinants of HCV Housing Cost Burdens

In this section, we examine the marginal contribution of various household, housing unit, and geographic factors on the odds of having any cost burden, a high cost burden, or a severe cost burden in 2015 using logistic regression models. In contrast to the longitudinal analysis described in the previous section which described longitudinal trends for those who entered the HCV program and leased a unit in 2003, the analysis described here reveals the determinants of housing cost burdens for those participating in the HCV program in the most recent period for which data are available (2015).

In the logistic regression models, the dependent variable is equal to 1 if a household has a cost burden of a given level or higher and 0 otherwise. Independent variables include various factors shown in the literature to be correlated with high housing costs, including family composition, disability, age of household head, race and ethnicity of household head, income (as a percentage of AMI), and source of income. We also control for various characteristics of the housing unit selected by the voucher recipient, including number of bedrooms, housing type, and housing age. Finally, 50 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

we include various controls to capture geographic variation in regional and neighborhood (census tract) housing market conditions. The estimated regression models can be roughly interpreted as Engel curves, named for the statistician Ernst Engel, who relied on Belgian survey data to demonstrate how household food expenditure data varied with income. Regression models based on this approach typically explain expenditure shares on a given commodity as a function of income and other demographic determinants of demand (Leser, 1963; Working, 1943).

The key policy variables are an indicator of whether the household has selected a unit with a rent above the payment standard and an indicator of whether the household falls into the category of those who are monitored by HUD for compliance with the 40-percent-of-income cap. Those falling into the 40-percent cap compliance category include all recently admitted to the HCV program 

and all recent movers. We should note that recent mobility may influence cost burden for reasons independently of program compliance requirements. Due to the presence of mobility costs, recent movers are more likely to be in equilibrium with respect to their most preferred housing bundle compared to nonmovers (Weinberg, Friedman, and Mayo, 1981). This implies that the estimates 

for the HUD program compliance indicator will reflect both a pure programmatic impact and any associated impacts due to recent mobility. Although our estimates should be interpreted with this qualification in mind, no reason exists to expect that the impacts of recent mobility should exhibit a threshold at the level of 40 percent of income. For this reason, we place emphasis on differences in the impact of recent mobility and recent program admission at the 40-percent cost-burden level relative to other thresholds.

We also control separately for length of HCV program participation, because program participation duration may reflect other unobserved household-level determinants of housing cost burden. For example, we might expect those with shorter HCV program durations to be more likely to rely 

on HCV assistance to address temporary conditions of housing instability, perhaps induced by 

short-term job loss or changes in family status, compared to long-term program participants who may be more heavily reliant on housing and other forms of governmental assistance. The models are estimated for the most recent period for which data are available (2015). Exhibit 7 provides a detailed description of the variable definitions and descriptive statistics for all variables used in the model, and exhibit 8 provides estimates of the logistic regression coefficients for the base models for the three different cost-burden thresholds. 

The results displayed in exhibit 8 suggest that the determinants of housing cost burden differ by level of cost burden. This difference is likely due to the impact of differential programmatic requirements at different cost-burden thresholds. Since most HCV households are required to spend at least 30 percent of adjusted monthly income on rent, few will fall below this threshold, and those that do are likely to exhibit unique characteristics that exempt these households from HUD’s minimum tenant payment requirements. For example, households with zero household income 

often receive special considerations in the calculation of cost burden, and HUD grants exceptions to minimum contribution requirements in cases of special hardship (HUD, 2001). Likewise, HUD’s 40-percent-of-income cap places an upper limit on housing cost burdens for most households. 

The only possible explanations for incurring cost burdens above this threshold are: (1) not having recently moved or recently entered the program, (2) exceptions to the 40-percent-of-income cap granted by local PHAs, or (3) inadequate program monitoring on the part of local PHAs.

Cityscape 51

Dawkins and Jeon

Exhibit 7

Descriptive Statistics and Variable Definitions for Logistic Regression Models

Variable

Definition

Mean Std Dev

Dependent variable

Any cost burden

1 = A household experiences any cost burden (31% or 

0.46

0.50

more)

High cost burden

1 = A household experiences a high cost burden (41% 

0.16

0.37

or more)

Severe cost burden

1 = A household experiences a severe cost burden (51% 

0.08

0.28

or more)

Independent variables

Rent and income

Rent above the  

1 = Gross rent above the payment standard

0.44

0.50

payment standard

Income as % of AMI

1 = Total annual income 10% of less than AMI

2.19

0.78

2 = Total annual income 11–30% of AMI

3 = Total annual income 31–50% of AMI

4 = Total annual income 51–80% of AMI

5 = Total annual income more than 80% of AMI

Household characteristics

Newly admitted or  

1 = New to the program or moved into a new unit

0.10

0.30

moved to a new unit

Length of participation

Length of participation (years)

7.94

6.68

Household size

Number of household members

2.39

1.58

Children

1 = A household has at least one child

0.47

0.50

Female

1 = A household head is female

0.81

0.39

Elderly

1 = A household head is elderly

0.22

0.41

Disabled

1 = A household head is disabled

0.27

0.44

Primarily wage

1 = Primary source of income is wage

0.31

0.46

Non-White

1 = A household head is non-White

0.51

0.50

Hispanic

1 = A household head is Hispanic

0.17

0.37

Housing unit type

Bedroom

Number of bedrooms

2.14

0.96

Single-family

1 = Single-family (detached/attached) home

0.59

0.49

Building age

Age of housing unit (years)

46.75

33.34

Geography

Midwest

1 = Midwest (Census region)

0.20

0.40

South

1 = South (Census region)

0.35

0.48

West

1 = West (Census region)

0.21

0.41

Metropolitan—central 

1 = Central city, metropolitan area

0.48

0.50

city

Metropolitan—suburb

1 = Suburb, metropolitan area

0.39

0.49

Micropolitan

1 = Micropolitan area

0.08

0.27

Neighborhood characteristics

Median rent

Census-tract level median gross rent

896.79 295.55

Median rent above  

1 = A census tract’s median gross rent is above the FMR

0.30

0.46

the FMR

Vacancy rate

Census-tract level vacancy rate

11.88

8.08

Poverty rate

Census-tract level poverty rate

21.77

13.09

% Minority population

Census-tract level percentage of minority population

53.24

32.34

% Housing voucher 

Census-tract level percentage of housing voucher 

12.71

10.30

recipients

recipients out of all renters

AMI = Area Median Income. FMR = Fair Market Rent. Std Dev = standard deviation.

52 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

Exhibit 8

Factors Associated With the Odds of a Housing Cost Burden, 2015 (Base Model)

Any Cost 

High Cost 

Severe Cost 

Category

Explanatory Variable

Burden

Burden

Burden

Rent and income

Rent above the payment standard

7.685***

2.925***

2.328***

Income as % of AMI

– 1.538***

– 1.573***

– 2.278***

Household 

characteristics

Newly admitted or moved to a new 

0.119***

– 1.547***

– 1.140***

unit

Length of participation

– 0.012***

0.018***

0.020***

Household size

– 0.129***

– 0.301***

– 0.347***

Children

– 0.129***

– 0.159***

– 0.150***

Female

– 0.037***

0.078***

0.078***

Elderly

– 1.267***

– 0.699***

– 0.699***

Disabled

– 1.420***

– 0.730***

– 0.662***

Primarily wage

– 0.876***

– 0.446***

– 0.457***

Non-White

0.093***

0.030***

0.052***

Hispanic

– 0.056***

0.075***

0.101***

Housing unit type

Bedroom

0.184***

0.548***

0.678***

Single-family

– 0.039***

0.102***

0.109***

Building age

– 0.000***

– 0.000***

0.000

Geography

Midwest

– 0.268***

0.004

– 0.022*

South

– 0.130***

0.171***

0.177***

West

– 0.364***

0.051***

0.084***

Metropolitan—central city

0.210***

0.028*

0.083***

Metropolitan—suburb

0.004

– 0.136***

– 0.119***

Micropolitan

0.126***

– 0.060***

– 0.035

Neighborhood 

characteristics

Median rent

0.000

0.000***

0.001***

Median rent above the FMR

0.179***

0.099***

0.100***

Vacancy rate

0.006***

– 0.002***

– 0.002***

Poverty rate

0.001***

– 0.000

– 0.000

% Minority population

0.001***

0.002***

0.003***

% Housing voucher recipients

0.008***

– 0.001***

– 0.001**

Constant

0.472***

– 1.002***

– 0.813***

Number of 

1,696,116

1,696,116

1,696,116

observations

Wald chi-square

407,228.33***

316,297.75***

194,781.55***

Pseudo  R2

0.775

0.336

0.346

***  p  < 0.01. **  p  < 0.05. *  p  < 0.1.

AMI = Area Median Income. FMR = Fair Market Rent.

Note: Northeast and Rural are omitted.

Cityscape 53

Dawkins and Jeon

In the models explaining the probability of a high or severe cost burden, the households most likely to suffer housing cost burdens include those without children, those headed by females, those with nonelderly household heads, those without disabled household heads, those headed 

by non-White household heads, those headed by Hispanic household heads, those reliant on 

government sources of income, and those earning lower incomes. The lower incidence of housing cost burden among households headed by a person with a disability is somewhat surprising, given evidence that such households are more likely to incur high housing cost burdens (Souza et al., 2011). One possible explanation is that rents are more stable in housing units specifically targeted to persons with a disability, particularly if other HUD place-based programs subsidize rents. This finding deserves further exploration.

The higher incidence of housing cost burden among female-headed and non-White households 

is consistent with McClure (2005), but in contrast with McClure we find that households without children are more likely to exhibit housing cost burdens than households with children. In a 

separate longitudinal analysis not reported here, we found that the prevalence of severe housing cost burden by household size has varied over time. In 2006, at the height of the housing boom, 8.4 percent of large households (five or more persons) experienced severe housing cost burdens, and 5.0 percent of single-person households experienced severe housing cost burdens. This gap declined over time, and in 2015, the year under consideration in the logistic regression analysis, single-person households were slightly more likely than large households to incur severe housing cost burdens. This finding deserves additional exploration to determine the influence of changing household size and composition on housing cost burdens. It is possible that household size is not a determinant of but a response to high housing cost burden, as couples forgo the decision to have children if doing so is likely to impose a financial burden (Colburn and Allen, 2018).

Those living in larger single-family units are more likely to experience a high or severe housing cost burden. The effect of the number of bedrooms on the odds of a severe housing cost burden is more than three times larger than the effect of number of bedrooms on the odds of any housing cost burden. The influence of housing unit age varies by severity of cost burden.

Exhibit 8 also provides evidence of significant geographic variation in the determinants of housing cost burdens. Those living in the South are more likely to experience a housing cost burden than those living in other census regions. Regarding the intra-metropolitan location of households, those living in central cities experience the highest cost burdens, and those living in suburban areas experience the lowest cost burdens.

Various census tract-level characteristics shape housing cost burdens. Those living in census tracts with higher median rents, median rents above the FMR, and a higher percentage of minority residents exhibit higher housing cost burdens. After controlling for other census tract-level variables, census tract poverty rates do not influence high or severe cost burdens at statistically significant levels, although high poverty rates are associated with a lower probability of having any cost burden. The finding that the probability of high and severe cost burdens is negatively associated with vacancy rates, and the percentage of census renters receiving vouchers is consistent with the prevailing wisdom among housing policy practitioners that the HCV program is most effective in loose housing markets with high vacancy rates and a healthy supply of properties managed by landlords that accept vouchers.

54 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

The key policy variables of interest—rent above the payment standard and newly admitted or 

moved to a new unit—are both consistent with expectations. Those households initially paying 

a rent above the payment standard are much more likely to exhibit housing cost burdens at any level. Furthermore, those recently admitted to the HCV program or that have recently moved to a new unit are less likely to exhibit high and severe housing cost burdens. The differences in the sign of the new admission-recent mover coefficient between any cost burden and high or severe cost burdens suggest that HUD income caps effectively constrain households to spend within 30 

and 40 percent of their income on housing when signing a new lease or entering the program. 

If compliance with this provision is taken as a metric of success, then the HCV program appears to be operating as expected. It is also interesting to note that after controlling for the categorical distinction between recent movers or recent admissions and other households, length of program participation is positively associated with high and severe housing cost burdens. This finding combined with the large reduction in housing cost burden associated with the receipt of income from wages suggests that households not able to work and confined to fixed incomes from different sources of government assistance are more likely to be cost burdened.

In addition to its direct impact on the probability of a housing cost burden, the categorical programmatic distinction between new movers or new admissions and other voucher recipients may also 

interact with other household, housing unit, and geographic variables to influence housing cost burden. To test this conjecture, exhibit 9 displays estimates from models stratified into new movers or new admissions versus all others. A key finding from this table is that renting above the payment standard has a different impact for new admissions or new movers versus others. In the model 

explaining the probability of a high housing cost burden (41 percent or higher), renting above the payment standard has little impact on new admissions or new movers, because these households 

are not allowed to take on this level of housing cost burden. Likewise, renting a unit above the payment standard actually reduces the probability of these households incurring a severe housing cost burden. For households not constrained by the HUD 40-percent-of-income requirement, the impact of renting a unit above the payment standard is positive and much larger in magnitude.

Exhibit 9 also suggests that the impact of household income is much smaller for those in the 

“other” category, particularly for cost burdens above the 40-percent threshold, because income changes for these households may be insufficient to offset the impact of rising rents in housing units already chosen. Several household and housing unit variables also have impacts that differ between the two samples. The magnitude of the impact of number of bedrooms on housing cost 

burden is much higher for those not recently admitted or recently moved, suggesting either that local PHAs are unsuccessful in adjusting rental subsidies to compensate for rising housing costs in larger units, or that mobility costs limit households’ ability to adjust housing costs according to changing housing needs. The relatively higher incidence of high and severe cost burdens for non-White and Hispanic households in previous models is not statistically significant in the subsample of new admissions or new movers.

The influence of regional and census tract-level housing market characteristics also differs significantly between new admissions or new movers versus others. Although renting a unit above the 

FMR has a positive impact on housing cost burden across both samples, census tract median rents Cityscape 55

Dawkins and Jeon

*** ***

*** *** *** *** *** *** *** *** ***

*** ***

* ***

*** *** ***

2.528

0.020

0.084

0.057 0.109

0.703 0.115 0.000

0.193

0.119 0.115

Burden

– 2.283

– 0.361 – 0.132

– 0.700 – 0.659 – 0.446

– 0.021

– 0.090 – 0.029

Severe Cost 

*** ***

*** *** *** *** *** *** *** *** ***

*** *** ***

***

*** *** *** **

3.127

0.018

0.086

0.035 0.081

0.570 0.106

0.006 0.180

0.081 0.061

Others

– 1.566

– 0.313 – 0.148

– 0.698 – 0.725 – 0.439

– 0.000

– 0.106 – 0.046

Burden

High Cost 

*** ***

*** *** *** *** *** *** *** *** ***

*** *** ***

*** ***

*** ***

***

7.697

0.098

0.189

0.203 0.001 0.111

– 1.526

– 0.013 – 0.134 – 0.120 – 0.033 – 1.260 – 1.418 – 0.874

– 0.045

– 0.058 – 0.001

– 0.251 – 0.114

– 0.344

Any Cost 

Burden

*** ***

*** * *** *** ***

**

*** ***

*** ** ***

0.021

0.074

0.025 0.043

0.059 0.048 0.001

Burden

– 0.304 – 2.734

– 0.004

– 0.314

– 0.558 – 0.696 – 0.729

– 0.396 – 0.401

– 1.084 – 0.177 – 0.402 – 0.068

Severe Cost 

*** ***

** ***

*** *** ***

*** ***

*** ***

*** *** *** ***

Burden

0.242

0.004

0.012 0.038

0.122 0.094 0.001

High Cost 

– 2.014

– 0.032 – 0.280 – 0.012 – 0.591 – 0.742 – 0.611

– 0.323 – 0.193

– 0.937 – 0.249 – 0.482 – 0.162

*** ***

** *** *** ** *** *** *** ** ***

*** *** **

*** ***

*** ***

***

den, 2015 (Stratified Models) (1 of 2)

Burden

7.627

0.064

0.134 0.074 0.001

0.253 0.011 0.190

Newly Admitted or Moved to a New Unit

Any Cost 

– 1.676

– 0.006 – 0.077 – 0.181 – 0.065 – 1.261 – 1.377 – 0.866

– 0.138

– 0.415 – 0.265

– 0.559

d

Explanatory Variable

en

oom

opolitan—central city opolitan—suburb opolitan

est

Rent above the payment standar Income as % of AMI

Length of participation Household size Childr Female Elderly Disabled Primarily wage Non-White Hispanic Bedr Single-family Building age

Midwest South

W Metr Metr Micr

Category

Exhibit 9

Factors Associated With the Odds of a Housing Cost Bur

Rent and income

Household characteristics

Housing unit type

Geography

56 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

*** *** ***

*** **

***

***

0.001 0.099

0.003 –  .001

0.354

Burden

– 0.003 – 0.000

– 1.067

Severe Cost 

1,510,059 179,201.01

*** *** *** * *** ***

***

***

0.000 0.099

0.002

0.345

Others

– 0.003 – 0.001

– 0.001

– 1.276

Burden

High Cost 

1,510,059 306,387.17

*** *** * *** ***

***

***

0.000 0.176 0.006 0.001 0.001 0.009

0.427

0.778

Any Cost 

Burden

1,510,059 365,554.96

*** *** ***

***

***

***

0.176 0.009 0.002 0.002 0.001

2.021

0.310

Burden

– 0.000

186,057

Severe Cost 

11,004.85

*** *** *** * ***

***

***

Burden

0.155 0.007 0.002 0.001 0.000

1.458

0.221

High Cost 

– 0.000

186,057 13,472.68

*** *** ***

***

***

***

den, 2015 (Stratified Models) (2 of 2)

Burden

0.201 0.004 0.005

0.004

1.029

0.748

Newly Admitted or Moved to a New Unit

Any Cost 

– 0.000

– 0.001

186,057 42,594.33

ecipients

ent ent above the FMR

Explanatory Variable

Median r Median r Vacancy rate Poverty rate % Minority population % Housing voucher r

p  < 0.1.

e

p  < 0.05. * 

Category

ald Chi-Squar

p  < 0.01. ** 

Exhibit 9

Factors Associated with the Odds of a Housing Cost Bur

Neighborhood characteristics

Constant

Number of observations W Pseudo R2

*** 

AMI = Area Median Income. FMR = Fair Market Rent.

Note: Northeast and Rural are omitted.

Cityscape 57

Dawkins and Jeon

negatively influence housing cost burden for new admissions or new movers. Similarly, cost burdens for others are more sensitive to local vacancy rates and the presence of other HCV households.

To get a better sense of the magnitude of the impacts displayed exhibits 8 and 9, exhibit 10 displays the predicted probability of a housing cost burden at different cost burden levels, allowing the household income, rent payment vis-à-vis the payment standard, and sample to vary, while holding other independent variables at their respective means. Exhibit 10 suggests that the probability of a housing cost burden at any level is highest for those with the lowest incomes. For those earning less than 10 percent of AMI, the probability of any cost burden is nearly 100 percent. For all income levels, the probability of a housing cost burden is higher for those renting a unit above the payment standard, but new admissions or new movers are substantially less likely to exhibit cost Exhibit 10

Predicted Probabilities of Housing Cost Burden by Income and Rent Payment

Newly Admitted or Moved  

Full Sample

Income

to a New Unit

Rent ≤ Payment  Rent > Payment  Rent ≤ Payment  Rent > Payment 

Standard

Standard

Standard

Standard

Predicted probabilities of any cost burden (%)

Total annual income  

13.53

99.71

18.30

99.78

10% or less than AMI

Total annual income  

3.25

98.65

4.02

98.85

11–30% of AMI

Total annual income  

0.72

94.02

0.78

94.15

31–50% of AMI

Total annual income  

0.15

77.15

0.15

75.07

51–80% of AMI

Total annual income  

0.03

42.05

0.03

36.03

more than 80% of AMI

Predicted probabilities of high cost burden (%)

Total annual income  

11.05

69.84

17.16

20.88

10% or less than AMI

Total annual income  

2.51

32.44

2.69

3.40

11–30% of AMI

Total annual income  

0.53

9.06

0.37

0.47

31–50% of AMI

Total annual income  

0.11

2.02

0.05

0.06

51–80% of AMI

Total annual income  

0.02

0.43

0.01

0.01

more than 80% of AMI

Predicted probabilities of severe cost burden (%)

Total annual income  

10.58

54.84

14.01

10.72

10% or less than AMI

Total annual income  

1.20

11.07

1.05

0.77

11–30% of AMI

Total annual income  

0.12

1.26

0.07

0.05

31–50% of AMI

Total annual income 

0.01

0.13

0.00

0.00

 51–80% of AMI

Total annual income  

0.00

0.01

0.00

0.00

more than 80% of AMI

AMI = Area Median Income.

58 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

burdens when renting above the payment standard than are others. For example, the probability of a severe housing cost burden for a recent mover or new admission who earns 10 percent or less of AMI and rents a unit above the payment standard is only 11 percent, and this probability is lower than those in the same group who rent a unit below the payment standard. By comparison, in the full sample, the probability of a severe housing cost burden for a household earning 10 percent or less of AMI and renting a unit above the payment standard is 55 percent.

Conclusion and Policy Implications

This study examined trends in housing cost burden for households participating in the HCV program during the 2003-to-2015 period. We found that the number and share of HCV households 

experiencing a housing cost burden has increased since 2003, and the year-to-year trend in HCV 

cost burden roughly corresponded to the recent housing market boom-bust cycle. We observed 

a dip in housing cost burden during the initial years of the housing market recession, which was likely induced by increased rental market affordability during this period. By 2015, 46 percent of HCV households experienced a cost burden of 31 percent or more, 16 percent experienced a cost burden of 41 percent or more, and 8 percent experienced a cost burden of 51 percent or more. 

Compared to McClure (2005), who examined cost burdens in 2002 just prior to our study period, the prevalence of housing cost burdens of 31 percent or more has increased from 38 percent in 2002 to 46 percent in 2015, whereas the percentage of those spending more than 40 percent of 

income on housing costs has remained approximately the same.

Cross-sectional and longitudinal analyses reveal that HUD program rules play an important role in shaping housing cost burdens, particularly HUD rules governing local payment standards and the restriction on the 40 percent-of-income cost burden cap to new admissions and recent movers. For all years since 2005, those households newly admitted to the HCV program were consistently less likely to exhibit severe housing cost burdens. Those renting a unit above the payment standard in 2003 were more likely to experience a cost burden at the beginning of the voucher contract and exhibit longer housing cost burden trajectories while participating in the HCV program. Residential mobility plays an important role in reducing the incidence of housing cost burdens throughout the HCV contract, despite households initially taking on higher housing cost burdens upon mobility.

These findings hold in logistic regression models explaining the determinants of housing cost burdens at different levels. Recent admissions and new movers are less likely to exhibit high or severe housing cost burdens and are less likely to incur a high or severe housing cost burden when renting above the payment standard. Recent admissions and new movers also respond differently than others to local housing market conditions. Simulations from the regression models suggests that among those earning the lowest incomes (10 percent of AMI), new movers and recent admissions 

are about 44 percent less likely to incur a severe housing cost burden than the average voucher recipient, when renting a unit above the payment standard.

These findings are a double-edged sword. On the one hand, HUD’s programmatic income floor and ceiling have helped to keep housing costs for voucher recipients within a range that is consistent with programmatic goals. On the other hand, our analysis suggests that housing cost burdens rise after a household’s initial lease has expired. HUD’s ability to keep housing cost burdens in check Cityscape 59

Dawkins and Jeon

is limited to some extent by the quality and frequency of local rent-reasonableness evaluations and the local payment standard provision, which places a cap on HUD’s contribution toward rent. 

Some have suggested removing the 40-percent cost burden threshold altogether or eliminating rent reasonableness evaluations to enable households to incur higher housing costs if they so choose (Turnham and Khadduri, 2001). Our evidence suggests that without further changes to the HCV 

program and how it is administered by local PHAs, both the 40-percent threshold on total tenant contributions and the payment standard cap on HAPs will continue to play an important role in shaping housing cost burdens for HCV-assisted households.

Acknowledgments

The authors wish to acknowledge financial support from the U.S. Department of Housing and 

Urban Development (HUD) Multidisciplinary Research Team grant program and the helpful com-

ments from Mark Shroder, Meena Bavan, Barry Steffen, and others in the HUD Office of Policy Development and Research. Special thanks to Ali Sayer from EconSys for managing this project and to Kirk McClure and Ingrid Gould Ellen for helpful comments on a previous draft of this paper.

Authors

Casey Dawkins is the Director of the Urban Studies and Planning Program, the Director of the 

Ph.D. Program in Urban and Regional Planning and Design, associate professor of urban studies and planning, and affiliate of the National Center for Smart Growth at the School of Architecture, Planning and Preservation at the University of Maryland.

Jae Sik Jeon is a housing research analyst at Sage Computing and affiliate of the National Center for Smart Growth at the University of Maryland.

References

Colburn, Gregg, and Ryan Allen. 2018. “Rent Burden and the Great Recession in the U.S.A.,” 

Urban Studies 55 (1): 226–243.

Collinson, Rob. 2011. “Rental Housing Affordability Dynamics, 1990–2009,”  Cityscape 13 (2): 71–103.

Dawkins, Casey J. 2007. “Income Targeting of Housing Vouchers: What Happened After the Qual-

ity Housing and Work Responsibility Act?”  Cityscape 9 (3): 69–94.

———. 2006. “Are Social Networks the Ties That Bind Families to Neighborhoods?”  Housing Studies 21 (6): 867–881.

Devine, Deborah J., Barbara A. Haley, Lester Rubin, and Robert W. Gray. 2000.  The Uses of Discretionary Authority in the Tenant-Based Section 8 Program: A Baseline Inventory of Issues, Policy, and Practice. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

60 Selected Outcomes of Housing Assistance

Housing Cost Burden in the Housing Choice Voucher Program:  

The Impact of HUD Program Rules

DiPasquale, Denise. 2011. “Rental Housing: Current Market Conditions and the Role of Federal 

Policy,”  Cityscape 13 (2): 57–70.

DiPasquale, Denise, and Michael P. Murray. 2017. “The Shifting Demand for Housing by American Renters and Its Impact on Household Budgets: 1940–2010,”  Journal of Regional Science 57 (1): 3–27.

Freeman, Lance. 2011.  The Impact of Source of Income Laws on Voucher Utilization and Locational Outcomes. Assisted Housing Research Cadre Report. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Joint Center for Housing Studies of Harvard University (JCHS). 2017.  The State of the Nation’s Housing. Cambridge, MA: President and Fellows of Harvard College.

Leopold, Josh, Liza Getsinger, Pamela Blumenthal, Katya Abazajian, and Reed Jordan. 2015. 

The Housing Affordability Gap for Extremely Low-Income Renters in 2013. Washington, DC: Urban Institute.

Leser, C.E.V. 1963. “Forms of Engel Functions,”  Econometrica 31: 694–703.

McClure, Kirk. 2005. “Cost Burden in the Housing Choice Voucher Program,”  Cityscape 8 (2): 5–20.

Pendall, Rolf. 2000. “Why Voucher and Certificate Users Live in Distressed Neighborhoods,”  Housing Policy Debate 11 (4): 881–910.

Souza, Maria Teresa, Robert A. Collinson, Marge Martin, Barry L. Steffen, David A. Vandenbroucke, and Yung-Gann David Yao. 2011.  2009 Worst Case Housing Needs of People With Disabilities. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development 

and Research.

Turnham, Jennifer, and Jill Khadduri. 2001.  Issues and Options for HUD’s Tenant-Based Housing Assistance Programs. Washington, DC: Millennial Housing Commission.

U.S. Department of Housing and Urban Development (HUD). 2001.  Housing Choice Voucher 

Program Guidebook. Washington, DC: U.S. Department of Housing and Urban Development.

Varady, David P., Xinhao Wang, and Jay Mittal. 2007. “Developing More Accurate Reasonable Rent Estimates in the U.S. Housing Choice Voucher Program,”  Geography Research Forum 27: 52–69. 

Watson, Nicole Elsasser, Barry L. Steffen, Marge Martin, and David A. Vandenbroucke. 2017.  Worst Case Housing Needs 2017 Report to Congress. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Weinberg, Daniel H., Joseph Friedman, and Stephen K. Mayo. 1981. “Intraurban Residential 

Mobility: The Role of Transaction Costs, Market Imperfections, and Household Disequilibrium,” 

Journal of Urban Economics 9: 332–348.

Cityscape 61

Dawkins and Jeon

Williamson, Anne R. 2011. “Can They Afford the Rent? Resident Cost Burden in Low Income 

Housing Tax Credit Developments,”  Urban Affairs Review 47 (6): 775–799.

Working, H. 1943. “Statistical Laws of Family Expenditures,”  Journal of the American Statistical Association 38: 43–56.

62 Selected Outcomes of Housing Assistance

Opting In, Opting Out: 

A Decade Later

Anne Ray    

University of Florida

Jeongseob Kim  

Ulsan National Institute of Science and Technology

Diep Nguyen

Jongwon Choi

University of Florida

Kelly McElwain

Keely Jones Stater

Public and Affordable Housing Research Corporation

Abstract

This article updates the U.S. Department of Housing and Urban Development (HUD) report 

Multifamily Properties: Opting In, Opting Out and Remaining Affordable  (Finkel 

et al., 2006). The original report examined the loss of affordable housing associated with HUD’s Section 8 project-based rental assistance and Section 236 and 221(d)(3) subsidized 

mortgage programs between 1998 and 2004. It found that properties with low rents 

compared to the surrounding Fair Market Rent (FMR), that serve a family population, and 

that are owned by for-profit corporations were particularly at risk for loss of affordability.

The analysis is updated here for the period 2005 to 2014. It shows that more owners 

made active decisions to opt in to Section 8 assistance in the latter period, while HUD’s older subsidized mortgage programs were largely being phased out. Factors such as for-profit ownership and low rent-to-FMR ratios continued to be associated with higher risk 

of loss of affordability, but these factors were less influential from 2005 to 2014 than in the original study.

The article also explores the use of the Low-Income Housing Tax Credit Program and 

HUD refinancing to preserve affordability in Section 8 developments. The analysis finds 

that these preservation tools are associated with extended affordability for thousands of HUD-assisted properties. Additional preservation initiatives and improved targeting may 

be needed to preserve other HUD-assisted properties, particularly smaller developments 

in strong real estate markets.

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 63

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Introduction

U.S. Department of Housing and Urban Development (HUD) multifamily programs are a critical 

source of affordable housing. As of 2016, these programs provided over 1.4 million assisted housing units: privately owned rental units whose owners receive subsidized mortgages, rent subsidies, or both in exchange for making these units affordable to low-income households through tenant income and rent restrictions. Affordable housing options provided by these programs serve a particularly vulnerable population. Three-fourths of developments funded through HUD’s multifamily programs serve extremely low-income households (under 30 percent of area median income). 

More than one-half serve elderly households (HUD, 2016).

The affordability of the assisted housing stock faces risk from two directions. First, tenant income and rent restrictions placed on properties are time-limited. When restrictions expire, owners of developments in strong real estate markets can opt out of subsidy programs and convert properties to market-rate rental units, condominiums, or other uses. Second, the HUD-assisted housing stock dates to the 1960s through 1980s. Many aging properties are at risk of loss because of foreclosure or abatement of rental assistance contracts due to poor physical conditions.

Pinpointing the properties that are most at risk of loss enables affordable housing developers, advocates, and funders to intervene early to preserve affordability and allow these properties to continue to serve low-income households. Preservation of assisted properties usually calls for new subsidized financing to rehabilitate aging facilities and extend affordability requirements. It often also includes transferring properties to owners whose mission is to provide affordable housing to low-income tenants over the long term.

Original Study

To help the affordable housing community develop early warning systems for at-risk properties, in 2006 HUD published  Multifamily Properties: Opting In, Opting Out and Remaining Affordable (Finkel et al., 2006). The study assessed the loss of HUD-assisted multifamily units over the 1998-to-2004 

period. The study included properties subsidized by HUD’s 1960s- to 1970s-era Section 221(d)(3) and Section 236 programs, which provided low-interest mortgages to housing owners, and HUD’s 

1970s- to 1980s-era project-based Section 8 program, through which HUD offered ongoing as-

sistance to subsidize rents of low-income tenants. In all, the study covered 22,471 rental developments across the country (Finkel et al., 2006).

The 1998-to-2004 period was a risky time for the HUD-assisted inventory because developments 

funded in the 1960s and 1970s had passed the 20-year mark. Although subsidized mortgages typically carried a 40-year term, most were eligible for prepayment after the first 20 years. If an owner prepaid a mortgage, rent and income restrictions on the property were extinguished. Similarly, owners could choose not to renew Section 8 rental assistance contracts after their initial term expired, also usually 20 years. This would also end income and rent restrictions.

Finkel et al. (2006) tracked losses to the affordable housing inventory from prepayments, contract opt-outs, foreclosures, and contract abatements due to poor conditions. It found that 4,100 properties with 303,638 affordable units were lost between 1998 and 2004, amounting to 19 percent of the 64 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

inventory. At the same time, owners of 11,126 properties with 785,143 units made active decisions to renew rental assistance contracts (Finkel et al., 2006). Most of the rest of the properties did not require active decisions during the study period, such as developments whose Section 8 contracts were not due to expire until after 2004.

The authors used descriptive cross-tabulations and multivariate analysis to identify the property characteristics associated with heightened risk of loss of affordability. The analysis identified the following characteristics associated with higher risk of property loss: smaller properties, low rents compared to the surrounding Fair Market Rent (FMR), funding from HUD’s early assistance programs, family occupancy, and for-profit ownership. Conversely, properties with nonprofit 

ownership, higher rents compared to the surrounding FMR, and elderly and disabled occupancy 

type were at lower risk of loss (Finkel et al., 2006). 

Updated Analysis

This study replicates the Finkel et al. (2006) analysis for a new time period, 2005 to 2014. It addresses two research questions—

1. Do basic characteristics such as location, ownership, physical attributes, and neighborhood characteristics explain differences in the prevalence of opt-outs/prepayments and opt-ins?

2. How have the patterns and trends in opt-outs and prepayments changed since the initial study took place?

To answer these questions, we assessed the status of over 18,000 properties that remained in HUD’s multifamily portfolio following Finkel et al. (2006). A baseline dataset of these properties from 2005 

was compared to the 2014 inventory to determine whether properties have continued to operate as affordable housing. As in the original study, properties that exited the affordable inventory were classified based on whether they exited through an owner’s choice not to renew Section 8 contracts ( opt-out), prepayment or maturing of subsidized mortgages, HUD foreclosure and contract abatement actions, or a combination of these. For properties that continued to operate in the assisted inventory, we determined whether the owner made an active choice during the study period to renew assistance ( opt-in). 

This article begins with a discussion of the data and methods used to classify the properties into opt-in, opt-out/prepay, and foreclosure/abatement outcome categories and to update the quantitative analyses. Next, the article provides the results of the cross-tabulations of outcomes by property characteristics and the multivariate analysis of the effects of selected characteristics on the owners’ 

opt-in/opt-out decisions.

Finally, we present additional analysis exploring preservation interventions among properties with an active Section 8 opt-in. The infrastructure for preserving at-risk assisted housing emerged during the 1990s and continued to mature during the study periods for both Opting In, Opting Out analyses. This article takes a first step toward tracking the effects of these initiatives by identifying properties with financial transactions potentially related to preservation, including Low-Income Housing Tax Credit Program (LIHTC) allocations, HUD’s Mark-to-Market process, and HUD-insured refinancing. The article also examines the extent to which preservation tools have been targeted toward properties that are particularly at risk of opt-out.

Cityscape 65

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Data and Methods

The new analysis mirrors the 2006 study’s methods and data sources for classifying properties by opt-in/opt-out/foreclosure-abatement status, cross-tabulating property characteristics with subsidy outcomes, and multivariate analysis of the opt-in/opt-out decision.

Active properties from 2005 were included if they had funding from at least one of these HUD 

programs, classified as “older” or “newer” assistance programs in the original study (Finkel et al., 2006)—

•  Older (1960s to 1970s): Rent Supplement (Rent Supp), Rental Assistance Payment (RAP), 

and Loan Management Set-Aside (LMSA) rental assistance programs; Section 221(d)(3) 

Below Market Interest Rate (BMIR) and Section 236 Mortgage Assistance Program subsidized 

mortgages.

•  Newer (1970s to 1980s): Section 8 New Construction/Substantial Rehabilitation (NC/SR) and 

Section 8 Moderate Rehabilitation (Mod Rehab).1 

For brevity, all the types of rental assistance listed previously, including the older Rent Supp and RAP programs, are hereafter referred to as “Section 8.” The mortgage programs hereafter are 

referred to together as “236/BMIR.”

Classifying Properties by Outcomes

HUD provided point-in-time property-level datasets for 2005 and 2014, listing all developments with active subsidies from Section 8, 236/BMIR, or both. The study universe was made up of properties in the HUD inventory in 2005: 18,107 developments with 1.49 million housing units. We 

compared the two datasets to determine whether each property continued as subsidized housing 

( stayers) or left the subsidized inventory between 2005 and 2014 ( leavers). Properties were placed in four categories based on their reasons for staying or leaving the subsidized housing inventory—

1.  Opt-in refers to stayers where the owner actively renewed a Section 8 contract during the study period. Most of these properties had no 236/BMIR mortgage. Some had mortgages that were still active or had matured by 2014.

2.  Opt-out refers to leavers with a Section 8 opt-out, 236/BMIR prepayment or both. Properties with both types of assistance were included if the owner actively terminated both subsidies or if the property had a Section 8 opt-out and a maturing mortgage.

3.  Foreclosure/abatement refers to leavers where HUD abated a Section 8 contract due to property conditions, foreclosed on a 236/BMIR mortgage, or both. A small number of stayers that were 

still undergoing the contract abatement process in 2014 were also included.

1 The original study noted that, “(t)he Office of Public and Indian Housing manages most projects assisted with Section 8 

moderate rehabilitation. These projects are not included in the Real Estate Management System (REMS) or a comparable database. The REMS database only includes the subset of such projects that are also associated with the Section 8 Property Disposition program. Consequently, our analysis of the Section 8 Moderate Rehabilitation projects in this study is limited to this part of the stock” (Finkel et al., 2006: 3). The same constraint applies to the updated version.

66 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

4. The  Other category covers both stayers and leavers without a clear opt-in, opt-out or foreclosure and abatement action. Most were stayers where the owner did not have to make a Section 8 

renewal choice because the Section 8 contract term continued through the entire 2005-to-2014 

period. It also includes stayers where the owner made a mixed decision to continue one type of assistance but not the other. Most notably, owners of hundreds of properties prepaid 236/BMIR 

mortgages from 2005 through 2014 but continued to have active Section 8 assistance. The 

category also includes a small number of leavers where 236/BMIR mortgages were terminated 

for reasons other than prepayment, maturity or foreclosure.

Property Characteristics for Descriptive Cross-Tabulations

As in the original study, properties in the four outcome categories were cross-tabulated with a series of property, financing, owner, location, and tenant characteristics. Exhibit 1 shows the data sources for each characteristic. Unless otherwise noted, properties were classified based on their characteristics during the 2005 baseline year. 

Exhibit 1

Data Sources for Property Characteristics

Variables

Data Source

Property geocoding (census tract, MSA, census division,  Generated from HUD iREMS

metropolitan location)

Property characteristics (size, occupancy type, building  2005 active properties, multifamily building type type, percent assisted, building age based on 

(generated from HUD iREMS)

occupancy date, REAC score)

Units by number of bedrooms

2005 active properties

HUD program type: older assisted versus newer assisted 2005 active properties

Detailed financing information (financed by FHA 

HUD 2005 and 2014 active properties, active 

insurance, Section 202, Section 811, USDA Section 

financing, active contracts, and multifamily 

515, state HFAs) 

building type files, iREMS; terminated 

contracts database; terminated multifamily 

mortgages database

Ownership type

2005 active participants database (generated 

from HUD iREMS)

Neighborhood characteristics (median household 

U.S. Census Bureau, 2005–2009 American 

income, median gross rent, median value of owner-

Community Survey. Summary files 

occupied housing, homeownership rate, poverty rate, 

constructed by Minnesota Population Center. 

homeowner and renter vacancy rate, racial/ethnic 

National Historical Geographic Information 

composition)

System: Version 2.0.

Home sales market (change in Housing Price Index)

Federal Housing Finance Agency Housing Price 

Index

Rental market (change in HUD FMR)

HUD annual FMR datasets, 2005–2014

Tenant characteristics (length of residence, household 

HUD Picture of Subsidized Households, 2005

size, percent minority headed, percent elderly, percent 

with children, household income as percentage of AMI)

AMI = Area Median Income. FHA = Federal Housing Administration. FMR = Fair Market Rent. HFA = housing finance agency. 

HUD = U.S. Department of Housing and Urban Development. iREMS = Integrated Real Estate Management System. MSA = 

metropolitan statistical area. REAC = Real Estate Assessment Center. USDA = U.S. Department of Agriculture.

Cityscape 67

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Multivariate Analysis

Both studies include a logistic regression model to isolate the effects of property, financing, and location characteristics on owners’ decisions to opt into or out of Section 8 assistance. As noted in the original report (Finkel et al., 2006), multivariate analysis adds depth to the initial findings of the descriptive statistics. The model isolates the influence of each variable on the opt-in/opt-out decision. The apparent relationships between some variables and property outcomes in the 

descriptive cross-tabulations may drop out when the other factors are controlled.

The original analysis included the Section 8 only properties where the owner had to make an 

explicit decision to renew or opt out of a contract from 1998 to 2004. The new version widens this universe slightly by adding the small group of Section 8 and 236/BMIR properties where the owner made at least one opt-in/opt-out choice during the 2005-to-2014 study period. Properties were excluded if they did not have Section 8 assistance, if their Section 8 contracts did not come up for renewal between 2005 and 2014, or if data were insufficient to evaluate all variables. In all, the multivariate analysis covers 10,023 properties.

Results

In this section, we report the results of the classification of properties by HUD funding program and outcome, the descriptive cross-tabulations, and the multivariate analysis. We find that many of the same factors associated with the loss of affordable properties from the earlier study period continued to apply but that these patterns were less pronounced in the 2005-to-2014 period.

Stayers and Leavers

The analysis shows that property losses had slowed considerably since the Finkel et al. (2006) study period. Only 8 percent of properties were lost from 2005 to 2014, compared to 19 percent of properties in the original study. Exhibit 2 shows the change in the number of properties with each combination of Section 8 and 236/BMIR subsidy. Exhibit 3 shows the breakdown of properties by the four outcome categories: opt-in, opt-out/prepay, foreclosure/abatement, and the miscellaneous Other category. 

Exhibit 2

Property Inventory Changes by Subsidy Type, 2005–2014

Remained in 2014 Inventory

Left Inventory by 2014

Leavers as  

Percent of  

Subsidy Type

Percent of  

Percent of  

Subsidy Type, 

Number

Number

Subsidy Type

Subsidy Type

Finkel et al. 

(2006)

Section 8 only

14,543

95

737

5

9

236/BMIR only

79

15

443

85

82

Section 8 and 236/BMIR

2,033

88

272

12

32

Total

16,655

92

1,452

8

19

236/BMIR = Section 236 Mortgage Assistance Program and Section 221(d)(3) Below Market Interest Rate Program.

Sources: U.S. Department of Housing and Urban Development, 2005 and 2014 active properties and active financing files, Integrated Real Estate Management System; Finkel et al. (2006)

68 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Exhibit 3

Properties by Summary Outcome Categories, 2005–2014

Opt-Outs/ 

Foreclosure/ 

Opt-Ins

Other

Total

Prepays

Abatement

Number of properties

12,786

748

293

4,280

18,107

Percent of properties

71

4

2

24

100

Sources: U.S. Department of Housing and Urban Development, 2005 and 2014 active properties, active financing, and active contracts files, Integrated Real Estate Management System; terminated contracts database; terminated multifamily mortgages database

•  Section 8 only.  In the original report, the most common outcome (68 percent of cases) was continued affordability in a Section 8 only property, either through an opt-in contract renewal or a  no choice continuation of an existing contract. This pattern was even stronger in the new analysis; 80 percent of the study dataset was made up of Section 8 only properties where either the owner renewed a contract between 2005 and 2014 or the contract extended throughout the 

entire period. Fewer than 4 percent of the Section 8 only properties left the inventory during the study period. Most did so through owner opt-outs, with a smaller number of HUD-abated 

contracts.

•  Section 236/BMIR only.  Within the small stock of 236/BMIR properties without Section 8 that remained in 2005, most left the subsidized inventory through mortgage maturity, prepayment, 

or another termination reason by 2014. 

•  Section 8 and 236/BMIR.  For properties with both types of assistance, the most common outcome was a Section 8 opt-in combined with an end to the 236/BMIR mortgage through 

prepayment, maturity, or other termination reasons. Surprisingly, Section 8 opt-outs upon 

mortgage termination did not appear to be a threat to the inventory. Owners of properties with prepaid or maturing mortgages opted out of Section 8 assistance in only 112 cases, comprising 9 percent of Section 8 properties with prepaid or maturing mortgages.

Overall, the 2005-to-2014 analysis reflects two trends: the continuation of most Section 8 assistance and the winding down of subsidized mortgage programs. 

Descriptive Cross-Tabulations

Exhibit 4 shows the cross-tabulations of properties in the four subsidy outcome categories by property, tenant, financing, and location characteristics.2 

The original report emphasized the loss of affordability in properties with family occupancy type, low rent-to-FMR ratios, and for-profit ownership. These patterns were also present in the 2005-to-2014 cross-tabulations.

First, housing for families was particularly at risk. Properties with family occupancy type made up 75 percent of opt-outs and 70 percent of foreclosure/abatements, even though they only 

made up 48 percent of the total property inventory. Single-parent households with children were 2 Ray et al. (2015) included a more extensive discussion of the rules for classifying properties by outcome, a table showing properties by detailed combinations of Section 8 and 236/BMIR outcomes, and a more detailed discussion of the descriptive cross-tabulation results.

Cityscape 69

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Exhibit 4

Property, Financing, Location, and Tenant Characteristics by Outcome (1 of 3)

Opt-Outs/ 

Foreclosure/ 

Opt-Ins

All Other

Total

Prepays

Abatement

Property characteristics

Number of properties

12,786

748

293

4,280

18,107

Percent of properties

71

4

2

24

100

Development size 

1–49 (%)

44

46

51

28

40

50–99 (%) 

27

23

22

32

28

100–199 (%) 

23

22

18

30

25

200 or more (%) 

6

8

9

10

7

Average number of units 

77

79

81

98

82

Unit size

0 bedrooms (%)

7

5

4

5

7

1 bedrooms (%)

55

37

29

46

52

2 bedrooms (%)

25

43

43

31

27

3 bedrooms (%)

11

14

20

15

12

4+ bedrooms (%)

2

1

3

2

2

Average number of bedrooms 

1.6

1.8

2

1.7

1.6

Occupancy type (%)

Elderly/disabled

59

25

30

38

52

Family

41

75

70

62

48

Building type (%)

Rowhouse

10

6

8

8

9

Townhouse 

3

5

2

4

3

Semidetached 

5

3

3

4

5

Detached

5

4

7

1

4

Walkup/garden

36

56

55

37

37

Midrise

3

1

2

3

3

Mixed

12

13

15

18

14

Highrise/elevator 

28

11

9

25

26

Categories of rent-to-FMR ratio (%)

Below 80% 

11

28

21

16

13

80–100%

24

37

41

30

26

101–120% 

27

20

22

30

27

121–130% 

11

6

5

8

10

131–40% 

8

3

5

6

7

141–160% 

10

4

2

6

9

Over 160% 

9

3

3

4

7

Building-age categories (%)

Before 1975 

17

38

33

30

21

1975–1979 

20

19

21

30

22

1980–1985 

46

35

31

37

43

After 1985

17

8

15

3

13

Ownership typea (%)

Nonprofit 

49

25

36

26

43

For profit 

33

40

32

39

35

Limited dividend 

15

27

28

30

19

Other

2

8

4

4

3

REAC Physical Inspection Score (1–100)

Median 

91

88

79

89

90

1–59 (%)

2

5

24

4

3

60–69 (%)

6

9

12

6

6

70–89 (%)

36

42

36

41

37

90–100 (%)

56

44

28

50

54

70 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Exhibit 4

Property, Financing, Location, and Tenant Characteristics by Outcome (2 of 3)

Opt-Outs/ 

Foreclosure/ 

Opt-Ins

All Other

Total

Prepays

Abatement

Financing characteristics

Newer (Section 8 NC/SR, Mod 

80

57

55

58

73

Rehab) (%)

Older (Rent Supp/RAP, LMSA, 

20

43

45

42

27

236/BMIR) (%)

Detailed HUD program type (%)

Sec. 8 NC/SR

25

25

16

25

25

Sec. 202

29

4

15

3

21

Sec. 8/LMSA 

19

25

41

25

21

Sec. 8/515

10

7

8

2

8

Sec. 8/HFDA 

12

11

7

27

15

Sec. 8/Preservation 

2

3

1

2

2

Sec. 8/PD 

3

10

9

1

3

Rent Supp/RAP 

1

0

0

6

2

No Rental Subsidy 

0

16

4

9

3

Average percentage of assisted 

94

60

82

83

90

units

Primary form of financing (%)

FHA insured

27

39

23

51

33

Section 202/811

27

4

12

2

20

Section 515

10

7

8

2

8

All other

36

51

57

45

39

HFA-related properties

Number of HFA-related 

1,553

82

21

1,140

2,796

properties

Percent of HFA-related 

56

3

1

41

100

properties

FHA insured (%)

24

21

19

24

24

Noninsured (%)

76

79

81

76

76

Location and market characteristics

Census division (%)

New England 

10

4

3

14

10

Mid Atlantic 

13

10

9

15

13

East North Central 

18

12

20

20

18

West North Central 

11

19

16

11

11

South Atlantic 

16

16

22

14

15

East South Central 

7

6

8

9

8

West South Central 

7

10

15

6

7

Mountain 

5

7

3

5

5

Pacific 

13

16

4

8

12

Metropolitan location (%)

Suburb 

31

28

22

33

31

Principal city 

51

60

61

53

52

Nonmetropolitan 

18

12

18

15

17

Neighborhood characteristics (averages for census tracts surrounding properties)

Median household income ($)

39,831

41,937

35,371

38,363

39,498

Median gross rent ($)

693

741

652

675

690

Median value of owner-

197,022

200,939

146,169

191,037

194,958

occupied housing ($)

Homeownership rate (%) 

52

51

51

50

52

Cityscape 71

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Exhibit 4

Property, Financing, Location, and Tenant Characteristics by Outcome (3 of 3)

Opt-Outs/ 

Foreclosure/ 

Opt-Ins

All Other

Total

Prepays

Abatement

Poverty rate (%) 

22

20

26

23

22

Homeowner vacancy rate (%)

3

3

4

4

3

Renter vacancy rate (%) 

7

8

10

8

8

Racial/ethnic composition (%)

White

60

59

48

58

59

African American 

19

19

36

22

20

Hispanic

15

15

11

14

15

Asian

3

3

2

3

3

Other

3

3

3

3

3

Minority 

40

41

52

42

41

Regional housing market (averages for MSAs or non-MSA state values surrounding property)

Home sales market 

Average percent change in 

3

6

5

1

3

HPI, full study period (2005 

Q1–2014 Q1)

Average percent change in 

14

16

14

12

14

HPI, strong market period 

(2005 Q1–2007 Q1)

Average percent change in 

– 13

– 14

– 11

– 13

– 13

HPI, weak market period 

(2007 Q1–2012 Q1)

Average percent change in 

6

8

4

6

6

HPI, recovering market 

period (2012 Q1–2014 Q1)

Rental market

Average percent change in 

27

27

29

26

27

FMR, 2005–2014

Average tenant characteristics

Length of residence (years)

5.9

5.9

5.2

5.8

5.9

Household size (people)

1.6

1.9

2.1

1.9

1.7

Percent minority headed

40

46

66

45

41

Percent of all persons with 

22

17

17

16

20

disability 

Percent elderly headed

49

30

20

39

46

Percent with 2+ adults and 

5

9

8

8

6

children 

Percent with 1 adult and children 

20

30

41

28

22

Household income as a 

23

22

17

22

22

percentage of AMI 

236/BMIR = Section 236 Mortgage Assistance Program and Section 221(d)(3) Below Market Interest Rate Program. AMI = 

Area Median Income. FHA = Federal Housing Administration. FMR = Fair Market Rent. HFA = housing finance agency.  

HFDA = Housing Finance and Development Agency. HPI = Housing Price Index. LMSA = Loan Management Set-Aside program. Mod Rehab = Section 8 Moderate Rehabilitation program. MSA = metropolitan statistical area. NC/SR = New Construction/Substantial Rehabilitation program. PD = Property Disposition. Q1 = first quarter. RAP = Rental Assistance Payment program. REAC = Real Estate Assessment Center. Rent Supp = Rent Supplement program. 

a Ownership type was missing for 15 percent of properties. Percentages refer to set of properties for which owner type was known.

Sources: U.S. Department of Housing and Urban Development, 2005 and 2014 active properties, active financing, active contracts, and multifamily building type files, Integrated Real Estate Management System; terminated contracts database; terminated multifamily mortgages database; 2005 and 2014 FMRs; U.S. Census Bureau, 2005–2009 American Community Survey; Federal Housing Finance Agency HPI; 2005  Picture of Subsidized Households

72 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

disproportionately present in the opt-out category and, in particular, foreclosure/abatement properties. Conversely, properties with elderly or disabled occupancy type made up over half of total properties, but only 25 to 30 percent of opt-outs and foreclosure/abatement properties.

Second, properties with rents below those of the regional market continued to be overrepresented among opt-outs. This is represented by the rent-FMR ratio. The numerator of this ratio is the rent that the property owner receives each month from HUD and the tenant; the denominator is the 

HUD FMR for the surrounding metropolitan area or county. Owners receiving below-market rents 

have a greater incentive to opt out, because they are more likely to be able to command higher rents from unsubsidized tenants. Only 13 percent of all properties in the study had rent-FMR 

ratios below 80 percent, but they represented 28 percent of all opt-outs. Properties with rents at 80 

to 100 percent of FMR made up 26 percent of the total inventory but 37 percent of opt-outs. 

Properties owned by for-profit and limited dividend corporations continued to be at higher risk of opt-out. Nevertheless, the 2005-to-2014 analysis shows some shift in opt-outs to nonprofits. 

Nonprofit owned properties made up 25 percent of opt-outs in the 2005-to-2014 study, compared to only 9 percent in the earlier study. 

Unsurprisingly, properties in neighborhoods with stronger markets were more prone to opt-outs, whereas those in weaker markets were more prone to foreclosure/abatement. Among opt-out 

properties, census tract-level median household income, median rent, and home values were 

higher, but the poverty rate was lower. For the foreclosure/abatement properties, these patterns were reversed. Also unsurprisingly, foreclosures and abatements were more common among the 

oldest properties (pre-1975) and those with failing physical inspection scores from HUD’s Real Estate Assessment Center (REAC).

In summary, the descriptive analysis showed that fewer properties left the assisted housing inventory from 2005 to 2014 compared to the original 1998-to-2004 study period. To the extent they did occur, opt-outs continued to disproportionately affect developments with rents below the regional FMR and in strong neighborhood markets. Opt-outs were also more common in properties with 

for-profit or limited dividend owners, although nonprofit-owned properties made up a larger share of opt-outs than in the original study. For their part, properties in weak neighborhood markets and with failing physical inspection scores were disproportionately affected by foreclosures and contract abatements. Finally, family-occupied properties continued to face higher risk from both opt-outs and foreclosures/abatements than developments with elderly and disabled occupancy. 

Multivariate Analysis

The second type of quantitative analysis in the original study was a logistic regression model to isolate the effects of property, financing, and location characteristics on owners’ decisions to opt into or out of Section 8 assistance. The independent variables were potential opt-out risk factors; the dependent variable was the decision to opt out of a Section 8 contract during the study period.

In the earlier study, the multivariate analysis identified these variables as key factors that increased opt-out risk: rent-FMR ratio below 80 percent, properties with all units receiving Section 8 assistance, for-profit/limited dividend ownership, family occupancy, property size below 50 units, and concentrations of units with fewer than three bedrooms (Finkel et al, 2006). 

Cityscape 73

Ray, Kim, Nguyen, Choi, McElwain, and Stater

The updated model includes all independent variables from the original study plus variables 

related to REAC scores, census tract minority and homeownership rates, and neighborhood and 

metropolitan housing market indicators. It also includes the presence of a maturing 236/BMIR 

mortgage as a potential opt-out risk factor, a variable that would not have been relevant during the previous study period.

Exhibit 5 shows the variables included in the updated analysis. The rightmost column describes the anticipated direction for each variable. For variables repeated in both study periods, this is a description of the results from the previous analysis. For new variables, we suggest an expected direction.

Exhibit 5

Variables for Regression Model (1 of 3)

Original Results or  

Variable Type

Variable

Categories

Expected Direction

Property

Property size (units) Less than 50 units (ref.) 

Original study: Reduced odds of opt-

50–99 units 

out as project size increases.

100–199 units 

200+ units 

Density 

Percent of units with 3 or 

Original study: Reduced odds of 

more bedrooms

opt-out as number of larger units 

increases.

Occupancy type 

Family

Original study: Increased odds of 

Elderly/disabled (ref.)

opt-out in family properties

Building type 

Detached/semidetached

Original study: Slightly increased 

Other types (ref.) 

odds of opt-out in detached/

semidetached.

Ownership type

Nonprofit

Original study: Increased odds 

For-profit/limited dividend 

of opt-out for for-profit-owned 

(ref.)

properties.

REAC

Physical REAC score 

Expected direction: Unknown. Higher 

(1-point increase)

REAC score may increase odds 

of opt-out because properties in 

good physical condition are more 

marketable to market-rate tenants. 

However, owners of properties in 

poor condition could opt out of 

subsidy programs to preempt a 

HUD decision to abate subsidies. 

Ratio of rent-to-FMR  Less than 80% 

Original study: Strongly increased 

80–99.9%

odds of opt-out for properties 

100–119.9% (ref.)

with lower rent-to-FMR ratios, 

120–129.9%

particularly less than 80 percent.

130–139.9%

140–159.9%

160% or more 

74 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Exhibit 5

Variables for Regression Model (2 of 3)

Original Results or  

Variable Type

Variable

Categories

Expected Direction

Financing

Older assisted HUD  Older assisted = 1 

Original study: Increased odds 

programs 

Newer assisted = 0

of opt-out for older assisted 

properties.

100-percent 

Projects with 100 percent 

Original study: Strongly increased 

assisteda 

of units with rental 

odds of opt-out for 100-percent 

assistance = 1 

assisted properties.

Other = 0

FHA insured

FHA insured =1

Expected direction: Lower odds 

Other = 0

of opt-outs for properties with 

insured mortgages because 

some FHA mortgages impose 

affordability restrictions.

HFA related

HFA related =1

Unknown: Original results were not 

Other = 0

statistically significant.

Mortgage matured

Matured mortgage of 236/

Expected direction: Higher odds of 

BMIR properties = 1

opt-out for properties with maturing 

Other = 0

mortgages, because restrictions 

associated with the mortgage are 

ending.

Neighborhood Poverty rate

Percent of persons below 

Original results: Lower odds 

poverty level in a census 

of opt-out for properties in 

tract 

neighborhoods with higher poverty 

rate.

Minority rate

Percent of minority (All races  Expected direction: Lower odds 

except non-Hispanic 

of opt-out because of difficulty 

White) in a census tract

marketing developments in racially 

segregated areas.

Homeownership rate Percent of owner-occupied  Expected direction: Unknown. Higher 

housing in a census tract

homeownership rate may signal 

an overall stronger neighborhood 

housing market but also may 

imply a smaller surrounding renter 

population, making it more difficult 

to attract tenants.

Location

Metropolitan location Suburbs (ref.)

Original study: Increased odds 

Principal cities

of opt-out for central city and 

Nonmetropolitan

nonmetropolitan locations

Census division 

New England 

Original study: Increased odds of 

Mid Atlantic 

opt-out for Mid Atlantic, East 

East North Central 

North Central, West North Central, 

West North Central 

West South Central, Mountain and 

South Atlantic (ref.)

Pacific regions; decreased odds 

East South Central 

for New England and East South 

West South Central 

Central.

Mountain 

Pacific 

Cityscape 75

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Exhibit 5

Variables for Regression Model (3 of 3)

Original Results or  

Variable Type

Variable

Categories

Expected Direction

Housing 

Neighborhood rent  Ratio of median gross rent  Expected direction: Increased 

market

level

between a census tract 

odds of opt-out for properties in 

and a county

neighborhoods with a high tract 

rent-county rent ratio.

Neighborhood 

Ratio of median value of 

Expected direction: Increased 

housing value

owner-occupied housing 

odds of opt-out for properties in 

between a census tract 

neighborhoods with a high tract 

and a county

value-county value ratio.

Regional sale market Percent change in HPI 2007  Expected direction: Increased 

Q1–2012 Q1

odds of opt-out for properties 

25% or more

in neighborhoods with positive 

0–24.99%

change in HPI.

– 25%~ 0.01% (ref.)

– 50%~ – 25.01%

Lower than – 50% 

Regional rental 

Percent change in FMR 

Expected direction: Increased 

market

during the recession, 2007  odds of opt-out for properties in 

Q1–2012 Q1

neighborhoods with higher positive 

20% or more

change in FMR.

10%–19.99%

0%–9.99%

Less than 0% (ref.)

236/BMIR = Section 236 Mortgage Assistance Program and Section 221(d)(3) Below Market Interest Rate Program.

FHA = Federal Housing Administration. FMR = Fair Market Rent. HFA = housing finance agency. HPI = Housing Price Index. 

HUD = U.S. Department of Housing and Urban Development. Q1 = first quarter. REAC = Real Estate Assessment Center.  

ref. = reference category. 

a Developments were classified as 100-percent assisted if total units exceeded assisted units by no more than two. The two-unit difference was allowed to account for developments where one or two units are used as management offices.

Exhibit 6 shows the results of the logistic regression model. The odds ratios refer to the odds that an owner will opt out of a Section 8 contract with that particular characteristic. For categorical variables (for example, property size), the odds ratios refer to the characteristics in comparison to the reference group. For continuous variables, the odds ratios show the direction of increase or decrease in odds of opt-out given a positive change in the indicator. 

Family properties continued to be at higher risk in the 2005-to-2014 study period. Odds of 

opt-out for family properties were approximately double those of properties restricted to elderly/

disabled tenancy, similar to the results in the original study. 

Smaller properties and units also continued to be at higher risk of loss. Properties with fewer than 50 units had a two- to four-times higher risk of opt-out than larger properties. Similarly, as the share of units with three or more bedrooms in a property increased, its risk of opt-out decreased, indicating that properties offering more studio and one- or two-bedroom units were at increased opt-out risk. Both of these results are consistent with the original study.

Two other key risk factors continued to affect opt-outs from 2005 to 2014, but they played a 

smaller role than in the original study. First, low rent-to-FMR ratios continued to be a risk factor. 

Odds of opt-out for properties with rent-to-FMR below 80 percent from 2005 to 2014 were three 76 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Exhibit 6

Odds Ratios for Opt-Out Decision Model, 2005–2014

Variable

Odds Ratio  p-value

Property size (ref. under 50 units)

Property size 50–99

0.388

***

Property size 100–199

0.339

***

Property size 200+

0.232

***

Density

Percent of units with 3 or more bedrooms

0.332

***

Occupancy types

Family (ref. elderly/disabled)

2.207

***

Building types

Detached or semidetached (ref. other)

1.044

Ownership types

Nonprofit (ref. for-profit/limited dividend)

0.472

***

REAC physical score

1-percentage-point increase

0.982

***

Program

Older assisted (ref. newer assisted)

0.552

***

100% assisted units (ref. partially assisted)

0.132

***

FHA insured

0.432

*

HFA related

1.999

**

Matured 236/BMIR mortgage

1.016

Neighborhood

Poverty rate

0.356

Minority rate

0.869

Homeownership rate

0.359

**

Rent-to-FMR ratio (ref. 100–119.9%) Less than 80%

2.990

***

80–99.9%

1.885

***

120–129.9%

0.730

130–139.9%

0.455

**

140–159.9%

0.564

*

160% or more

0.512

*

Metropolitan location (ref. suburbs)

Principal cities

1.213

Nonmetropolitan 

1.088

Census division (ref. South Atlantic)

New England

0.363

***

Mid Atlantic

0.827

East North Central

0.612

**

West North Central

1.233

East South Central

0.790

West South Central

0.780

Mountain

1.879

**

Pacific

0.853

Neighborhood market

Neighborhood rent/county ratio

4.036

***

Neighborhood housing value

1.365

**

Regional sale market (2007–2012) 

Severe decline: – 50% or more 

0.929

(ref. moderate decline; – 25% to 

Decline: – 25% to – 50 %

0.942

0% change )

Growing: 0–25%

1.634

***

Strongly growing: 25% or more

1.483

Regional rental market (2007–2012)  Stable: 0–10%

1.385

(ref. FMR declining)

Growing: 10–20%

1.197

1.218

Total number of properties

10,023

Opt-outs

361

Pseudo  R2

0.2101

236/BMIR = Section 236 Mortgage Assistance Program and Section 221(d)(3) Below Market Interest Rate Program.

FHA = Federal Housing Administration. FMR = Fair Market Rent. HFA = housing finance agency. REAC = Real Estate Assessment Center. ref. = reference category.

***  p  < 0.01. **  p  < 0.05. *  p  < 0.10. 

Sources: U.S. Department of Housing and Urban Development, 2005 and 2014 active properties, active financing, active contracts, and multifamily building type files, Integrated Real Estate Management System; terminated contracts database; terminated multifamily mortgages database; 2005  Picture of Subsidized Households ; 2005 and 2014 FMRs; U.S. Census Bureau, 2005–2009 American Community Survey; Federal Housing Finance Agency Housing Price Index Cityscape 77

Ray, Kim, Nguyen, Choi, McElwain, and Stater

times those of the 100-to-120 percent of FMR reference case. In the original study, odds of opt-out among properties with rent-to-FMR below 80 percent were nearly 12 times higher than the reference case. Second, properties owned by profit motivated companies continued to be at higher risk. 

Odds of opt-out for for-profit/limited dividend owned properties from 2005 to 2014 were double those for nonprofit owned properties (compared to a factor of six in the original study). These results suggest that the opportunity for higher rents in the private market continued to play a role in pushing profit-motivated owners of properties with below-market rents to opt out of subsidies, but not to the same degree as in earlier years. 

For another key variable, the 2005-to-2014 results differed from the regression analysis in the original study. The original study showed that properties with rental assistance in all units were nearly 14 times more likely to opt out than those with partial Section 8 assistance. In the current analysis, however, fully assisted properties were seven to eight times less likely to opt out than partially assisted properties.

Financing characteristics also had significant effects that differed from the original study. Properties operating under HUD’s older assisted programs (see the Data and Methods section) were only one-half as likely to opt out as those funded under newer HUD programs. Properties financed by state housing finance agencies (HFAs) were more likely to opt out than other properties. In contrast, the original study showed that older assisted properties were more likely to opt out and showed no statistically significant effect from HFA financing (Finkel et al., 2006). A possible explanation for the current results is that that the newer assisted properties and the state-financed properties were more likely to be reaching their first opt-out decision during the 2005-to-2014 study period, whereas the older assisted and non-state-financed properties would have actively opted in to Section 8 assistance at least once before 2005. We speculate that owners of properties are more likely to take their first opportunity to opt out rather than to renew a contract initially and subsequently opt out. 

We added several variables to the model for the new study. Most of these related to neighborhood and regional market conditions. Opt-out risk was significantly higher for properties in strong neighborhood housing markets, signified by higher ratios of neighborhood (census tract) rents and housing values compared to the surrounding county. On the other hand, although a high 

homeownership rate might be expected to signal a strong neighborhood housing market, opt-outs were actually less likely in neighborhoods with higher homeownership rates. The lack of an active rental market in heavily owner-occupied neighborhoods may discourage market-rate conversion of properties. At the regional market level, properties in regions where home prices were moderately increasing from 2007 to 2012 were at higher risk of opt-out. Regional rental market changes, 

measured by changes in the HUD FMR over the same period, did not have a significant effect.3 

3 To account for the volatility in the overall housing market during the years under study, we also tested a model that segmented properties by the year of opt-in/opt-out decision into three periods: strong market (2005 to 2007), weak market (2008 to 2011), and recovering market (2012 to 2014). Surprisingly, results did not vary greatly across the three phases. 

Opt-outs were somewhat less likely among larger properties during the 2012-to-2014 recovering market phase, and during the weak market period the effect of family occupancy on opt-out risk was more pronounced. In general, however, results were consistent across the housing market time periods.

78 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Two other newly added variables showed unanticipated results. First, we expected that Section 8 

properties with a maturing 236/BMIR mortgage would be at higher risk of opt-out. The layer of affordability restrictions associated with the mortgage would be expiring. Absent other affordable financing layers, the owner would be free to convert the property to unrestricted rents upon opting out of the Section 8 contract. However, we found no significant effect from a 236/BMIR mortgage maturity. 

Second, the risk associated with REAC physical inspection scores was difficult to predict. Higher inspection scores might be expected to increase the risk of opt-outs, as owners seek to convert the properties in the best physical condition to market-rate use. Instead, the model shows a 1-point increase in REAC score resulted in a  decrease in odds of opt-out of approximately 1.6 percent. 

Owners of properties in poor physical condition may opt out of assistance preemptively if they expect HUD to abate a subsidy contract. Also, owners who are planning to opt out of subsidies and sell or convert properties to market-rate may delay investing in physical improvements until after the opt-out (Finkel et al., 2006).

In sum, the results confirm the original study’s emphasis on the risk of loss for smaller properties and units and for developments serving families. The analysis also substantiates the emphasis on market factors at the property and in the surrounding community. Properties in neighborhoods 

with higher rents and home values were at heightened risk of opt-out, as were those with profit-motivated owners and rents below the surrounding FMR. However, owner type and rent-FMR ratio 

demonstrated considerably less influence on opt-outs during the 2005-to-2014 study period than in earlier years.

Preservation of Opt-In Properties

The opt-in/opt-out analysis showed considerably fewer losses in the Section 8-assisted inventory between 2005 and 2014 compared to the original 1998-to-2004 study period. Many of the properties most at risk of loss, either weak properties in financial and physical distress or strong properties with potential to attract market-rate tenants, likely left the assisted inventory during the earlier wave of opt-outs documented by the 1998-to-2004 analysis. The middle-of-the-road inventory 

that remained from 2005 to 2014 was more stable, particularly as an increasing number of owners had already been required to make one or more active decisions to renew subsidy contracts. 

However, another reason that the 2005-to-2014 period may have generated fewer opt-outs is the maturing of the assisted housing preservation infrastructure. In the wake of well-publicized opt-outs in the 1990s, a variety of federal, state, local, and extra-governmental initiatives were put in place to preserve at-risk properties (HUD, 2013b).

•  HUD Mark-to-Market.  The Mark-to-Market program was put in place by the Multifamily Assisted Housing Reform and Affordability Act of 19974 to reduce above-market rents among 

HUD-financed properties with Section 8 assistance. The full restructuring option in Mark-to-

Market provides restructured, favorable mortgage terms to owners of Section 8 developments 

4 Pub. L. 105–65. 111 Stat. 1344, October 27, 1997.

Cityscape 79

Ray, Kim, Nguyen, Choi, McElwain, and Stater

in exchange for reducing rents to market values. It acts as a preservation program in that 

participating owners agree to long-term affordability, typically through a 20-year Section 8 

contract and a 30-year use agreement5 (HUD, 2015, 2002).

•  Low-Income Housing Tax Credit Program allocations for preservation.  LIHTC can be used to finance the acquisition and rehabilitation of at-risk Section 8 properties by preservation-minded owners. According to the National Housing Trust, 45 states provide incentives for 

preservation through allocation of competitive (9-percent) tax credits, including 16 states with explicit set-asides for preservation (National Housing Trust, n.d.). States also devote private activity bonds and noncompetitive (4-percent) credits to preservation projects. 

Adding tax credits to an existing Section 8 development provides resources to improve financial and physical conditions at aging properties. LIHTC also imposes at least 30 years of tenant 

income and rent restrictions, which reduces the incentive for owners to opt out of Section 8 

affordability provisions.

•  Federal Housing Administration (FHA)-insured mortgage refinancing.  A number of Section 8 

properties have undergone refinancing through market-rate FHA-insured funding programs. 

These funding sources do not carry income or rent restrictions, so their presence does not 

necessarily signal a long-term commitment to affordability. Nevertheless, many recipients of 

these mortgages commit to long-term Section 8 contract renewals.

•  Preservation funding from state HFAs.  Many HFAs use affordable housing trust funds, grants and loan programs to provide additional predevelopment and gap financing for preservation of 

federally assisted units.

•  Preservation databases, including risk-targeting data.  A number of states and cities have launched property databases or improved existing data tools to flag properties at risk of loss to the affordable inventory.6 

•  Additional state, local, and nongovernmental initiatives.  Beginning in 2001, the John D. and Catherine T. MacArthur Foundation’s Window of Opportunity initiative underwrote tremendous 

growth in the rental preservation infrastructure, including support for capacity building among national and local nonprofit developers, building sources of private capital for preservation, local and state interagency preservation councils, legal assistance and organizing support for tenants, and policy advocacy and research (MacArthur Foundation, 2009).

5 Mark-to-Market also offers a “Lite” option, which calls for reduction of rents without Federal Housing Administration mortgage restructuring. It is not included here as a preservation initiative because it only requires a 5-year renewal of Section 8 assistance and no long-term use agreement. 

6 See, for example, the Shimberg Center’s Florida Assisted Housing Inventory at http://flhousingdata.shimberg.ufl.edu/

AHI_introduction.html; CEDAC’s database for Massachusetts at https://cedac.org/housing/housing-preservation; and the 

NYU Furman Center’s Subsidized Housing Inventory Project at http://datasearch.furmancenter.org. The National Housing 

Preservation Database website includes a page of links to additional state and local preservation databases, at http://www.

preservationdatabase.org/preservation-resources/local-partner-databases.

80 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Preservation Transaction Types

In this analysis, we describe the presence of the first three financing tools—Mark-to-Market full restructuring, LIHTC allocations, and FHA mortgage refinancing—among the properties classified as Section 8 opt-ins in the 2005-to-2014 dataset. In addition, we explore the extent to which preservation resources are targeted toward at-risk properties. The potentially preserved properties are compared with other opt-in properties in terms of the risk factors identified in the multivariate analysis, such as family occupancy and rent-FMR ratios below 80 percent. 

To determine whether the opt-in properties received these types of financing, the dataset of opt-in properties was matched to financing information from the National Housing Preservation Database (NHPD) and HUD’s Mark-to-Market transactions database. Opt-in properties were considered 

to have a potential preservation transaction if the NHPD identified LIHTC as a funding source or HUD’s database indicated the properties completed a Mark-to-Market full restructuring. A 

property also was considered to have a potential preservation transaction if the NHPD indicated FHA-insured refinancing  and the property owner renewed the Section 8 contract for 19 to 20 years or longer during the 2005-to-2014 study period. We reasoned that the contract extension signaled a commitment to long-term affordability at the time the property was refinanced.7 

In most cases, the preservation transactions took place during the 2005-to-2014 study period. 

Transactions were also included if they predated 2005, because earlier subsidies still could 

incentivize or require owners to opt in to the Section 8 program during the study period. Specifically, for LIHTC, 62 percent of properties had transactions between 2005 and 2014. Most of the rest received tax credits from 1997 to 2003. For HUD refinancing, 87 percent of properties had transactions between 2005 and 2014, with the remainder occurring from 1998 to 2004. For Mark-to-Market, 67 percent of properties closed on restructuring between 2005 and 2014, with the 

remainder occurring between 1999 and 2004.8

As a convenient shorthand, properties with at least one of these interventions are referred to as 

“preserved” in the following discussion, and the opt-ins without any of these interventions are referred to as “nonpreserved.” In fact,  preservation has no standard definition. Some of the preserved properties would have been unlikely to leave the inventory even without these additional interventions, and some of the nonpreserved properties may have undergone preservation interventions 

other than the three types tracked here. In particular, we are unable to track the use of grants and loans from state HFAs and local funders to preserve properties, because no single database tracks these funding sources for the national Section 8 inventory.

7 Specifically, properties were included if their financing included one or more of these HUD programs: Section 207/223(f) and Section 223(a)(7), which provide mortgage insurance for purchase or refinancing of existing multifamily housing, and Section 542, under which HUD provides mortgage insurance in a risk-sharing agreement with state HFAs that lend to affordable housing projects. Section 8 contract renewal terms were measured in months from the date of renewal to the date of expiration. A small number of contracts had terms of 228 to 239 months; that is, 20 years minus a few months. These were included in the top category to account for delays in the contract renewal process that might slightly shorten the term.

8 Note that the preservation analysis includes only properties with opt-ins during the 2005-to-2014 study period. It does not include previously preserved properties where the owner did not make an active opt-in choice between 2005 and 2014. For example, properties would not be included if they completed Mark-to-Market restructuring and signed a 20-year Section 8 contract prior to 2005.

Cityscape 81

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Prevalence of Preservation Transactions

Use of the preservation tools was widespread among opt-in properties. In all, 3,561 properties with 328,394 units underwent at least one potential preservation transaction. This amounts to 28 

percent of opt-in properties and 34 percent of units.

Exhibit 7 shows the distribution of opt-in properties across combinations of preservation interventions. It also shows the length of the term for the last Section 8 contract renewal executed during the 2005-to-2014 study period.

Among LIHTC-funded developments, most were funded using 4-percent credits, alone (830 prop-

erties/53 percent) or in combination with 9-percent credits (438 properties/28 percent). 9-percent credits alone were used only 19 percent of the time (285 properties).9  

A common use of the HUD tool was the refinancing of direct loans from HUD’s Section 202 

program. From 1959 to 1990, the 202 program provided 40- to 50-year low-interest loans to 

nonprofit organizations for construction, rehabilitation, and acquisition costs for housing for elderly residents and persons with disabilities. HUD provides owners of Section 202 developments with the option to prepay and refinance 202 loans to reduce interest rate and debt service and to make capital improvements (HUD, 2013a). Nearly half (44 percent) of HUD-financed preservation properties in the opt-in dataset had inactive Section 202 loans. The preservation and stability of Section 202 developments are explored in more depth in Ray et al. (2015). 

Exhibit 7

Opt-In Properties by Preservation Transactions and Contract Renewal Length

Section 8 Renewal Term (%)

Preservation Indicator

Total Properties

1–5 Years

5–18 Years

19 Years or More

LIHTC only

1,559

33

7

61

HUD refinancing only

1,045

NA

NA

100

Mark-to-Market only

240

15

27

59

LIHTC with HUD

279

NA

NA

100

LIHTC with Mark-to-Market

86

9

16

74

Mark-to-Market with HUD

282

NA

NA

100

All three programs

70

NA

NA

100

All preserved

3,561

16

5

79

All nonpreserved

9,111

62

6

32

HUD = U.S. Department of Housing and Urban Development. LIHTC = Low-Income Housing Tax Credit Program. NA = not applicable.

Notes: Renewal term refers to last contract renewal executed during the 2005–2014 study period. HUD refinancing was used as a preservation indicator only if the property also had a contract renewal of at least 19–20 years, so all HUD-financed preserved properties are in the 19 Years or More category by definition. Also, Mark-to-Market properties are required to remain affordable for 30 years, so shorter-term contracts are expected to be renewed.

Sources: HUD, 2005 and 2014 active properties, active financing, and active contracts files, Integrated Real Estate Management System; National Housing Preservation Database

9 Percentages refer to developments for which credit type was available in the NHPD (1,556 of 1,994 developments).

82 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

Length of Affordability

The preservation tools appear to be effective in ensuring long-term affordability. Most of the preserved properties were operating under rental assistance contracts with terms of 19 to 20 years or more by the end of the 2005-to-2014 period (exhibit 7). All HUD-refinanced preservation 

properties were operating under contracts of at least 20 years by our own preservation definition, but the long-term contracts were also prevalent among LIHTC-only preservation properties (68 

percent), Mark-to-Market-only properties (59 percent), and LIHTC/Mark-to-Market properties (74 

percent). Moreover, LIHTC-funded properties are further protected by tenant income and rent restrictions extending at least 30 years from the date of tax credit funding, and longer in some states. 

Three-fourths of LIHTC-preserved opt-in properties in the dataset had income and rent restrictions extending until 2030 or later, including 32 percent with restrictions extending beyond 2040.

Nonpreserved properties were far more likely to be operating under short-term contracts. Of nonpreserved properties, 62 percent were operating under contracts renewed for 5 years or less at the end of the 2005-to-2014 study period. Although many owners do continue to renew short-term 

Section 8 contracts when they expire, the short-term contracts leave the opt-out choice open at every renewal point. These properties will require special attention and expansion of preservation initiatives to ensure that they continue to remain affordable.

Preservation Status and Opt-Out Risk Factors

Although preservation transactions offered widespread and long-term affordability protections, the record of targeting these protections toward properties most at risk of opt-out was mixed. Exhibit 8 

shows the prevalence of the key opt-out risk factors among properties with each type of preservation transaction, compared with the nonpreserved opt-in properties.  

Preservation efforts using LIHTC and Mark-to-Market do appear to be more targeted toward 

properties with two key opt-out risk factors: for-profit/limited dividend ownership and family occupancy. First, most properties with LIHTC funding and, in particular, Mark-to-Mark restructuring were owned by for-profit or limited dividend corporations. In contrast, most nonpreserved properties were owned by nonprofits. Note that this risk factor tracks the ownership type at the 2005 

baseline; some of the preserved properties may have been transferred to nonprofit organizations subsequently as part of the preservation transaction. Second, most LIHTC and Mark-to-Market 

properties were designated for family occupancy, compared to a minority of nonpreserved properties. 

In contrast, preserved developments with HUD-insured mortgages were less likely than nonpre-

served properties to be owned by for-profits or to have family occupancy. This is linked to the heavy use of HUD refinancing to preserve properties with Section 202 loans. The 202 program 

requires elderly or disabled occupancy type and nonprofit ownership.

The other risk factors generally were equally or even less prevalent among the preserved properties compared to other opt-ins. Two risk factors stand out. First, small properties are at heightened risk of opt-out, but preservation resources were disproportionately found in larger properties. Half of the nonpreserved properties had fewer than 50 units, but only about one-fourth of the preserved properties did. Second, rent-FMR ratios below 80 percent were not more prevalent among 

Cityscape 83

Ray, Kim, Nguyen, Choi, McElwain, and Stater

Exhibit 8

Presence of Risk Factors Among Opt-In Properties by Preservation Type

Properties in Preservation Category With Risk Factor (%)

LIHTC 

Risk Factor

HUD  

Mark-to- LIHTC 

Mark-to-

LIHTC 

With 

All Three  All Non-

Refinancing  Market  With 

Market 

Only

Mark-to-

Programs preserved

Only

Only

HUD

With HUD

Market

Rent-to-FMR ratio 

11

3

5

6

8

2

4

11

< 80% (percent 

of properties)

For-profit/limited 

62

24

87

56

93

88

91

36

dividend 

ownership 

(percent of 

properties)

Family occupancy 

54

15

68

39

70

63

66

38

(percent of 

properties)

1–49 units (percent 

29

28

24

19

19

29

21

51

of properties)

Share of 0- to 

85

96

81

88

79

80

80

90

2-bedroom units 

(average for 

properties)

Partially assisted 

16

7

6

13

14

5

11

14

(percent of 

properties)

Neighborhood 

23

20

30

22

25

26

27

21

poverty rate 

(average for 

properties; lower 

is risk factor)

Neighborhood 

49

53

46

49

51

50

52

53

homeownership 

rate (average for 

properties; lower 

is risk factor)

Neighborhood 

89

92

86

90

87

91

84

92

rent/county rent 

ratio (average 

for properties; 

higher is risk 

factor)

FMR = Fair Market Rent. HUD = U.S. Department of Urban Housing and Development. LIHTC = Low-Income Housing Tax Credit Program.

Sources: U.S. Department of Housing and Urban Development, 2005 and 2014 active properties, active financing, active contracts, and multifamily building type files, Integrated Real Estate Management System; terminated contracts database; terminated multifamily mortgages database; U.S. Census Bureau, 2005–2009 American Community Survey; National Housing Preservation Database

84 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

LIHTC-preserved properties than nonpreserved properties, and they were nearly absent among the HUD-financed and Mark-to-Market properties.10 That is, it does not appear that LIHTC and other preservation resources were particularly targeted toward properties whose low rents compared to the surrounding market may encourage owners to convert to market rate.

Similarly, preservation resources do not appear to have been targeted toward neighborhoods with demographic and market conditions that raise opt-out risk. Averages for neighborhood indicators such as poverty rate, homeownership rate, and the ratio of neighborhood to county median rents were similar between preserved and nonpreserved properties. In fact, compared to the nonpreserved properties, Mark-to-Market properties were in neighborhoods with higher poverty rates and lower neighborhood rent-county rent ratios—both factors that point away from opt-out risk.

To summarize, a substantial number of opt-in properties have received preservation assistance. The preservation interventions are working as intended by encouraging long affordability periods, in contrast to the short contract renewal periods common among unpreserved properties. Preservation interventions have been effectively targeted toward properties with two well-understood risk factors, for-profit ownership and family occupancy. Other identified risk factors such as small property size, low rent-FMR ratios, and strong neighborhood market characteristics are not more prevalent among preserved properties. Future preservation initiatives can further reduce the risk of opt-outs by more careful targeting of developments with these characteristics. 

Conclusion

The comparison of property outcomes between Finkel et al. (2006) and this update shows how 

HUD’s multifamily portfolio has shifted in the last two decades. The Section 8-assisted inventory demonstrated more continuity between 2005 and 2014 than in the original 1998-to-2004 study 

period, even as HUD’s original 221(d)(3) and 236 subsidized mortgage programs were largely 

phasing out. Fewer properties underwent opt-out, and far fewer were subject to foreclosure and contract abatement. At the same time, more owners actively opted to continue participation in the Section 8 program. 

The 2005-to-2014 analysis shows that to the extent Section 8 opt-outs continued to occur, many properties were subject to similar risk factors to those identified in the original study, including family occupancy, for-profit ownership, low rent-FMR ratios, and location in less distressed neighborhoods. Although these factors were present in the second study phase, several were less influential. The descriptive cross-tabulations showed more variability in these characteristics among the properties lost to the affordable inventory, and the regression analysis showed that together these characteristics explained less variation in the opt-in/opt-out decision than before. For example, of the 748 opt-outs, over one-third (271, or 36 percent) was owned by nonprofit organizations, served elderly or disabled tenants, or both.

10 The lack of Mark-to-Market properties with rents below 80 percent of FMR is to be expected. Mark-to-Market is targeted toward properties with  above-market rents. For properties that had not completed the Mark-to-Market process at the time of contract renewal during the 2005-to-2014 study period, the rent-FMR ratio would reflect the above-market rents. For those that had completed restructuring, it still is unlikely that rents would be reduced as far as a level below 80 percent of FMR.

Cityscape 85

Ray, Kim, Nguyen, Choi, McElwain, and Stater

The analysis shows that a significant minority of the remaining Section 8 properties operate under conditions that protect them from opt-out risk. Thousands of properties operate under extended affordability periods associated with preservation financing tools. In particular, preservation with LIHTC has extended affordability periods for many Section 8 developments well into the future. 

Most of the preserved properties are owned by for-profit corporations and serving families, two characteristics that signaled higher opt-out risk in both the original study and the 2005-to-2014 

update. 

At the same time, the preservation analysis points up risk to properties without these additional interventions. The majority of Section 8 contracts in nonpreserved developments were renewed for terms of 5 years or less. The preservation tools do not seem to be targeted toward small developments in strong neighborhood and regional housing markets—exactly the types of properties that might be at risk of market-rate conversion as rental markets tighten and neighborhoods revitalize or gentrify. Preserving these developments may require efforts that bundle several small properties into single transactions or new financing tools that can be scaled for small developments. 

We suggest several additional areas of research to improve our understanding of risks and preservation in the assisted housing inventory. First, we recommend detailed, year-over-year analysis of contract renewal histories to determine the extent to which the proliferation of short-term renewals signals future risk to the inventory. Do owners who renew a contract for 1 to 5 years tend to renew these contracts again upon expiration, or do short-term renewals signal an impending opt-out? 

Constructing full opt-in histories will require annual Section 8 contract datasets. The two pointin-time datasets available for this study provided a partial picture of renewals, but information was not available about short-term renewals in the intervening years between 2005 and 2014.

Second, given the weakened influence of the traditional opt-out risk factors, we recommend 

further examination of opt-outs in developments without these risk factors. These include 

developments serving elderly residents or persons with disabilities, nonprofit-owned properties, and developments that do not appear to be especially vulnerable to market-rate conversion (for example, those in distressed neighborhoods or whose contract rents are in line with or higher than the surrounding market rate). Case studies could help us understand the factors that lead to non-traditional opt-outs, such as changes in nonprofits’ interest and ability to maintain aging subsidized properties. 

Third, we recommend more indepth analysis of state policies on the use of LIHTC to preserve the HUD-assisted inventory. As noted previously, nearly all states provide some type of set-aside or other incentive to promote use of tax credits for preservation. These policies are present in states’ 

Qualified Allocation Plans (QAPs) and other public documentation of scoring for LIHTC competitive allocations. Do states with robust preservation incentives in their QAPs report more opt-ins, potential preservation actions, and 20-year Section 8 contract renewals compared to states with weaker incentives? What types of incentives are most effective in promoting preservation?

Finally, the changes between Finkel et al. (2006) and this update demonstrate the value of continuing to update property characteristics and opt-in/opt-out histories. Conditions in the assisted housing inventory are not static. Ownership changes. Properties may age and deteriorate, or they may be rehabilitated. Length of remaining affordability changes over time; even 20-year contracts 86 Selected Outcomes of Housing Assistance

Opting In, Opting Out: A Decade Later

put into place in the early part of the 2005-to-2014 study period are already more than halfway to their expiration date. Neighborhood and regional housing markets also are not static. With rental markets tightening and affordable housing in short supply, properties in neighborhoods where 

opt-outs would have seemed unlikely in the past may become ripe for market-rate conversion. Accurate and up-to-date property information will be critical to continue the preservation efforts that have successfully maintained the assisted housing inventory.

Acknowledgments

The authors express their appreciation to the team that produced the original  Opting In, Opting Out report. The authors also thank Nancy Augustine, Meena Bavan, Danilo Pelletiere, Ali Sayer, Barry Steffen, and Lydia Taghavi for their input and assistance with the Multi-Disciplinary Research Team’s  Opting In, Opting Out a Decade Later report.

Authors

Anne Ray is Manager of the Florida Housing Data Clearinghouse at the University of Florida, 

Shimberg Center for Housing Studies. 

Jeongseob Kim is an assistant professor at Ulsan National Institute of Science and Technology.

Diep Nguyen is a database manager at the University of Florida, Shimberg Center for Housing 

Studies. 

Jongwon Choi is a doctoral student in the Department of Urban and Regional Planning at the 

University of Florida. 

Kelly McElwain is a research analyst III at the Public and Affordable Housing Research Corporation and is the database manager of the National Housing Preservation Database.

Keely Jones Stater is Director of Research and Industry Intelligence at the Public and Affordable Housing Research Corporation.

References

Finkel, Meryl, Charles Hanson, Richard Hilton, Ken Lam, and Melissa Vandawalker. 2006.  Multifamily Properties: Opting In, Opting Out and Remaining Affordable. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. huduser.org/

Publications/pdf/opting_in.pdf.

John D. and Catherine T. MacArthur Foundation (MacArthur Foundation). 2009. “Window of Op-

portunity: Preserving Affordable Rental Housing Investment Summary,”  Evidence Matters. https://

www.macfound.org/press/publications/window-opportunity-state-and-local-housing-preservation-

project/.

Cityscape 87

Ray, Kim, Nguyen, Choi, McElwain, and Stater

National Housing Trust. n.d. “National Trends (Incentives for Preservation in State Qualified Allocation Plans and Set-Asides for Preservation in State Qualified Allocation Plans).” http://www.

prezcat.org/related-catalog-content/incentives-preservation-state-qualified-allocation-plans-2014.

Ray, Anne, Jeongseob Kim, Diep Nguyen, and Jongwon Choi. 2015.  Opting In, Opting Out a Decade Later. Multi-Disciplinary Research Team. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

U.S. Department of Housing and Urban Development (HUD). 2016. “Picture of Subsidized House-

holds.” huduser.gov/portal/data sets/assthsg.html. 

———. 2015. “M2M Transactions Report.” http://portal.hud.gov/hudportal/documents/

huddoc?id=m2mstran.xls. 

———. 2013a.  Updated Requirements for Prepayment and Refinance of Section 202 Direct Loans. HUD 

Notice H-2013-17. Washington, DC: U.S. Department of Housing and Urban Development. http://

portal.hud.gov/hudportal/documents/huddoc?id=13-17hsgn.pdf.

———. 2013b. “Preserving Affordable Rental Housing: A Snapshot of Growing Need, Current 

Threats, and Innovative Solutions,”  Evidence Matters. huduser.org/portal/periodicals/em/

EM_Newsletter_Summer_2013_FNL.pdf.

———. 2002. “Frequently Asked Questions on Mark-to-Market for Property Owners and Manag-

ers.” https://portal.hud.gov/hudportal/documents/huddoc?id=DOC_19531.pdf.

88 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing 

in the United States

Sandra Newman

Scott Holupka

Johns Hopkins University

Abstract

This article uses the 2011 American Housing Survey to develop three indices of housing 

quality, test their validity, apply them to both the assisted and unassisted stock, and assess whether the Fair Market Rent (FMR) aligns with good assisted housing quality. The 

market value index, developed using hedonic regression, performs poorly and is dropped 

from further consideration. The consumer rating index, based on an ordered logistic re-

gression of the respondent house rating on a 1-to-10 scale, and the normative standards 

index, based on a factor analysis, perform well, are highly correlated, and achieve con-

vergent and predictive validity. Both of these indices indicate that the quality of assisted housing is comparable to that of unassisted housing. The analysis also supports the 

40th percentile of rents definition of the FMR, which is roughly the inflection point for maximizing assisted housing quality on both housing quality indices tested. The findings 

demonstrate that the current inspection and quality control systems appear to be achiev-

ing the goal of providing physically adequate housing to assisted housing residents. 

Introduction

The U.S. Department of Housing and Urban Development (HUD) plays a key role in designing, 

implementing, and monitoring most of the nation’s assisted housing programs, including public housing, privately owned, publicly subsidized housing (commonly referred to as “multifamily”), and vouchers.1 Central to this responsibility is ensuring that the units receiving HUD assistance are physically adequate. This, in turn, verifies that recipient households live in decent and safe dwellings and reassures the public that tax dollars are not supporting deficient housing, or worse. 

1 This article is based heavily on Newman and Holupka (2017), which contains greater detail along with additional analyses, tables, and appendices. 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 89

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Newman and Holupka

To accomplish this objective, HUD imposes a set of housing quality standards (HQS) that assisted units must meet, requires periodic inspections to confirm that standards are being met, and when necessary, issues citations of violations that must be corrected within a specified time frame.2

Several recent circumstances prompt a reexamination of assisted HQS. First, HUD’s  Strategic Plan 2014–2018 calls for the development of a “uniform asset risk assessment management model,” 

which requires systematic evidence on the most meaningful approaches to measuring the quality of the assisted housing stock (HUD, 2014: 19). Second, the fiscal year 2013 Senate Appropriation Committee Report raises concerns about violations of HQS in housing units participating in the Section 8 voucher program and “directs HUD to take meaningful and timely steps to strengthen 

oversight and quality control” of the public housing agency (PHA) inspection process (U.S. Senate, 2012: 92). An additional concern is that reports by HUD’s Office of the Inspector General and the U.S. Government Accountability Office note poor reliability of assisted housing inspections using the HQS (HUD OIG, 2008; GAO, 2000). This conclusion is based on a comparison of the PHA 

inspector scores with those collected by an independently trained rater.

This article is designed to contribute to the reexamination of assisted housing quality. We develop composite measures or indices of housing quality, test their validity, and apply them to both assisted and unassisted housing to examine possible disparities in quality between these two housing stocks. We also examine how well assisted housing quality aligns with HUD’s Fair Market Rent 

(FMR). A longstanding policy question is whether good housing quality in the assisted stock aligns with the FMR, now generally set at the 40th percentile of rents in each housing market. One objective in setting the FMR at a particular point in the distribution of rents is the household’s ability to find physically decent rental housing at or below the FMR threshold.

The analysis relies on rich data from the 2011 American Housing Survey (AHS) to describe the 

quality of the assisted housing stock and to highlight geographic areas, types of households, housing types, and housing assistance programs most likely to experience quality problems. The 2011 AHS sample was matched to administrative records on assisted housing receipt, alleviating concerns about the validity of self-reported housing assistance receipt. 

It is important to acknowledge at the outset that the concept of housing quality is not based on definitive criteria and has no precise quantifiable definition of where “bad” ends and “good” begins (Newman, 2008). As we subsequently explain more fully, because we lack a consensus definition of housing quality, we construct measures that characterize the dwelling’s physical integrity (for example, holes in the floor) or housing systems (for example, heating system breakdown) and 

exclude those that are more likely to reflect the resident’s housekeeping or behavior (for example, leaving unsafe chemicals within a child’s reach).

We find that the quality of assisted housing is comparable to the quality of unassisted housing. 

Multivariate models reveal modest heterogeneity in assisted housing quality, with the Northeast region and households that include a disabled member experiencing lower housing quality than 

2 Inspection protocols and processes differ by program. Public housing and multifamily housing—for example, Section 8 

new construction; Section 221(d)(3)—must meet property standards, while Section 8 vouchers must meet HQS. Inspectors employed by the local public housing agency conduct inspections on public housing and voucher housing, while inspectors contracted by the HUD regional offices inspect multifamily housing.

90 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

average. The analysis also provides hard evidence supporting HUD’s definition of the FMR as 

the 40th percentile of rents. The FMR is set at a level that is roughly at the inflection point for maximizing assisted housing quality as measured by the housing quality indices developed in this article.

The next section reviews the literature on housing quality, emphasizing past research on  assisted housing quality. This is followed by a discussion of the research approach, including a description of the AHS data, analysis samples, methods, alternative measures of housing quality, and tests of their validity. We then examine assisted housing quality compared with unassisted housing quality, whether assisted housing quality varies by where it is located, such as in a central city or a suburb, or the characteristics of the residents. We also explore how well assisted housing quality aligns with the FMR. The final section discusses the results and their implications for policy and future research.

Literature Review

The substantial literature on housing quality spans nine decades and demonstrates both the 

importance and the challenges of conceptualizing and measuring housing quality. Three relevant strands characterize past work: housing quality measurement and data collection methods, the 

AHS measurement of housing quality, and the quality of assisted housing.

Housing Quality Measurement and Data Collection Methods 

The American Public Health Association provided some of the earliest contributions to the housing quality literature. APHA (1938) highlighted the connection between housing conditions and health, and APHA (1945) recommended that data be collected through a field survey of many 

individual features of each dwelling unit, with penalty scores for each feature that falls below an established standard. The sum of all scores represents the quality of the dwelling. This methodology is roughly similar to that used for physical inspections of assisted housing under HUD’s 

Uniform Physical Condition Standards (UPCS).3 

Another important early contribution was the U.S. Census Bureau’s methodological study of housing quality measurement (Census Bureau, 1967).4 For decades, the decennial census included interviewer observations of housing features. In 1940, housing condition was measured by a dwelling’s 

“state of repair,” with trained enumerators rating the structure as either needing “major” repairs or not. In 1950, this approach was replaced by another dichotomous classification of structures as either 

“dilapidated” or “not dilapidated.” This dichotomy was refined in 1960 by further classifying those structures designated as “not dilapidated” as either “sound” or “deteriorating.” Following the 1960 

census, the Census Bureau launched a detailed and thorough evaluation of its approach to measuring housing conditions that resulted in the 1967 publication. Its unambiguous conclusion was that 3 UPCS currently exists for public housing and for multifamily housing. A version for the voucher program, UPCS-V, is under development and will replace the current HQS system (Cota, 2017).

4 This discussion draws on Newman (2008).

Cityscape 91

Newman and Holupka

housing conditions collected through interviewer observations are unreliable and, therefore, inaccurate. As a result, subsequent decennial censuses that relied on interviewers to administer the survey dropped the interviewer observations of housing unit condition. The AHS followed suit in 1997.5 

Alongside concerns in the literature about the best way to collect data on housing quality is the issue of the best way to measure housing quality. Curiously, much more attention has been paid to developing a summary measure than to identifying the individual housing features that should comprise the summary measure. The pioneering work of Kain and Quigley (1970) established the 

feasibility of using housing unit measures of quality as predictors of house prices and rents, also known as hedonic models. The coefficients in these models can be viewed as weights in a hedonic price index. Kain and Quigley’s work led to a burgeoning of hedonic modeling over the ensuing decades (for example, Coulson and Li, 2013; Kriström, 2008; Merrill, 1980; Thibodeau, 1995). 

Three features of Kain and Quigley’s approach are particularly relevant to the current article. First, they apply factor analysis to reduce the 39 separate measures of housing quality in their St. Louis survey data to a manageable number. Second, they find that the seven survey measures pertaining to the quality of the individual dwelling unit interior formed a single index or factor.6 Third, in multivariate hedonic regressions, the dwelling unit quality factor has a statistically significant effect on rent. Consistent with most of the literature in this area, the authors do not take on the question of how best to conceptualize housing quality and, instead, assume that this concept is captured by their 39 variables pertaining to “the physical or visual quality of the bundle of residential services” 

(Kain and Quigley, 1970: 534).

AHS Measurement of Housing Quality

The AHS is the most comprehensive data source on the U.S. housing stock.7 Policymakers, 

practitioners, and researchers seeking answers to questions about the conditions, costs, and various other attributes of the nation’s housing rely on it. It is also relied on as a source of housing questions for those developing their own surveys. A prominent example is the Moving to Opportunity for Fair Housing Demonstration (Shroder, 2001). Of particular interest to many users is the AHS 

composite measure of housing inadequacy available on the public use database, which is a variable labeled ZADEQ. The measure combines multiple items on housing conditions into an index, 

setting numerical thresholds for the presence or absence of physical deficiencies in the dwelling to distinguish among “adequate,” “moderately inadequate,” and “severely inadequate” units. Both the AHS and data users refer to this composite as the “AHS housing quality measure.” Numerous published articles include the AHS measure in their analyses (for example, Carter, 2011; Friedman and Rosenbaum, 2004; Khadduri, 2007; Ross, Shlay, and Picon, 2012). It plays a prominent role in HUD’s  Worst Case Housing Needs reports (for example, HUD, 2015) and is also included in the frequently cited Joint Center for Housing Studies’  The State of the Nation’s Housing reports (for example, JCHS, 2017) and Millennial Housing Commission (2002). However, not until the last few years was the AHS quality measure subjected to careful examination.

5 The AHS began interviewing returning households by phone, when possible, in 1997. In 2011, a phone-first policy was instituted for both new and returning households (Vandenbroucke, 2016).

6 Their survey included many other items focusing on the condition of adjacent structures, parcels, and block faces, along with the structure’s exterior condition.

7 Drawn in part from Newman and Garboden (2013). 

92 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

Eggers and Moumen’s (2013a) analysis of the 2005, 2007, and 2009 AHS data implicitly raises 

some concerns about whether the ZADEQ measure accurately reflects the quality of the housing 

stock. The measure produces a very low prevalence of severe inadequacy (2 percent); only two 

items—sharing a bathroom and heating problems—account for most of the cases considered 

severely inadequate, and these problems generally do not persist over a 2-year period. The authors also conclude that the shared bathroom item is likely to have been measured incorrectly. More generally, they conclude that the AHS quality measure may provide a reasonable cross-sectional estimate of the most severely inadequate units, but provides little information on the roughly 91 

percent of units considered adequate.8 

In a second paper, Eggers and Moumen (2013b) proposed an alternative to ZADEQ that is designed to provide more information about gradations within the adequate housing stock. A major motivation is their particular interest in being able to study filtering, which requires a measure that reveals increases in deficiencies or inadequacies over time as a unit deteriorates and is presumably filtered down from higher income to lower income residents. They develop an alternative measure, the poor quality index (PQI), which is a numeric scale of housing defects that draws on additional measures in the AHS (for example, exterior structure) along with those included in ZADEQ. Lacking a reliable source on how to weight each item in the index, they assign weights based on a combination of ZADEQ’s definitions and their own judgment. The PQI appears to achieve the goals of its creators. 

By contrast to ZADEQ, which estimates that a large majority of units had no problems, 47 percent of units had at least one PQI inadequacy.9 The stability of the classification of the unit also differs for the two indices (Eggers and Moumen, 2013b). With ZADEQ, 95 percent of adequate units in 

one survey remain adequate 2 years later, whereas roughly 30 to 35 percent of units categorized as moderately or severely inadequate in one survey remained inadequate in the subsequent survey. 

Using the PQI, a smaller share, 63 percent of units, remained adequate from one survey to the next, and a greater share of inadequate units, roughly 60 percent, retained that designation over 2 years.

Emrath and Taylor (2012) examined the AHS ZADEQ index using a hedonic model. Because of 

the multicollinearity among the individual measures that comprise ZADEQ, the authors test each ZADEQ item separately, along with other features of the dwelling (for example, number of rooms, geographic region, and square footage). They report that none of the ZADEQ items reach statistical significance and, in some cases, have an unexpected sign. A major policy concern of the authors is that the very small rate of housing units meeting the definition of physically inadequate using ZADEQ leads to the conclusion that the nation’s housing stock has no serious housing problems. 

They challenge this conclusion by identifying measures in the AHS, many of which are not 

included in ZADEQ, that have a strong effect on rents and prices. These items are similar to those included in Eggers and Moumen’s (2013b) PQI. It is likely that Emrath and Taylor’s ZADEQ results occur because of the very low variance of each individual item. This was part of Kain and Quigley’s (1970) motivation for using factor analysis, which produced a single dwelling unit quality factor.10  

8 Authors’ estimate based on the 2011 AHS.

9 The PQI rate is based on the 1993 AHS. Because the two Eggers and Moumen reports (2013a; 2013b) rely on different AHS years, it is impossible to make direct comparisons between ZADEQ and PQI results.

10 Merrill (1980) applied a somewhat similar approach in her hedonic modeling using data from the Experimental Housing Allowance Program demand experiment.  

Cityscape 93

Newman and Holupka

A fourth recent paper assesses the reliability, consistency, and validity of the AHS ZADEQ index (Newman and Garboden, 2013). Like Eggers and Moumen (2013a; 2013b) and Emrath and 

Taylor (2012), the authors conclude that the index identifies only a very small share of units with multiple inadequacies and provides little information about variations among units classified as adequate. They also find that the items included in the index do not appear to be tapping the same underlying construct of housing quality. However, the two subindices within ZADEQ, moderate 

inadequacy and severe inadequacy, are strong and statistically significant predictors of residents’ 

housing satisfaction. 

Quality of Assisted Housing

The research literature on the physical quality of the assisted housing stock is sparse, at least in part, because the AHS, the main data source on housing, typically relies on respondent self-reports of the receipt of housing assistance, which are known to be unreliable (Shroder, 2002). The present article utilizes the 2011 AHS data, which identifies assisted housing receipt, by program type, on the basis of a match to administrative records, not self-reports. Validation of assisted housing receipt was previously done in the 1989 AHS.11 One paper used these validated data and a version of ZADEQ to study the assisted housing profiles of households with children (Newman and Schnare, 1993). The authors report that 15 percent of public housing units occupied by households with children had either a moderate or severe defect, compared with 5 percent of multifamily housing and 12 percent of voucher units. The average number of defects, however, was generally similar across the programs.

A more recent study examined the quality of housing in the voucher program (Buron et al., 2003). 

Data on voucher housing come from the 2000 Customer Satisfaction Survey (CSA). The authors 

developed two measures of housing quality, one relying on all quality-related items in the CSA and another using CSA items that align with those in the AHS. The CSA-based measure was used to 

explore voucher-housing quality, and the CSA-AHS measure was used to compare housing quality 

in the voucher program with housing quality in a matched comparison sample of unassisted 

renters. The CSA-based summary measure combined items into four categories: (1) severely inadequate quality, (2) moderately inadequate quality, (3) adequate quality, and (4) high quality. Based on voucher respondent reports to the CSA, 41 percent of voucher housing was considered high 

quality, 33 percent adequate quality, 4 percent moderately inadequate, and 23 percent severely inadequate (numbers rounded). The rate of severe inadequacy is higher than the 12- to 21-percent range in Gray, Haley, and Mast (2008), HUD’s report on the first-year results of the CSA, which relied on similar though not identical quality measures. Buron et al. (2003) based their analysis of voucher and comparable nonvoucher housing quality on a statistical match between the households in the CSA voucher sample and households in the AHS. They use two different measures 

of housing quality, one a simple count of problems aggregated into four categories (0, 1–2, 3–4, and 5+ problems) and another indicating whether at least one problem was reported for each of five housing dimensions (for example, kitchen and bathrooms; electrical). Both measures yield the similar finding of lower-quality housing of voucher users than housing occupied by unassisted 11 Documentation on this validation can be found in Newman and Schnare (1993).

94 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

renters. For example, 59 percent of voucher renters reported no housing problems compared with 66 percent of unassisted renters. The authors cautioned that differences between the CSA and AHS 

may account for some or most of these disparities.

Research Approach

In the next section, we describe the AHS data we use in the analysis, the analysis samples and the different facets of our methodological approach. We also review the three main alternative measures of housing quality that are the focus of our analysis. 

Data

The main data source for the current analysis is the 2011 AHS. The AHS began in 1973 and is sponsored by HUD and conducted by the Census Bureau. As previously noted, the 2011 assisted housing cases are identified based on matching sample addresses to HUD administrative data on HUD-assisted housing programs.12 The sample includes 9,721 assisted housing units13 and 40,030 unassisted rental housing units in single-family or multifamily properties, the housing types that dominate the assisted stock.14 Because we will ultimately apply the quality indices to the assisted stock, we rely on the unassisted sample to develop the housing quality indices. These indices are based on 33 housing quality items that are collected from both single-family and multifamily rental units.

Methods

Construction of the Comparison Groups

We compare the quality of assisted housing to two comparison groups of unassisted housing, 

one including all rentals and the other limited to units with rents at or below the FMR. For both comparison groups, we limit cases to units in a single-family or multifamily property and exclude unassisted cases that are rare or nonexistent in the assisted stock (for example, manufactured housing; reduced rents because of relationship between renter and landlord). We also exclude vacant or vacation units and units where no interview was conducted.15

Housing Quality Indices

Because the concept of housing quality is not based on explicit criteria, the large number of housing quality indices that have been developed with the AHS yield dramatically different prevalence rates (Newman and Schnare, 1988). The core challenge is well known; a housing unit is a bundle of attributes that extend beyond the dwelling itself, and it is unclear which of these attributes 12 The match to HUD data excludes housing units assisted by state and local programs and the federal Low-Income Housing Tax Credit (LIHTC) program, which is under the auspices of the Department of the Treasury. However, because a sizable share of LIHTC units also receives HUD subsidies, such as vouchers, these units are included in the HUD administrative files. O’Regan and Horn (2012) estimated that 46 percent of LIHTC households receive some form of rental assistance, Buron et al. (2000) put the estimate at 37 percent, and GAO’s (1997) estimate is 39 percent.

13 Based on sample design appendix to 2011 AHS documentation (HUD, 2011).

14 See Newman and Holupka (2017), table A1 for all selection criteria for the unassisted sample.  

15 Supplementary analysis using propensity score matching to create comparison groups produce similar results.

Cityscape 95

Newman and Holupka

should be included in the definition of the dwelling’s quality and how each should be weighted in determining overall quality (Aaron, 1972; Merrill, 1980). In the absence of a consensus view, the next best option is to rely on an external criterion, as suggested by Merrill (1980). We examine three alternatives: market value, consumer rating, and normative standards. 

Market Value Index.  The market value approach assumes that the unit’s rent is correlated with the quantity and quality of housing such that higher rents reflect better quality. Consistent with the literature (for example, Coulson and Li, 2013; Kriström, 2008; Thibodeau, 1995), this theory can be tested with a hedonic regression where the dependent variable is the natural log of rent and covariates include characteristics of the housing bundle. In this article, covariates include multiple features of the housing unit, geographic location, the respondent’s rating of the neighborhood (the only neighborhood measure available in the 2011 AHS), and the FMR. Although our main interest is the contribution of housing quality to market value, this effect could depend on the nonhousing features included in the model, such as perceived neighborhood quality or location in a central city or suburb. Because of substantial multicollinearity among the 33 housing quality items, we estimate two hedonic models, one including all 33 items despite this collinearity problem, and the other testing each of the 33 items separately.

Consumer Rating Index.  The consumer rating criterion identifies the dwelling features that are most closely associated with the resident’s assessment of the dwelling as a good place to live, regardless of what the market price of these features might be. This criterion broadens the concept of housing quality beyond specific housing features to the welfare of residents as they themselves report it (Goodman, 1978). It is consistent with the renewed interest by economists in happiness and subjective well-being as a measure of the utility an individual derives from goods and services (Dolan, Peasgood, and White, 2008).

The AHS question asks the respondent: “On a scale of 1 to 10, how would you rate your unit as a place to live?” Although the original coding designates 10 as best and 1 as worst, we reverse these codes for consistency with the normative standards index (discussed next). Thus, a higher value on this ordinal scale indicates lower housing quality. We test the consumer rating model using ordered logistic regression, which generates coefficients expressed as odds ratios: how much a unit change in housing quality item X changes the house rating. As with the market value criterion, we test the consumer rating index using each individual quality item separately and all quality items combined. Because the results are similar, we only present the results from the separate quality measure tests.

Normative Standards Index.  The normative standards criterion is designed to reflect community concerns and policy decisions about housing quality, such as state building codes and assisted housing physical inspection standards. We use factor analysis to develop the normative standards index. Factor analysis examines the correlations among measures to determine the amount of 

common variance among them. The analysis produces factor “loadings,” which indicate how much 

variance is shared among the observed measures and the unobserved construct (here, housing 

quality). The loadings or scores constitute the weights that we use to create the factor analysis index. Because many of the quality measures are dichotomous, we estimate polychoric correlations 96 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

(Jöreskog and Sörbom, 1996). Following Preacher et al. (2013), we select the smallest number of factors for which the root mean square error of approximation (RMSEA) is below 0.05. This approach identifies the measures that most accurately reflect housing quality.

Assessing Index Validity.  We assess the convergent and predictive validity of the resulting housing quality indices (Carmines and Zeller, 1979). Convergent validity is based on the correlations among the indices and between each index and other attributes with which the index should be 

associated, such as the resident’s house rating or satisfaction. Predictive validity is based on the predictive power and significance of the indices in multivariate models predicting two outcomes, the resident’s house rating on a 0-to-10 scale and rent.

Assisted Housing Quality

The analysis of assisted housing quality proceeds in three steps. First, we begin by examining differences between the assisted and unassisted housing stock for each of the 33 individual housing quality measures included in the housing quality indices. This analysis also includes three additional measures from the 2011 AHS Healthy Homes modules. Second, we look at variations in quality within the assisted housing stock by program type, household type, and location. Because most of these analyses are based on the large sample sizes available in the AHS, measures of statistical significance are not very useful to gauge substantive importance. Therefore, we rely heavily on the size of the effect as measured by Cohen’s  d.16 In a final step, we estimate a series of multivariate models predicting each housing quality index controlling for housing, location, and household characteristics. The first set of models is limited to the assisted housing sample, and includes assisted housing program type as one of the explanatory variables. The second set of models includes both assisted and unassisted housing, initially testing assisted housing as a whole, and then distinguishing this stock by program type. Because the data are heavily skewed, and the distributions have considerable dispersion, we use negative binomial modeling as the estimation technique.

Assisted Housing Quality and FMRs

To explore the alignment between the FMR and assisted housing quality, we calculate each household’s housing cost relative to the FMR (that is, gross rent divided by the FMR). We then divide this relative housing cost scale into equal units (for example, 40 to 45 percent of the FMR, 45 to 50 percent of the FMR, and so on) so that the FMR, the 40th percentile, sits in the middle of the distribution.

Results

Exhibit 1 lists the AHS housing quality measures in this analysis and their means. Consistent with much past AHS housing quality research, the prevalence rates of almost all problems are very low. 

Most (55 percent) dwellings have no problems, and fewer than 5 percent of units account for more than 75 percent of problems.

16 Cohen’s  d is the difference in means between two groups divided by the standard deviation for the pooled sample of the two groups (Cohen, 1977).

Cityscape 97

Newman and Holupka

Exhibit 1

2011 AHS Housing Quality Measures, Mean Prevalence Rates for U.S. Rental Housing

Mean Prevalence Rate Average for Counts

Not all rooms have plugs

1

# times blown fuses last 3 months

9

0.23

Exposed wiring

2

Unit does not have electricity

0.03

Unvented room heaters

1

No heating equipment

1

Use stove/oven for heat

0.1

# heating breakdowns last winter

3

0.8

Unit cold 24+ hours last winter

10

Cold due to utility interruption last winter

1

Cold due to inadequate heating capacity last winter

2

Cold due to inadequate insulation last winter

2

Cold due to other reason last winter

2

Roof leak last 12 months

5

Leak in wall/closet last 12 months

3

Leak in basement last 12 months

1

Leak other source last 12 months

1

Leaking pipes last 12 months

5

Leaking plumbing fixture last 12 months

2

Leak unknown source last 12 months

4

Crack in wall

7

Holes in floor

1

Peeling paint

3

Signs of rodents last 12 months

3

Signs of rats last 12 months

1

Signs of mice last 12 months

9

Signs of cockroaches last 12 months

5

Incomplete plumbing

0.3

# times toilet broke 6+ hours last 3 months

2

0.05

Share plumbing facility

2

Incomplete kitchen

4

# sewage disposal breakdowns last 3 months

1

0.03

No working elevator

5

Any mold

5

Broken/missing steps

1

Broken/missing stair railings

1

AHS = American Housing Survey.

Notes: N = 40,830 unassisted rental units from 2011 AHS. Excludes manufactured housing and units where a relationship exists between renter and landlord. See Newman and Holupka (2017) text and Appendix Table A-1 for more details. Weighted data. Average times for counts = average for entire sample, including zeros for those not reporting the problem. “# times no water last 3 months” never reported and we do not include in the exhibit. Last three items from Healthy Homes module.

Housing Quality Indices

Despite its intuitive appeal and the rich hedonic literature, the market value index performs poorly. 

Roughly 85 percent of the AHS housing quality items either do not reach statistical significance despite the very large sample, or operate in the opposite direction of expectations. The results do not appreciably improve after adjusting the threshold required for statistical significance using the Bonferroni correction to account for multiple comparisons. The results also are remarkably 98 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

consistent whether each item is tested separately or they are combined. It is likely that the rarity of each individual quality problem provides too little variance to make a meaningful contribution to the rent. In addition, rent appears to have a nonlinear relationship with the quality index. Rents are essentially flat across most of the distribution of the housing quality index but then fall significantly at the tail that represents the most housing problems. However, the hedonic formulation assumes a linear relationship between rent and dwelling features. Given its poor performance, we drop the market value housing quality index from the rest of the analysis.17

In contrast to the market value index, both the consumer rating index and the normative standards index perform well. The scores on both of these indices are highly skewed, as vividly demonstrated in exhibit 2. Most housing units have none of the 33 housing quality problems included in this analysis, with only a small fraction experiencing one or more problems.

As shown in exhibit 3, the consumer rating index model has strong explanatory power and the large majority of items operate in the expected direction (that is, odds ratio greater than 1).18 Most of the items are also statistically significant although, as previously noted, this is a less useful test given the large sample size. The five measures that make the largest contribution to house rating are (1) holes in the floor, (2) peeling paint, (3) cracks in the walls, (4) presence of rodents, and (5) cold due to inadequate insulation. For example, the presence of holes in the floor makes it 3.5 times more likely that the consumer’s house rating is poorer, peeling paint makes a poorer score 3.3 times more likely, and rodents make this 2.7 times more likely. To create a housing quality index based on the consumer rating criterion, we use the odds ratio for each quality measure as a weight.

Exhibit 2

Distribution of Consumer Rating and Normative Standards Index Scores

Note: High values for both indices converted into categories: 16 = 15 to 20; 17 = 20 to 30; and 18 = 30+.

17 See Newman and Holupka (2017), table 3.

18 One exception is the use of an oven for heat, which affects a very small proportion of rental units (see exhibit 1).

Cityscape 99

Newman and Holupka

Exhibit 3

Housing Quality Predictions of Consumer Rating Index

Odds Ratio

p-Value

Not all rooms have plugs

1.973

.000

***

# times blown fuses last 3 months

1.175

.000

***

Exposed wiring

1.363

.000

***

Unit does not have electricity

1.770

.573

Unvented room heaters 

1.075

.588

No heating equipment 

1.261

.106

Use stove/oven for heat

0.725

.269

# heating breakdowns last winter

1.225

.000

***

Unit cold 24+ hours last winter

2.017

.000

***

Cold due to utility interruption last winter

1.731

.000

***

Cold due to inadequate heating capacity last winter

2.212

.000

***

Cold due to inadequate insulation last winter

2.656

.000

***

Cold due to other reason last winter

1.531

.000

***

Roof leak last 12 months

1.987

.000

***

Leak in wall/closet last 12 months

1.801

.000

***

Leak in basement last 12 months

1.921

.000

***

Leak other source last 12 months

1.541

.000

***

Leaking pipes last 12 months

1.678

.000

***

Leaking plumbing fixture last 12 months

1.904

.000

***

Leak unknown source last 12 months

1.482

.000

***

Crack in wall

2.708

.000

***

Holes in floor

3.509

.000

***

Peeling paint

3.253

.000

***

Signs of rodents last 12 months 

2.657

.000

***

Signs of rats last 12 months

2.127

.000

***

Signs of mice last 12 months

1.626

.000

***

Signs of cockroaches last 12 months

2.052

.000

***

Incomplete plumbing

1.430

.197

# times toilet broke 6+ hours last 3 months

1.275

.000

***

Share plumbing facility

1.111

.172

Incomplete kitchen

1.208

.000

***

# sewage disposal breakdowns last 3 months

1.243

.000

***

No working elevator

1.262

.002

**

***  p  < .001. **  p  < .01. 

Notes: Results from 33 separate ordinal logistic regressions on subjective housing rating (reverse coded so high score = 

poorer quality). Other covariates in each regression include dummy variables for room air conditioner, central air conditioner, dishwasher, garbage disposal, clothes dryer, washing machine, electric heat, gas heat, oil heat, den/TV room, dining room, family room, working fireplace, garage/carport, half-bathrooms, laundry room, porch/deck/patio, connected to public sewer, use well water, electricity included in rent, gas included in rent, oil included in rent, and other fuel included in rent. Also included in the regressions are number of bathrooms; number of bedrooms; number of floors in unit; number of floors in building; building age; building age squared; building age cubed; unit type (for example, single-family, single-family attached, multifamily); number of months in unit; neighborhood self-rating; and U.S. Department of Housing and Urban Development Fair Market Rent. N = 36,833. Odds ratio =  eβ  where “β ” is the coefficient from the logistic model.

In the normative standards index based on a factor analysis, the RMSEA results support the use of a one-dimensional index for measuring housing quality. These results also provide weights for an index based on the factor scores.19 The factor loadings, shown in exhibit 4, are consistent with 19 Both the average and lower bound of the RMSEA scores are below 0.05, the criterion set by Preacher et al. (2013) for selecting the number of factors. See Newman and Holupka (2017), table 6.

100 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

Exhibit 4

Housing Quality Components of the Normative Standards Index

Factor Analysis Weights

Not all rooms have plugs

1.000

# times blown fuses last 3 months

0.921

Exposed wiring

1.465

Unit does not have electricity

1.649

Unvented room heaters 

0.968

No heating equipment 

– 1.435

Use stove/oven for heat

– 1.497

# heating breakdowns last winter

0.747

Unit cold 24+ hours last winter

3.340

Cold due to utility interruption last winter

2.593

Cold due to inadequate heating capacity last winter

2.626

Cold due to inadequate insulation last winter

—

Cold due to other reason last winter

2.184

Roof leak last 12 months

2.019

Leak in wall/closet last 12 months

1.920

Leak in basement last 12 months

1.755

Leak other source last 12 months

1.454

Leaking pipes last 12 months

1.812

Leaking plumbing fixture last 12 months

1.787

Leak unknown source last 12 months

1.263

Crack in wall

2.667

Holes in floor

2.702

Peeling paint

2.685

Signs of rodents last 12 months 

3.629

Signs of rats last 12 months

2.441

Signs of mice last 12 months

2.783

Signs of cockroaches last 12 months

2.071

Incomplete plumbing

1.629

# times toilet broke 6+ hours last 3 months

0.717

Share plumbing facility

0.270

Incomplete kitchen

0.707

# sewage disposal breakdowns last 3 months

0.207

No working elevator

0.482

Notes: Factor analysis estimated in Mplus using polychoric correlations. “Cold due to inadequate insulation last winter” 

dropped from factor analysis because perfectly it correlated with other measures.

the odds ratios produced by the consumer rating model. The highest factor loadings are presence of rodents; number of times the dwelling was cold for 24 hours or longer; presence of mice; holes in the floor; peeling paint; and cracks in the walls. Items with the lowest factor loadings include using the oven for heat; lacking heating equipment; number of toilet breakdowns lasting 6 hours or longer; incomplete plumbing; and sharing plumbing facilities.20 

20 The reference variable is whether all rooms have electrical outlets (“plugs” in exhibit 1). 

Cityscape 101

Newman and Holupka

Validity Tests

The consumer rating and normative standards indices are highly correlated ( r =.967,  p =.000 with two-tailed test), suggesting that they appear to be measuring the same underlying phenomenon 

and, therefore, have strong convergent validity. It is worth noting that two additional indices we developed that are more ad hoc versions of a normative standards index also are highly correlated with both the consumer rating and normative standards factor analysis indices. One ad hoc index applies the weights from Eggers and Moumen (2013b), which are based on a combination of the 

AHS ZADEQ housing quality measure and the authors’ judgment, whereas the other simply assigns a weight of 1 to each of the 33 housing quality items (see Newman and Holupka, 2017, table 8).

Exhibit 5 shows the results for a second test of convergent validity—the correlations between each housing quality index and other AHS measures associated with housing quality. In addition to actual and logged rent, house rating, and building age, we also include the AHS’ ZADEQ.21 All of the correlations are statistically significant and operate in the expected direction. Although the correlation between each index and ZADEQ is higher than it is with rent, house rating, and building age, it is less than half the correlation between the consumer rating and normative standards indices.

The predictive validity of the two housing quality indices is somewhat mixed. As shown in exhibit 6, both the consumer rating index and the normative standards factor analysis index are significant predictors of the respondent’s rating of their house on a 10-point scale. Poorer housing quality, indicated by higher scores on each index, is associated with a worse (that is, higher) house rating even after controlling for household and geographic location characteristics. The coefficients suggest that a one standard deviation increase in each index produces nearly a 20-percent improvement (that is, reduction) in house rating. However, the consumer rating index is not a statistically significant predictor of rent (although it has the expected negative sign), and the normative standards index is statistically significant only at the more liberal .10 level. The unusual shape of the relationship between rent and each index—essentially flat until the highest values at the tail of the index—may contribute to the muted statistical significance.

Exhibit 5

Convergent Validity: Correlations With AHS Measures Related to Housing Quality

Consumer Rating Index

Normative Standards Index

Rent

– .017***

– .022***

(.000)

(.000)

Log rent

– .015 ***

– .019 ***

(.000)

(.000)

House rating

– .341***

– .338***

(.000)

(.000)

Building age

.214***

.223***

(.000)

(.000)

ZADEQ

.419***

.375***

(.000)

(.000)

AHS = American Housing Survey.

***  p  < .001.

Notes: Weighted data. Two-tailed significance test. ZADEQ is a measure of housing unit quality computed in the AHS.

21 The respondent’s housing rating is distinct from the consumer rating index. The house rating measure is the respondent’s rating, from 1 to 10, of the dwelling as a good place to live, without any direct reference to the 33 quality measures. By contrast, the index assigns weights to each of the 33 quality measures based on the respondent’s dwelling rating. 

102 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

Exhibit 6

Predictive Validity of Housing Quality Indices: Regression Results for Rent and 

House Rating

Consumer Rating Index

Normative Standards Index

Log rent

– 1.518

– 2.376+

(.157)

(.055)

House rating

.848 ***

.830 ***

(.000)

(.000)

+  p  < .10. ***  p  < .001.

Notes: Top number is regression coefficient for rent, odds ratio for house rating. Bottom number is  p -value. Log rent uses ordinary least squares, house rating uses ordinal logistic regression. Log rent models also control for log household income; household head’s age, race, and ethnicity; air conditioning; washer and dryer; type of heat; fireplace; garage; laundry room; porch; number of rooms; pay for utilities; number of months in unit; age of building; area Fair Market Rent; number of bedrooms; number of bathrooms; and geographic location (region and central city, suburban, or rural). House rating models also control for log household income; household head’s age, gender, race, and ethnicity; and geographic location (region and central city, suburban, or rural).

Assisted Housing Quality

Exhibit 7 compares the quality of the assisted and unassisted stock. As shown in the column head-ings, we define two unassisted housing comparison groups: housing units with rents that equal or fall below the FMR, and all rental units. Results from a more rigorous approach to matching assisted and unassisted cases through propensity score matching (PM) produced similar results to those shown in the table.22 The table includes 36 housing quality items—the 33 we have referred to throughout this article plus three additional items from the AHS Healthy Homes module: mold, broken railings, and broken steps.

Although the difference in housing quality between the assisted and unassisted stock is statistically significant in more than half of the 36 quality items, statistical significance is not a sensitive test with very large samples. A more useful metric is the effect size, measured by Cohen’s  d. As the table shows, none of the housing quality items that achieve statistical significance at the  p ≤ .001 (for example, problems with heating, roof leaks, or rodents) attains a Cohen’s  d of 0.2, the accepted threshold for a small effect. Thus, it is not surprising that the consumer rating and the normative standards index scores also do not differ for the two housing stocks. This leads to the conclusion that, based on the housing quality items examined here, the quality of assisted housing is comparable to the quality of unassisted housing. This conclusion applies whether we limit the unassisted stock to units with rents at or below the FMR, to all unassisted rental units, or to propensity score-matched assisted and unassisted units. 

Within the assisted housing stock, although the consumer rating and normative standards housing quality index values are always worst for public housing and best for multifamily housing, the effect sizes never reaches the 0.2 threshold for a small effect (not shown). Characterizing assisted housing by a three-category household type measure—elderly, nonelderly family, nonelderly disabled—reveals that assisted housing for the elderly enjoys the best housing quality and housing for 22 PM models controlled for an array of both household and housing unit characteristics. Results available from the authors.

Cityscape 103

Newman and Holupka

Exhibit 7

Prevalence Rates of 2011 Housing Problems, by Housing Assistance Receipt

Assisted Housing Versus  

Assisted Housing Versus  

Rentals ≤ FMR

All Rentals

Assisted  Unassisted 

Assisted  Unassisted 

Cohen’s  d

Housing

Housing

Housing

Housing

Not all rooms have plugs

1.3

1.1

+

1.3

0.9

**

Ever blown fuses

8.6

8.9

8.6

9.3

+

# times blown fuses

22.1

22.3

22.1

23.3

Exposed wiring

2.8

2.3

*

2.8

2.2

**

Unit does not have electricity

0.00

0.02

0.00

0.03

Unvented room heaters

0.3

1.3

***

.091

0.3

1.0

***

No heating equipment

0.2

1.0

***

.094

0.2

0.7

***

Use stove or oven for heat

0.1

0.2

0.1

0.1

Ever heating breakdowns

4.6

3.4

***

.060

4.6

3.1

***

# times heating broke down

11.3

9.4

+

11.3

8.1

***

Unit cold 24+ hours last winter

17.5

12.3

***

.100

17.5

11.3

***

Cold: utility interruption

1.2

1.1

1.2

1.0

Cold: inadequate heating

3.0

2.2

**

3.0

1.9

***

Cold: inadequate insulation

2.3

1.8

*

2.3

1.7

***

Cold: other reason

2.3

1.8

*

2.3

1.5

***

Roof leak last 12 months

3.2

5.3

***

.098

3.2

4.9

***

Leak in wall/closet

3.4

3.0

+

3.4

2.9

*

Leak in basement

1.3

1.2

1.3

1.4

Leak other source

1.4

1.3

1.4

1.3

Leaking pipes

5.4

5.2

5.4

4.9

Leaking plumbing fixture

2.4

2.4

2.4

2.4

Leak unknown source

5.0

3.7

***

.069

3.6

5.0

***

Cracks in walls

7.5

7.7

7.5

6.9

Holes in floor

2.1

1.6

*

2.1

1.4

***

Peeling paint

3.9

3.7

3.9

3.1

***

Signs of rodents

5.3

4.0

***

.066

5.3

3.1

***

Signs of rats last

1.1

1.3

1.1

1.2

Signs of mice last

12.8

10.2

***

.087

12.8

9.0

***

Signs of cockroaches

5.8

6.3

5.8

4.5

***

Incomplete plumbing

0.2

0.5

**

0.2

0.3

Toilet ever broke

3.3

2.5

***

.048

3.3

2.3

***

# times toilet broke

6.7

4.9

**

6.7

5.0

**

Share plumbing facility

2.3

1.7

**

2.3

1.6

***

Incomplete kitchen

4.8

4.3

4.8

4.0

**

Ever sewage breakdown

1.5

1.5

1.5

1.3

# sewage breakdowns

3.8

3.2

3.8

3.0

No working elevator

6.1

4.7

***

.066

6.1

5.0

***

Any mold

8.0

8.5

8.0

7.9

Broken/missing steps

0.6

0.8

0.6

0.8

Broken/missing stair railings

0.9

1.2

0.9

1.3

+

FMR = Fair Market Rent.

+  p  < .10. *  p  < .05. ** p < .01. ***  p  < .001.

Notes: Assisted housing N = 8,472; All rentals unassisted housing N = 40,830; Rentals ≤ FMR unassisted housing N = 24,190. 

Weighted data. Average for counts (“#”) = average for entire sample, not just those reporting the problem. Cohen’s  d  computed for all effects significant at .001 or less. Values < 0.2 indicate virtually no difference. 

104 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

the nonelderly disabled has the worst.23 As shown in exhibit 8, the difference in the index scores of these two household types reaches a medium effect size (0.50 for consumer rating and 0.51 for normative standards). Bivariate analysis of additional housing, household, and location attributes suggests other variations in housing quality within the assisted stock. These include whether the unit is located in a central city, suburb or rural area, household size, and region. A central city location is associated with a worse score on both housing quality indices (Cohen’s  d = 0.24 to 0.25), as the household size grows, quality worsens (Cohen’s  d = 0.65 to 0.68), and assisted housing in the Northeast has the worst quality ratings (Cohen’s  d = 0.41 to 0.45).

Multivariate models predicting the housing quality score on each of the two indices, controlling for housing, location, and household characteristics, produce very similar results to those in the bivariate descriptive analysis. We estimate two sets of regression models using negative binomial modeling to account for the severe skew in the housing quality indices.24 The first is limited to the assisted housing sample and the policy variable of primary interest is assisted housing program type (the voucher program is the excluded category). Next, we pool the assisted and unassisted housing samples and test whether, all else equal, living in assisted housing has a sizable effect on the housing quality index score, and then test whether the assisted housing program type affects the housing quality index score.

Exhibit 9 displays the results. Regardless of whether the sample is limited to the assisted housing stock (the top set of rows) or includes both the assisted and unassisted stock (the bottom set of rows), none of the odds ratios on any of the assisted housing measures, whether the general category or distinguished by program type, achieves even a small effect size despite several statistically significant coefficients (Chen, Cohen, and Chen, 2010). Among the other covariates, only two—whether anyone in the household is disabled and whether the household lives in the Northeast region—have small effect sizes in each of these models. Holding other characteristics constant, households living in the Northeast and those with a disabled member have worse housing quality.

Exhibit 8

Housing Quality Index Ratings of 2011 Assisted Housing Units, by Household Type

Consumer Rating Index

Normative Standards Index

Elderly

1.71

1.57

< 62 disabled

3.79

3.45

< 62 family

2.57

2.31

p-value

.000

.000

Cohen’s  d

.500

.514

Notes: Weighted data. Elderly n = 3,165; < 62 disabled n = 1,597; < 62 family n = 2,648. Excludes 14 percent of assisted housing cases where head < 62, not disabled, and no children.  p -value tests significance of difference among all three groups. Cohen’s  d  compares “elderly” to “< 62 disabled.” Values < 0.2 indicate virtually no difference.

23 The AHS does not identify housing for the elderly, families, or young disabled. To construct these categories, we assume a household head 62 years of age or older is living in elderly housing, that families with children 18 or younger and without a disabled member are living in family housing, and that nonelderly persons younger than 62, even if they are living with family members, are in housing for the disabled. This is admittedly a very blunt approach but is the best that can be done with the AHS data.

24 More than one-half of the samples report no housing quality problems in either index, the dispersion ratios are roughly 1.8, and the standard deviation is nearly twice as large as the mean. 

Cityscape 105

Newman and Holupka

Exhibit 9

Multivariate Models Predicting Housing Quality Index Ratings

Negative Binomial

Odds Ratio

p-Value

Assisted housing only

Consumer rating index

Public housing

.952

Multifamily 

.987

Normative rating index

Public housing

.924

Multifamily 

.962

Assisted and unassisted (≤ FMR) housing

Consumer rating index

Assisted housing

.925

*

Normative rating index

Assisted housing

.956

Consumer rating index

Public housing

.869

*

Multifamily

.942

Voucher 

.934

+

Normative rating index

Public housing

.894

*

Multifamily

.966

Voucher 

.970

FMR = Fair Market Rent.

+  p  < .10. *  p  < .05.

Notes: Total unweighted n = 25,808. Weighted data. Covariates: census region; metropolitan location; head’s age, race, gender, and marital status; number of persons in household; whether anyone in household disabled; income quartile; and structure type. Because negative binomial models cannot use decimals, dependent variables multiplied by 100 and rounded. 

A small effect, equivalent to a Cohen’s  d  of 0.2, would be an odds ratio > 1.4 if > 1 or < .71. Odds ratios between .71 and 1.4 are not significant. Vouchers excluded from assisted housing models; unassisted housing excluded from models pooling assisted and unassisted housing. 

Assisted Housing Quality and Fair Market Rents

As noted in the discussion of methods, we develop a relative housing cost scale, gross rent divided by the FMR, to examine the relationship between the consumer rating housing quality index and the FMR. The results, graphed in exhibit 10, indicate that housing quality is maximized roughly at the FMR, indicated by the vertical line. The worst housing quality occurs at about the 24th percentile of rent.25 In some cases, HUD approves payment standards up to 120 percent of the FMR, such as when a disabled household member requires reasonable accommodations in the voucher 

program. The figure shows no appreciable difference in housing quality between the FMR and 120 

percent of the FMR. The results are similar for the normative standards index.

25 This occurs at approximately the 60th percentile; that is, 60th x 40th (the FMR) = 24th.

106 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

Exhibit 10

Consumer Rating Index, by Relative Cost and FMR

FMR = Fair Market Rent.

Notes: Housing cost = rent + utilities obtained from the U.S. Department of Housing and Urban Development Housing Affordability Data System. Housing cost relative to FMR = (Housing Cost / FMR) * 100. The consumer rating index top-coded at the 99th percentile to avoid outliers. FMR scale excludes top and bottom 5 percent to avoid outliers.

Discussion

The substantial literature on housing quality demonstrates both the importance and the challenges of conceptualizing and measuring housing quality. The growing interest in how housing matters, primarily in the low-rent unassisted housing stock, and the ongoing concerns about HQS in the assisted stock, make this an opportune time to revisit housing quality. In this article, we review the relevant literature, develop alternative housing quality indices, test their validity, and apply them to both the assisted and unassisted housing stock. We focus on indicators of physical integrity of housing systems and exclude measures that are more likely to reflect residents’ housekeeping or behaviors.

Because no consensus exists about the features of the housing bundle that should be included 

in the definition of a dwelling’s quality and how each should be weighted in determining overall quality, we rely on three external criteria first suggested by research on the Experimental Housing Allowance Program (EHAP; Merrill, 1980): market value, consumer rating, and normative standards. We test the market value criterion with a hedonic approach. The consumer rating criterion identifies the dwelling features most associated with a resident’s assessment of the dwelling as a good place to live regardless of what the market price of these features might be. This criterion is consistent with the renewed interest by economists in happiness and subjective well-being. The normative standards criterion reflects community concerns and policy decisions such as building codes. We rely on the 2011 national AHS for the analysis, which provides large national samples of the assisted and unassisted stock, and identifies assisted housing based on address matching to HUD administrative data, not respondent self-report.

Despite its intuitive appeal, the market value criterion performs poorly. This could occur because of the rarity of each item, or the nonlinearity of the relationship between rents and the housing Cityscape 107

Newman and Holupka

quality index, which cannot be accommodated by the hedonic approach. However, it may also 

occur because the national sample comprises widely varying housing markets, and our controls 

for market attributes are relatively coarse. Another explanation is that the individual dwelling quality measures may not be the main drivers of rents, as suggested by Merrill (1980). Therefore, we drop this market value criterion from further analysis. These poor results call into question the applicability of hedonic models using individual measures of physical inadequacies as predictors for rental properties (for example, Emrath and Taylor, 2012). Merrill (1980) raised this same issue in her EHAP analysis.

For the consumer rating index, we use the odds ratios from ordered logistic regressions as the weights. For the normative standards index, we use weights derived from a factor analysis. Both of these indices perform well. Consistent with much past AHS housing quality research, the 

prevalence rate of almost all problems is very low, with most dwelling units having no problems. 

However, notable overlap occurs between the measures that are the strongest predictors of the consumer ratings index and the factor analysis normative standards index. These are presence 

of rodents, cold dwelling unit, holes in the floor, peeling paint, and cracks in the walls. These represent a mix of high and low prevalence dwelling conditions, which make this overlap of items between the two indices unlikely to be driven by simple math. The indices also achieve both 

convergent validity and predictive validity.

We find that the quality of assisted housing is comparable to the quality of unassisted housing. 

This conclusion applies whether we limit the unassisted stock to units with rents at or below the FMR, to all unassisted rental units, or to housing units emerging from statistically matching the assisted and unassisted units using PM techniques.

The type of assisted housing program does not appear to have an appreciable effect on housing quality. Although we control for an array of housing, household, and location characteristics in multivariate models predicting the housing quality score on either index, only two of these covariates—whether anyone in the household is disabled and whether the household lives in the Northeast region—achieves even a small effect, in both cases reducing housing quality.

This research provides hard evidence supporting the current 40th percentile of rents definition of the FMR. We find that the FMR is set at a level that is roughly at the inflection point for maximizing assisted housing quality as measured by the consumer rating and normative standards housing quality indices.

Overall, these positive findings demonstrating the comparable quality of the assisted and unassisted housing stock suggest that the current assisted housing inspection and quality control systems appear to be achieving the goal of providing physically adequate housing to assisted housing residents. They also lend support to the shift to biennial inspections in the voucher program and the biennial and triennial inspections for standard and high performers, respectively, in the public housing program. The findings reported may also be useful to HUD as it finalizes plans for a demonstration program to test a new approach to physical inspections including a single inspection protocol for public housing and voucher units.26 

26 See the joint Explanatory Statement accompanying the Consolidated Appropriations Act, 2016 (Pub. L. 114–113).

108 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

We view this research as one step along the path toward improving our understanding of housing quality. One important enhancement of this work would be to expand the measures to include 

additional aspects of the full housing bundle, particularly neighborhood features. Linking the AHS data via confidential geocodes to census tract data and an array of administrative data at the neighborhood level could accomplish that goal. Another extension would be to compare these 

results with housing inspection scores from HUD’s administrative data (that is, the Public Housing Assessment System and Real Estate Assessment Center housing inspection ratings for public housing and multifamily housing, respectively). At the more conceptual end of the continuum, this article does not focus on what measures  should be included in a measure of housing quality, only on how well the measures included in the AHS appear to be reliable and valid and form a coherent index. A consideration of what measures currently missing from the AHS should be included in the future is worth serious attention.

Acknowledgments

The research presented in this article was supported by the U.S. Department of Housing and Urban Development, Office of Policy Development and Research, HUD-MDRT-55510-Quality of Assisted 

Housing Stock.

Authors

Sandra Newman is a professor of policy studies at the Johns Hopkins University.

Scott Holupka is a senior research associate at the Johns Hopkins University.

References

Aaron, Henry. 1972.  Shelter and Subsidies: Who Benefits From Federal Housing Policies. Washington, DC: The Brookings Institution. 

American Public Health Association (APHA). 1945. “An Appraisal Method for Measuring the Qual-

ity of Housing: A Yardstick for Health Officers, Housing Officials and Planners,”  American Journal of Public Health 35 (8): 866–867. 

———. 1938.  Basic Principles of Healthful Housing. Washington, DC: Committee on Hygiene of Housing. 

Buron, Larry, Bulbul Kaul, and Rhiannon Patterson. 2003.  Quality of Housing Choice Voucher Housing. Bethesda, MD: Abt Associates. 

Carmines, Edward, and Richard Zeller. 1979.  Reliability and Validity Assessment: Quantitative Applications in the Social Sciences. Thousand Oaks, CA: Sage Publications. 

Carter, George R. III. 2011. “From Exclusion to Destitution: Race, Affordable Housing, and Homelessness,”  Cityscape 13 (1): 33–70. 

Cityscape 109

Newman and Holupka

Chen, Henian, Patricia Cohen, and Sophie Chen. 2010. “How Big Is a Big Odds Ratio? Interpreting the Magnitudes of Odds Ratios in Epidemiological Studies,”  Communications in Statistics-Simulation and Computation 39 (4): 860–864. 

Cohen, J. 1977.  Statistical Power Analysis for the Behavioral Sciences. New York: Routledge.

Cota, Wolf. 2017. Personal communication (e-mail). U.S. Department of Housing and Urban 

Development, Real Estate Assessment Center.

Coulson, N. Edward, and Herman Li. 2013. “Measuring the External Benefits of Homeownership,” 

Journal of Urban Economics 77: 57–67. 

Dolan, Paul, Tessa Peasgood, and Mathew White. 2008. “Do We Really Know What Makes Us 

Happy? A Review of the Economic Literature on the Factors Associated With Subjective Well-

Being,”  Journal of Economic Psychology 29 (1): 94–122.

Eggers, Frederick, and Fouad Moumen. 2013a.  American Housing Survey: Housing Adequacy and Quality as Measured by the AHS. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. 

———. 2013b.  American Housing Survey: A Measure of (Poor) Housing Quality. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research. 

Emrath, Paul, and Heather Taylor. 2012. “Housing Value, Costs, and Measures of Physical Ad-

equacy,”  Cityscape 14 (1): 99–125. 

Friedman, Samantha, and Emily Rosenbaum. 2004. “Nativity-Status and Racial/Ethnic Differentials in Access to Quality Housing: Does Home Ownership Bring Greater Parity?”  Housing Policy Debate 15 (4): 865–902. 

Goodman, J.L. 1978. “Causes and Indicators of Housing Quality,”  Social Indicators Research 5 (1): 195–210.

Gray, Robert W., Barbara A. Haley, and Brent D. Mast. 2008.  Tell Us About Your Home: Three Years of Surveying Housing Quality and Satisfaction in the Section 8 Housing Choice Voucher Program. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and 

Research, Program Monitoring and Research Division.

Joint Center for Housing Studies of Harvard University (JCHS). 2017.  The State of the Nation’s Housing 2017. Cambridge, MA: Joint Center for Housing Studies of Harvard University. 

Jöreskog, Karl G., and Dag Sörbom. 1996.  LISREL 8: User’s Reference Guide. Chicago: Scientific Software International.

Kain, John F., and John Quigley. 1970. “Measuring the Value of Housing Quality,”  Journal of the American Statistical Association 65 (330): 532–548. 

Khadduri, Jill, and Kathryn P. Nelson. 2007. “Targeting Housing Assistance,”  Journal of Policy Analysis and Management 11 (1): 24–41. 

110 Selected Outcomes of Housing Assistance

The Quality of Assisted Housing in the United States

Kriström, Bengt. 2008. “Applying Hedonics in the Housing Market: An Illustration.” In  Hedonic Methods in Housing Markets: Pricing Environmental Amenities and Segregation, edited by Andrea Baranzini, José Ramierz, Caroline Schaerer, and Philippe Thalmann. New York: Springer: 247–258. 

Merrill, Sally. 1980.  Hedonic Indices as a Measure of Housing Quality. Cambridge, MA: Abt Associates. 

Millennial Housing Commission. 2002.  Report of the Bipartisan Millennial Housing Commission. 

Washington, DC: The Millennial Housing Commission.

Newman, Sandra. 2008. “Does Housing Matter? A Critical Summary of Research and Issues Still To Be Resolved,”  Journal of Policy Analysis and Management 27 (4): 895–925. 

Newman, Sandra, and Philip Garboden. 2013. “Psychometrics of Housing Quality Measurement in 

the American Housing Survey,”  Cityscape 15 (1): 293–306. 

Newman, Sandra, and C. Scott Holupka. 2017.  The Quality of America’s Assisted Housing Stock: Analysis of the 2011 and 2013 American Housing Surveys. Multi-Disciplinary Research Team. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development 

and Research.

Newman, Sandra, and Ann Schnare. 1993. “Last in Line: Housing Assistance for Households With 

Children,”  Housing Policy Debate 4 (3): 417–455. 

———. 1988.  Subsidizing Shelter: The Relationship Between Welfare and Housing Assistance. Washington, DC: Urban Institute.

O’Regan, Katherine, and Keren Horn. 2012.  What Can We Learn About the Low-Income Housing Tax Credit Program by Looking at the Tenants?  New York: New York University, Furman Center for Real Estate and Housing Policy.

Preacher, Kristopher, Guangjian Zhang, Cheongtag Kim, and Gerhard Mels. 2013. “Choosing 

an Optimal Number of Factors in Exploratory Factor Analysis: A Model Selection Perspective,” 

Multivariate Behavioral Research 48 (1): 28–56.

Ross, Lauren M., Anne B. Shlay, and Mario G. Picon. 2012. “You Can’t Always Get What You Want: The Role of Public Housing and Vouchers in Achieving Residential Satisfaction,”  Cityscape 14 (1): 35–54.

Shroder, Mark. 2002. “Does Housing Assistance Perversely Affect Self-Sufficiency? Review Essay,” 

Journal of Housing Economics 11 (4): 381–417.

———. 2001. “Moving to Opportunity: An Experiment in Social and Geographic Mobility,” 

Cityscape 5 (2): 57–67.

Thibodeau, Thomas. 1995. “House Price Indices From the 1984–1992 MSA American Housing 

Surveys,”  Journal of Housing Research 6 (3): 439–481.

U.S. Bureau of the Census (Census Bureau). 1967.  Measuring the Quality of Housing: An Appraisal of Census Statistics and Methods. Washington, DC: Census Bureau.

Cityscape 111

Newman and Holupka

U.S. Department of Housing and Urban Development (HUD). 2015.  Worst Case Housing Needs: 

2015 Report to Congress. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

———. 2014.  Strategic Plan 2014–2018. Washington, DC: U.S. Department of Housing and Urban Development.

———. 2011.  Codebook for the American Housing Survey Public Use File: 1997 and Later. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and 

Research.

U.S. Department of Housing and Urban Development, Office of Inspector General (HUD OIG). 

2008.  HUD Lacked Adequate Controls Over the Physical Condition of Section 8 Voucher Program Housing Stock. Washington, DC: U.S. Department of Housing and Urban Development, Office of Inspector General.

U.S. Government Accountability Office (GAO). 2000.  HUD Housing Portfolios: HUD Has Strengthened Physical Inspections But Needs To Resolve Concerns About Their Reliability. Washington, DC: U.S. 

General Accounting Office. 

———. 1997.  Tax Credits: Opportunities To Improve Oversight of the Low-Income Housing Program. 

Washington, DC: U.S. General Accounting Office.

U.S. Senate. 2012.  Senate Committee Report To Accompany the T-HUD Appropriations Act for FY 2013. 

S. Rept. 112–157 (April 19). 92 p. https://www.congress.gov/112/crpt/srpt157/CRPT-112srpt157.

pdf. 

Vandenbroucke, David. 2016. Personal communication (e-mail). Senior Economist, U.S. Depart-

ment of Housing and Urban Development.

112 Selected Outcomes of Housing Assistance

An International Perspective:  

Reflection on the Symposium

Kwan Ok Lee

National University of Singapore

For decades, housing affordability has remained one of the main issues in social science research in most developed countries. Therefore, I believe that the four articles in this  Cityscape symposium will provide great insight to both domestic and international readers by documenting the recent state of housing assistance programs in the United States. I also appreciate the authors’ efforts to tackle critical dimensions of U.S. federal housing programs, ranging from housing quality and preservation of assisted housing units to the mobility and cost burdens of tenants. As the findings of articles are quite relevant to the non-U.S. contexts, it is worth discussing how they can provide broader implications and some cross-country lessons, including how each article relates to other articles on similar topics in the international literature.

Ray et al. (2018) address the continuing loss of the assisted housing inventory, raising the question of the long-term sustainability of affordable housing. Both McClure (2018) and Dawkins and Jeon (2018) find temporal variation in the length of stay in assisted housing and housing cost burdens in the Housing Choice Voucher program—two critical issues in the stability of assisted housing programs. Some scholarly works in Asian countries share similar questions and tend to put an 

emphasis on the appropriate role of the government. In the case of Hong Kong, the government 

has effectively preserved the public rental housing stock through large-scale redevelopment of obsolete units even after its adoption of the Private Sector Priority Strategy, which actively involves the private sector in providing assisted owner-occupied units (La Grange, 1998). In Singapore, evidence suggests that the price movement of public and private housing markets are closely 

interrelated, and policies that aim at any change in supply, financing, or regulations affecting one market have significantly influenced the other market (Ong and Sing, 2002; Phang and Wong, 

1997). Finally, Newman and Holupka (2018) focus on the quality of assisted housing and find that the government inspection and quality control systems play a role in providing physically adequate housing to assisted housing residents.

Although challenging in terms of collecting data and standardized measures, comprehensive, 

comparative analyses between different countries and among different assisted housing programs would be extremely useful for the future research. For example, unlike the United States, many countries including Singapore, China, and Hong Kong have successfully implemented place-based public housing programs. These countries have shown much stronger government commitment in 

public housing programs and adopted policy measures for ethnic integration and effective property management. Hence, it would be interesting to comparatively analyze key features of these 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 113

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Lee

programs and draw lessons from their successes and failures. A further discussion on the transfer-ability of such successful programs would be informative to many scholars and policymakers who have to deal with broader housing affordability issues.

Author

Kwan Ok Lee is an assistant professor at the National University of Singapore.

References 

Dawkins, Casey, and Jae Sik Jeon. 2018. “Housing Cost Burden in the Housing Choice Voucher 

Program: The Impact of HUD Program Rules,”  Cityscape 20 (1): 39–62.

La Grange, Adrienne. 1998. “Privatising Public Housing in Hong Kong: Its Impact on Equity,” 

Housing Studies 13 (4): 507–525.

McClure, Kirk. 2018. “Length of Stay in Assisted Housing,”  Cityscape 20 (1): 11–38.

Newman, Sandra, and Scott Holupka. 2018. “The Quality of Assisted Housing in the United 

States,”  Cityscape 20 (1): 89–112.

Ong, Seow-Eng, and Tien-Foo Sing. 2002. “Price Discovery Between Private and Public Housing 

Markets,”  Urban Studies 39 (1): 57–67.

Phang, Sock-Yong, and Wing-Keung Wong. 1997. “Government Policies and Private Housing 

Prices in Singapore,”  Urban Studies 34 (11): 1819–1829. 

Ray, Anne, Jeongseob Kim, Diep Nguyen, Jongwon Choi, Kelly McElwain, and Keely Jones Stater. 

2018. “Opting In, Opting Out: A Decade Later,”  Cityscape 20 (1): 63–88.

114 Selected Outcomes of Housing Assistance

Refereed Papers

Refereed papers that appear in  Cityscape  have undergone a 

thorough and timely double-blind review by highly qualified 

referees. The managing editor reviews submitted manuscripts 

or outlines of proposed papers to determine their suitability 

for inclusion in this section. To submit a manuscript or 

outline, send an e-mail to cityscape@hud.gov.

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 115

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

116 Refereed Papers

Prioritizing Homeless Assistance 

Using Predictive Algorithms:  

An Evidence-Based Approach

Halil Toros

Daniel Flaming

Economic Roundtable

Abstract

In this article, we present a predictive model for identifying homeless persons likely to have high future costs for public services. We developed the model by linking administrative records from 2007 through 2012 for 7 Santa Clara County, California agen-

cies and identifying 38 demographic, clinical, and service utilization variables with 

the greatest predictive value. We modeled records for 57,259 individuals from 2007 to 

2009, and the algorithm was validated using 2010 and 2011 records to predict high-cost 

status in 2012. A business case scenario shows that two-thirds of the top 1,000 high-cost users predicted by the model are true positives, with estimated posthousing cost reductions of more than $19,000 per person in 2011. The model performed very well in giv-

ing low scores to homeless persons with one-time cost spikes, achieving the desired result of excluding cases with single-year rather than ongoing high costs.

Overview

Homelessness is a major social problem in the United States, with large public health impacts affecting millions of individuals and families and costing billions of dollars. The most recent annual numbers available from the U.S. Department of Housing and Urban Development (HUD) are for 

2016 and show 1,421,196 people used an emergency shelter or a transitional housing program 

at some point during the year (HUD, 2016). The most recent point-in-time numbers are for 2017 

and show, on a night in January, 553,742 people were homeless. Among individuals, 24 percent 

were chronically homeless (HUD, 2017).

The health, personal, and economic challenges that chronically homeless individuals experience and the lack of effective, coordinated services to address these problems often lead to a vicious cycle of diminished well-being with serious implications for their service utilization patterns 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 117

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Toros and Flaming

(Economic Roundtable, 2015a). The impairments of some of these individuals might impede 

access to needed health services and other support systems, such as employment services. Consequently, they cycle through costly emergency-driven public systems without getting the ongoing care they need to address severe mental illness, substance use disorders, or chronic health conditions (Caton, Wilkins, and Anderson, 2007; Folsom et al., 2005).

The number of individuals experiencing homelessness substantially exceeds the number of af-

fordable housing units available for them. This shortfall includes permanently affordable housing with supportive services that is needed for many chronically homeless individuals. Managing the gap between housing supply and demand is a challenge for city and county housing agencies. 

The predictive algorithm we describe provides a fair, objective tool for triage—prioritizing which individuals may receive Housing First. 

Growth in homelessness over the last three decades has been exacerbated by economic downturns, loss of affordable housing and foreclosures, stagnating wages, an inadequate safety net, and the clos-ing of state psychiatric institutions. In response to this growing need, the federal response to homelessness shifted in 2009 from uncoordinated short-term responses to avert homelessness—primarily using shelters—to long-term housing solutions. Permanent housing subsidies have since been shown to significantly increase housing stability, food security, and child well-being (Gubits et al., 2016).

The first component of the federal strategy shift was providing permanent supportive housing 

(PSH), that is, housing that is permanently affordable combined with ongoing supportive services for people experiencing chronic homelessness, and prioritizing those individuals with the most severe disabilities for assistance. The second component was connecting PSH to street outreach, shelter, and institutional “in-reach” to identify and engage people experiencing chronic homelessness. The third component was communitywide adoption of Housing First to provide permanent 

housing as quickly as possible in order to end chronic homelessness and prevent its recurrence (USICH, 2015).

The Housing First model was introduced by Pathways to Housing, a New York City nonprofit, to 

provide homeless intervention services to adults with psychiatric diagnoses and substance abuse problems. The nonprofit provided immediate housing and services to homeless adults with co- 

occurring diagnosis as a matter of right, with no preconditions. It also incorporated a harm reduction approach to psychiatric and substance abuse treatment and empowered the consumers of 

services to make choices about housing and services (Greenwood, Stefancic, and Tsemberis, 2013).

The Housing First approach makes housing stabilization the centerpiece of homeless assistance and recognizes that some people need more than housing assistance to stabilize. A small but highly visible segment of the chronically homeless population has substantial service needs. PSH with a Housing First approach enables chronically homeless individuals with disabilities that interfere with maintaining housing on their own to become stable renters.

PSH with a Housing First approach is an effective intervention for enabling chronically homeless individuals to permanently exit homelessness. However, because housing resources are limited, one of the key challenges is identifying and targeting the “highest priority” individuals so as to allocate scarce housing in a way that produces the greatest benefit. It is well documented that 118 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach costly interventions, such as PSH, are not likely to generate cost offsets equal or higher than the cost of the interventions, except for the most costly users (Culhane, 2008; Poulin et al., 2010).

A 2015 study in Santa Clara County, California, confirmed that chronic homelessness is very costly. 

The 10 percent with the highest costs, the 10th decile, accounted for 61 percent of all public costs for homelessness, and the top 5 percent accounted for 47 percent of all costs (Economic Roundtable, 2015b). Studies in Los Angeles County found that PSH provided to chronically homeless 

individuals in the 10th decile generated large enough cost offsets to cover the costs of housing and services (Economic Roundtable, 2015b, 2009). However, the scarce supply of PSH is often rented out to the eligible population based on crude screening processes that rely on self-reported data. 

Given that PSH is proven to have a large impact on reducing chronic homelessness and associated public costs, a strong argument can be made for using more accurate screening tools to identify individuals who should have first priority for access to permanently affordable housing.

This article extends previous research applying predictive models to homelessness and high-cost service users. The model presented in this article predicts who will or will not become a high-cost public service user in the next year, given various person-level characteristics in the current year and previous year, providing a predictive score (probability) for each individual in order to determine housing priorities across large numbers of individuals.

Prioritizing high-cost homeless persons for whom the solution of housing costs less than the problem of homelessness improves the efficiency of PSH. Cost offsets from reduced service use after high-cost people are stably housed can be stretched across a larger pool of homeless people whose housing can be subsidized with those offsets.

This is a triage tool for connecting homeless persons who are high-cost users of public services with permanently affordable housing, community-based healthcare, and support services. The 

tool applies a statistical predictive model to administrative data in order to prioritize homeless adults with the highest needs and public costs. It provides highly accurate predictions comparable to those developed through studies of high-cost health system users. Because no other models 

predict high-cost service users within the homeless population, health sector models provide the closest comparison. These models identify patients at high risk of readmission to a hospital based on demographics, prior hospital admissions, and clinical conditions (Ash et al., 2001; Billings et al., 2013, 2006; Chechulin et al., 2014; Fleishman and Cohen, 2010; Moturu, Johnson, and Liu, 2010; Tamang et al., 2016).

This tool improves on earlier predictive models for identifying homeless individuals in the 10th cost decile (Economic Roundtable, 2012, 2011). Several other studies have also used predictive models to assess homeless risks. Byrne et al. (2016) estimated predictors of homelessness and developed methods for more efficiently targeting homeless prevention services. A recent study of the Home Base prevention program for families in New York City1 showed that adoption of 

1 Home Base was a homeless prevention program operated by the New York City Department of Homeless Services from 2004 to 2008. Workers interviewed applicants about potential risk factors for homelessness, including human capital, housing conditions, disability, interpersonal discord, childhood experiences, and shelter history. The study compared the accuracy of judgments made by workers in determining eligibility for services to the results produced by a screening model in predicting whether families would enter a homeless shelter in the following 3 years.

Cityscape 119

Toros and Flaming

an empirical model for deciding which families to serve can make homeless prevention more 

efficient (Shinn et al., 2013). Also, the U.S. Department of Veterans Affairs (VA) has explored using predictive models in screening homeless veterans (Montgomery et al., 2013). However, to date, no studies have examined the relationship of past service utilization to future high-cost homelessness using predictive algorithms to prioritize which homeless people get housing.

In this article, we describe the predictive modeling methodology used to develop a triage tool to prioritize housing access for an efficient and cost effective PSH program. After presenting the results and validation of the model, we develop a business scenario to estimate the cost savings after implementation of the triage tool. The article ends with a discussion of ways to use the tool, limitations of it, and recommendations.

Chronic Homelessness

The majority of people who become homeless remain so for less than a year. A smaller number of people, however, remain homeless much longer, experiencing continuous and chronic homelessness. 

According to federal guidelines, an individual is chronically homeless if he or she has a diagnosed disability—such as serious mental illness, substance use disorder, posttraumatic stress disorder, cognitive impairments or chronic physical illness or disability—and has been homeless and lives in a place not meant for human habitation, a safe haven, or in an emergency shelter for at least 1 continuous year or has experienced at least four episodes of homelessness in the past 3 years where the cumulative total of the four occasions is at least 1 year.2

Needs of chronically homeless individuals that are essential for their well-being go unmet, including connections to housing, income, family, and health. This leads to stress, anxiety, depression, deprivation, and chaos, thus destabilizing their lives. Over time, chronically homeless individuals have increasingly complex and costly needs, including serious health and mental health conditions and disabilities that result in cycling in and out of hospitals, jails, prisons, psychiatric hospitals, and homeless shelters.

Several studies describe the clinical and social characteristics and patterns of service utilization among people who are chronically homeless. The majority of individuals have a serious mental illness such as schizophrenia, bipolar disorder, or major depression. They also experience high rates of substance abuse disorders, physical disability, or chronic disease. Many experience co-occurring mental illness and substance use problems (Burt, 2002; Caton et al., 2005; Caton, Wilkins, and Anderson, 2007; Folsom et al., 2005; Rosenheck, 2000). In addition to serious disability, the lives of chronically homeless people are compromised by persistent unemployment and lack of earned 

income forcing dependence on public assistance for sustenance, healthcare, and, if fortunate, an eventual exit from homelessness (Caton et al., 2005; Caton, Wilkins, and Anderson, 2007). 

Moreover, chronically homeless individuals often have a long arrest history, cycling through jail and prison (Caton et al., 2005; Kushel et al., 2005; Metraux and Culhane, 2004; Zugazaga, 2004).

Chronically homeless individuals spend a disproportionate number of days in the shelter system (Kuhn and Culhane, 1998; Metraux et al., 2001). In addition, because of their complex and 

2 “Homeless Emergency Assistance and Rapid Transition to Housing: Defining ‘Chronically Homeless.’”  Federal Register 80 

(75791–75806). 2015. https://www.hudexchange.info/resource/4847/hearth-defining-chronically-homeless-final-rule/.

120 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach co-occurring disabling conditions, poor health status, and elevated rates of unintentional injuries and traumatic injuries from assault, chronically homeless persons have high rates of hospital emergency rooms use and hospitalization, and longer hospital stays for mental health and 

substance abuse problems (Culhane, Metraux, and Hadley, 2002; Folsom et al., 2005; Kuno et al., 2000; Kushel et al., 2002). As the chronically homeless population ages, its utilization of emergency rooms and hospital rooms increases (Caton, Wilkins, and Anderson, 2007). High incarceration rates, coupled with heavy use of mental health and medical facilities in jails and prisons are also well documented (Kushel et al., 2005; McNiel, Binder, and Robinson, 2005; Metraux and Culhane, 2004).

Heavy use of acute and behavioral healthcare, criminal justice involvement, and use of social services may cost tens of thousands of dollars per individual annually (Ly and Latimer, 2015; Culhane, Metraux, and Hadley, 2002; Gilmer et al., 2009; Larimer et al., 2009; Martinez and Burt, 2006; McLaughlin, 2011). Although chronically homeless people represent only 20 percent of 

shelter users, they consume the largest share of health, social, and justice services with enormous costs (Ly and Latimer, 2015). In Los Angeles County, among homeless recipients of General Relief cash aid,3 the highest cost decile accounted for 56 percent of all public costs for homeless single adults (Economic Roundtable, 2011, 2009). A recent study using Santa Clara County data also 

showed that public costs for homelessness are heavily skewed toward a comparatively small number of frequent users of public and medical services. Among residents experiencing homelessness in 2012, the 10th decile, accounted for almost two-thirds of costs, and the top 5 percent accounted for almost half of costs (Economic Roundtable, 2015b). 

Federal funding for homeless programs increased from $3.7 billion in 2010 to nearly $5.5 billion in 2016 (USICH, 2016). In addition, federal expenditures for homeless individuals are also distributed through Medicaid, Medicare, and the VA, as well as large expenditures by state and county governments and institutions such as hospitals, jails, and social service agencies.

Although public outlays to address chronic homelessness have been growing since 2010, the 

prevalence and costs of homelessness remain high. With finite resources for homeless assistance, prevention services and cost-effective interventions, such as PSH, have attracted growing interest from policymakers and academic research over the past decade (Apicello, 2010; Burt and Pearson, 2005; Byrne et al., 2014; Culhane, Metraux and Byrne, 2011).

Preventive Services and Permanent Supportive Housing

The logic of prevention requires definition of what is to be prevented (such as chronic homelessness) and specification of the association (preferably causal) between the intervention and 

prevention of the undesirable condition. Several frameworks have been suggested for developing prevention strategies for homelessness (Burt and Pearson, 2005). The high-risk framework is the most appropriate framework for conceptualizing how to design homeless prevention policies 

because it draws attention to the need for direct intervention among those individuals at greatest risk. This framework focuses on alleviating the causes of homelessness for the most vulnerable subpopulations (Apicello, 2010).

3 General Relief is a cash aid program that provides a maximum of $221 a month for destitute adults. Roughly two-thirds of the caseload is estimated to be homeless. This program is called General Assistance in other California counties.

Cityscape 121

Toros and Flaming

To be successful, prevention strategies for high-risk individuals need to be both effective and efficient (Burt and Pearson, 2005; Culhane, Metraux, and Byrne, 2011; Shinn, Baumohl, and Hopper, 2001). In this context, effectiveness refers to how capable a program is of facilitating the desired goal—prevention of homelessness with reasonable costs. Effectiveness should be evaluated with robust designs by comparing a treatment group of persons who received services to a control group of individuals not subject to the intervention. Otherwise, the effect of the services in preventing homelessness cannot be assessed accurately, because it is unrealistic to assume that all the people who received services would have become or stayed homeless in the absence of those services. It is also possible that the effect of services might have not been significant; homelessness might have been merely postponed; or the ranks of high-risk individuals might simply have been reshuffled, allowing some to “jump the queue” and push others back in the line (Shinn, Baumohl, and Hopper, 2001).

As noted previously, recent research has shown that PSH, using a Housing First approach, is a very effective homeless prevention service and has led to widespread and successful efforts to reduce chronic homelessness (Byrne et al., 2014; Culhane, Metraux, and Hadley, 2002; Greenwood, Stefancic, and Tsemberis, 2013; Larimer et al., 2009; Rog et al., 2014; Tsemberis and Eisenberg, 2000; USICH, 2015, 2010). Based on increasing evidence, the U.S. federal government has endorsed 

PSH using a Housing First approach as the “clear solution” to chronic homelessness, and PSH has become an important priority for HUD. The number of beds in PSH projects increased nearly 60 

percent between 2007 and 2014, when an estimated 285,400 people lived in PSH (HUD, 2014; 

USICH, 2010).

Research has also demonstrated the effectiveness of PSH in generating cost offsets. Many studies have shown that PSH and Housing First interventions for chronically homeless individuals lead to cost savings through reduced shelter costs, decreases in both psychiatric and medical inpatient hospitalization costs, lower emergency room visit costs, reduced substance abuse treatment 

costs, and reduced criminal justice costs due to fewer arrests, detentions, and court appearances (Culhane and Byrne, 2010; Henwood et al., 2015; Ly and Latimer, 2015; Martinez and Burt, 

2006; Shinn, Baumohl, and Hopper, 2001; Shinn et al., 2013; Toros and Stevens, 2012). Cost 

savings from providing PSH to homeless people with mental disorders was shown to be substantial (Culhane, Metraux, and Hadley, 2002; Gilmer et al., 2009; Larimer et al., 2009; McLaughlin, 2011; Sadowski et al., 2009).

Despite such successes, the high cost of PSH would limit its availability to chronically homeless individuals with the greatest service needs if cost offsets are the benchmark for determining eligibility. Culhane (2008) reviewed several studies and concluded PSH is not likely to generate cost offsets equal to the cost of the interventions, except for the most costly users. Other studies also support the view that only frequent users of higher-cost services are likely to have sufficiently high costs to fully or mostly offset the costs of a PSH placement. Some research indicates that group may be limited to the most costly 10 percent of the chronically homeless (Poulin et al., 2010; Rosenheck, 2000). Moreover, since homeless people are typically placed in PSH programs at times when they are in crisis and have had relatively high service use, regression to the mean results in decreasing costs for many of these people, even if they are not placed in PSH (Ly and Latimer, 2015).

122 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach Hence, the research demonstrates that, although PSH is effective in reducing chronic homelessness and yields significant cost offsets, to be efficient, it should target high-cost homeless persons so that offsets will cover program and housing costs. In the context of homeless prevention, efficiency refers to targeting high-risk individuals. Efficient targeting is critical in the design and success of prevention services (Apicello, 2010; Burt and Pearson, 2005; Culhane, Metraux, and Byrne, 2011; Shinn, Baumohl, and Hopper, 2001). An efficient program should use empirically and/or theoretically derived risk factors to identify high-risk individuals who are likely to stay homeless and use costly public services unless they receive the prevention services.

However, the efficiency criterion introduces a serious challenge. Predictive models and screening tools are subject to the well-known tradeoff between sensitivity (the probability of correctly identifying true positives, or those individuals who will remain or become low-cost persons in Santa Clara County in the absence of the prevention program) and specificity (the probability of correctly identifying true negatives, or those individuals who would stay as low-cost homeless persons). If a low cutoff is selected, while the sensitivity increases and the model capturing more true positives, the specificity decreases leading to higher numbers of false positives. On the other hand, fewer false positives occur if the targeting cutoff is increased but many true positives are missed. This difficult tradeoff is at the core of the efficiency issue, as savings realized through placing a high-cost homeless person in PSH will be washed out if many low-cost homeless persons are also placed 

(Culhane, Metraux, and Byrne, 2011).

In the literature, it is argued that the common failing of many prevention efforts is their targeting inefficiency, which leads to ineffective programs (Burt and Pearson, 2005). It is also argued in the literature that available screening models are not sensitive or accurate enough to yield high hit rates without missing a large number of high-risk persons who would benefit from the program 

while producing cost savings (Apicello, 2010; Shinn, Baumohl, and Hopper, 2001). However, 

recent technological advances in the fields of predictive analytics and data mining together, with the availability of digital integrated administrative datasets with rich service utilization fields, allow significant improvement in prediction ability over earlier approaches and models (Larson, 2013).

This article presents the Silicon Valley Triage Tool. The County of Santa Clara supported the development of this tool so that it could identify homeless individuals in jails, hospitals, and clinics who have continuing crises in their lives that create very high public costs, and also give them first priority for access to PSH. This effort took roughly 2 years and included linking records of homeless clients across county agencies, analyzing attributes and costs for these individuals, and developing the triage tool. The model is very robust and accurate, taking advantage of advanced prediction methodologies and a unique and exceptionally valuable database created by Santa Clara County, home to Silicon Valley, linking service and cost records across county departments for the entire population of residents who experienced homelessness over a 6-year period—a total of 104,206 individuals. The tool accurately identifies individuals experiencing homelessness whose acute needs create the greatest public costs and is expected to serve as a screening tool for efficient and effective PSH programs.

Cityscape 123

Toros and Flaming

Methods

Four steps were involved in developing the Silicon Valley Triage Tool: first, linking agency records to create an integrated dataset; second, analyzing the data and developing the triage tool; third, testing and validating the tool; and fourth, developing a business model to project cost savings from using the tool. Each step is described in the following sections.

Data

By collaborating in linking their client records, seven agencies in Santa Clara County4 provided information on medical care (inpatient and outpatient), Emergency Medical Services (EMS), drug and alcohol treatment services, mental health treatment services (inpatient and outpatient), incarceration (arrest, court, and medical and mental health services in custody), and HUD-funded social and homeless services (Economic Roundtable, 2015b).

The Silicon Valley Triage Tool was developed using records for a subgroup of the total population that experienced homelessness. This subgroup included 57,259 individuals who used a homeless 

service and also had a linked record in another agency during the 6-year study window from 2007 

through 2012. This subgroup of records was used to develop the triage tool so as to avoid using records that may have had incomplete data because of uncompleted linkages across some agencies.

We benchmarked the tool against the total population that was homeless during the 6-year time window rather than just against individuals who were documented as being homeless at a specific time. 

We considered this time period because the problems that result in chronic homelessness are usually structural conditions in people’s lives—mental illness, trauma, debilitating health conditions, addiction, absence of qualifications or opportunities for employment, extreme poverty, and absence of sustaining personal connections. These problems do not go away just because someone is not documented as being homelessness in a given month; rather, they are drivers for the person’s life trajectory.

To accurately identify high-cost homeless individuals, the triage tool must use multi-year information about individuals, assessing service encounters over a larger rather than narrower interval in their lives. It is likely that individuals with the highest 5 percent of costs move in and out of institutional care settings without being consistently documented as homeless. In addition, homeless individuals who are admitted to private hospitals, state psychiatric facilities, or incarcerated by the state correctional system would not be documented as homeless in county data systems.

Because of these data gaps, the homeless and persistently homeless status of individuals in the top 5 percent often is not evident, so we made the assumption that individuals documented as having been homeless who have ongoing public costs in the top 5 percent are likely to be persistently homeless.

Linked datasets provided information about factors that affect the outcome of interest: being a high-cost user next year. These included demographic variables such as age, gender, and ethnicity; clinical variables such as ICD-9 (International Classification of Diseases, Ninth Revision) medical 4 The seven agencies participating in the record linkage were: the HUD Continuum of Care Board, Criminal Justice Information Control system of the Sheriff Department, Department of Alcohol and Drug Services, Emergency Management System, Mental Health Department, Social Services Agency, and Valley Medical Center.

124 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach diagnoses, and utilization variables for all service types from the current and previous year, including number of clinic or emergency room visits, number of hospitalizations and number of arrests, as well as the cost of services. The variables used in the model are listed in exhibit 1.

Exhibit 1

Averages of Model Variables for High-Cost and Other Homeless Persons (Validation 

Sample) (1 of 2)

High Cost 

Other  

Variable

(N = 5,726) (N = 51,533)

Demographics (%)

Age less than 18

5

10

Age 18–45

56

55

Age 46–65

36

31

Age 65+

3

4

Female

42

54

Criminal justice

100+ days of probation in the last 2 years (%)

18

5

Arrested in last 2 years (%)

46

16

Jail booking in last 2 years (%)

23

9

Jail security classification of 3 or 4 (that is, high risk) this year (%)

10

1

Arrested for inebriation and released within 48 hours—this year (%)

8

1

Mean number of arrests this year

0.78

0.16

Mean number of days in jail this year

32.9

5.2

Health diagnoses 

Diagnosed with chronic medical condition; Chronic Condition Indicator for 

68

35

ICD-9-CM diagnosis codes by HCUP (%)

Medical encounter with diagnosis of adjustment reaction ICD-9 309 in last 

11

3

2 years (%)

Medical encounter with diagnosis of heart disease ICD-9 401-429 in last 

6

2

2 years (%)

Mean number of medical encounters with diagnosis of organ failure ICD-9 

0.6

0.1

569-573, 576-578, 585-594, or 596 in last 2 years

Medical encounter with diagnosis of schizophrenia ICD-9 295 in last 2 

14

2

years (%)

Mean number of medical encounters with diagnosis of neoplasm (ICD-9 

0.4

0.1

140 to 239) in last 2 years

Medical encounter with diagnosis of “other ill-defined and unknown 

17

4

causes of morbidity and mortality” (ICD-9 799) in last 2 years (%)

Medical encounter with diagnosis of high-cost ICD-9 in last 2 years (%)

52

20

Health and emergency services

EMS encounters this year (%)

30

7

EMS encounters last year (%)

29

7

Two or more EMS encounters in last 2 years (%)

12

1

Admitted as hospital inpatient via emergency unit admission or transfer 

20

4

from psychiatric facility in last 2 years (%)

Outpatient psychiatric emergency services or ambulatory surgery this year 

41

15

(%)

Mean number of hospital inpatient admissions this year

0.30

0.06

Mean number of hospital inpatient days in last 2 years

3.7

0.6

Non-inpatient (ER or clinic visit) health system encounter this year (%)

68

43

Mean number of non-inpatient (ER or clinic visits) encounters this year

6.2

2.3

11 or more non-inpatient (ER or clinic visits) health system encounters this 

20

6

year (%)

Cityscape 125

Toros and Flaming

Exhibit 1

Averages of Model Variables for High-Cost and Other Homeless Persons (Validation 

Sample) (2 of 2)

High Cost 

Other  

Variable

(N = 5,726) (N = 51,533)

Behavioral health

Mean number of mental health outpatient days in the last 2 years

11.1

2.1

Two or more mental health outpatient visits in the last 2 years (%)

27

9

Mean number of mental health inpatient admissions this year

17.6

1.2

Two or more mental health inpatient admission in the last 2 years (%)

20

6

Substance abuse indicated by any recorded medical diagnosis or justice 

61

31

system charge (%)

Mean number of drug abuse and alcohol service encounters in the last 2 

14.3

3.9

years

HUD-funded homeless services and county public assistance

Chronic homeless flag in any HUD-funded homeless service provider 

27

11

record (%)

Public assistance benefits received this year (%)

46

40

Two or more months of food stamp payments received in the past 2 years 

47

44

(%)

EMS = Emergency Medical Services. ER = emergency room. HCUP = Healthcare Cost and Utilization Project. HUD = U.S. 

Department of Housing and Urban Development. ICD-9 = International Classification of Diseases, Ninth Revision.

The binary target variable indicated whether or not homeless persons were in the top 10 percent of high-cost users in 2009 (training cohort) and 2012 (validation cohort). In order to identify high-cost status, costs were summed across all service types and then ranked separately for the training and validation cohorts.

Analysis

The model predicts who is in the 10 percent of the homeless persons with highest public services costs in 2009, using data from 2007 and 2008. The model was validated by using 2010 and 

2011 records to predict high-cost status in 2012. The sample size for the training and validation cohorts was 57,259 records. The target group was 5,726 homeless individuals who made up the 

10 percent with the highest costs. It was important to test the model using data for 2010 to 2012 

in order to assess its out-of-sample predictive power. Strong predictive power is often observed based on in-sample performance if the model over-fits the data. When that is the case, the model only effective for explaining the training data, and out-of-sample performance is very poor. Because a predictive model is intended to be applied to new data with unknown outcomes, validation is needed to assess a model’s performance.

Model development was conducted in two stages. In the preprocessing stage, potential variables that might have an effect on becoming a high-cost user were identified based on earlier research and a series of  F-tests (for categorical variables) and  t-tests (for continuous variables). This step generated the first iteration of variable selection after eliminating redundant and irrelevant factors with  p-values greater than 0.25. The initial set of selected variables was transformed and prepared for model development using several techniques such as binning continuous variables, clustering categorical variables, and generating binary and count variables. All variables were generated for the current and previous years, and a total of 256 input variables were selected to be included in the model development.

126 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach

Several models for predicting high-cost users were developed and their performance was assessed using the SAS Enterprise Miner platform (Sarma, 2013; SAS, 2013). Several regression techniques were implemented to build models predicting the status of each person in the dataset as a high-cost user in the next year. We tested three techniques—logistic regression, least-angle regression, and decision tree models that are capable of explaining the classification or decision process, rather than using machine-learning algorithms that do not explain how given types of information are used to make predictions.

A comparison of the models’ performance based on the receiver operating characteristic (ROC) 

curve led to selecting a logistic regression model as the champion model. The ROC curve shown in exhibit 2 plots the tradeoff between sensitivity or true positive rate (probability of true prediction) and specificity or false positive rate (probability of false prediction). The ROC curve is used to quantify how accurately a model can discriminate between two states—typically referred to 

as “event” and “nonevent”—or to compare two alternative models predicting the same event. In 

the final phase, this model was fine-tuned, introducing interactions between variables, testing the nonlinearity of variables and applying a sensitivity analysis to decrease the number of variables—

particularly testing if current and previous year variables could be aggregated into a single variable without sacrificing the model’s performance.

Exhibit 2

ROC Curve

ROC = receiver operating characteristic.

Cityscape 127

Toros and Flaming

The final model was validated using the 2010–2012 cohort to assess the out-of-sample predictive power of the model. Sensitivity, specificity, positive predictive value (PPV), and accuracy measures, as well as the area ROC curve, were used to assess the out-of-sample model performance (Gonen, 2007).

The sensitivity statistic measures the proportion of high-cost homeless persons correctly identified by the model with high scores. It is also known as the true positive rate and reflects how well the model performs in capturing those homeless persons with high future costs. If the level is too low, a large number of high-cost homeless persons would not be provided with PSH.

The specificity statistic measures the proportion of not-high-cost homeless persons correctly identified by the model with low scores. If the level is too low, this is translated into to a high false positive rate (1-specificity), meaning a large number of homeless persons with low public costs would be provided with PSH.

The PPV statistic estimates the accuracy of the model by measuring the proportion of true positives (correctly classified high-cost homeless persons) within the population of all persons identified as high-cost persons. In other words, it is the probability that persons with a high score (above a defined cost threshold) truly are high-cost persons. Finally, the accuracy statistic measures the proportion of true positives and true negatives out of all persons.

The validated model was later utilized to estimate the potential costs and benefits of applying the model under several cutoff thresholds, using experience-based assumptions about costs of PSH 

and likely reduction in service use attributable to PSH placement.

Results

The final model had 38 variables with main effects out of 256 input variables tested and 11 variables with interactions. The descriptive values of model variables are shown in exhibit 1. The significance of the parameter estimates ( p-values) and odds ratios are presented in exhibit 3. As shown in exhibit 1, high-cost homeless persons in Santa Clara County represent a higher proportion of males than the overall population that experienced homelessness, and are slightly older. Their rate of engagement in the criminal justice system is very high relative to the rest of the population. 

Almost half of them were arrested during the previous 2 years compared to only 16 percent for the rest of the population. Their average number of days in jail is more than six times greater than the rest of the population—32.9 days versus 5.2 days.

After testing 970 3-digit ICD-9 medical diagnoses, 43 diagnostic groups, and 18 body system diagnostic categories, the model retained six effective diagnosis codes or groups—adjustment reaction, organ failures, heart diseases, schizophrenia, neoplasm, and other ill-defined and unknown causes of morbidity and mortality. In addition, two other factors were included, which are the aggrega-tions of chronic medical conditions and high-cost ICD-9. The high-cost homeless group shows 

much higher rates of encounters with these diagnoses whereas overall averages vary between  

6 percent (heart diseases) and 68 percent (chronic medical condition). More than one-half of the high-cost group had been diagnosed with 1 or more of the 59 high-cost ICD-9s, while only a fifth of the lower-cost population had any of these diagnoses.

128 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach Exhibit 3

Logistic Regression Adjusted Odds Ratios and 95-Percent Confidence Limits for 

Predictor Variables (Validation Sample)

95-Percent 

Odds 

Variable

Confidence 

Ratio

Limits

Demographics (%)

Age 18–45 versus less than 18* 

1.21

1.06–1.38

Age 46–65 versus less than 18

0.98

0.85–1.13

Age 65+ versus less than 18***

0.88

0.69–1.14

Female versus male***

1.07

1.00–1.14

Female

42

54

Criminal justice

100 or more days of probation in the last 2 years*

1.15

1.03–1.28

Arrested in last 2 years*

1.74

1.58–1.92

Jail booking in last 2 years*

1.14

1.04–1.26

Jail security classification of 3 or 4 (that is, high risk) this year*

1.63

1.41–1.89

Arrested for inebriation and released within 48 hours this year*

1.48

1.26–1.73

Number of arrests this year**

1.06

1.01–1.11

Number of days in jail this year*

1.007

1.005–1.009

Health diagnoses 

Diagnosed with chronic medical condition* 

1.21

1.10–1.33

Diagnosed with adjustment reaction in last 2 years*

1.26

1.06–1.49

Diagnosed with heart disease in last 2 years*

1.41

1.15–1.72

Number of medical encounters with diagnosis of organ failure in last 2 years*

1.08

1.06–1.11

Diagnosed with schizophrenia in last 2 years**

1.23

1.03–1.46

Number of medical encounters with diagnosis of neoplasm in last 2 years*

1.05

1.03–1.07

Diagnosed with “other ill-defined and unknown causes of morbidity and 

1.28

1.05–1.58

mortality” in last 2 years **

Diagnosed with high-cost ICD-9 in last 2 years**

1.120

1.009–1.240

Health and emergency services

EMS encounter this year*

1.27

1.14–1.41

EMS encounter last year*

1.26

1.14–1.40

Two or more EMS encounters in last 2 years*

1.34

1.12–1.60

Admitted as hospital inpatient via emergency unit admission in last 2 years*

1.35

1.19–1.54

Outpatient psychiatric emergency services or ambulatory surgery this year*

1.21

1.11–1.33

Number of hospital inpatient admissions this year*

1.16

1.09–1.25

Number of hospital inpatient days in last 2 years*

1.011

1.006–1.016

Non-inpatient (ER or clinic) health system encounter this year*

1.20

1.10–1.32

Mean number of non-inpatient (ER or clinic visits) encounters this year

6.2

2.3

11 or more non-inpatient (ER or clinic visits) health system encounters this 

20

6

year (%)

Number of non-inpatient (ER or clinic visits) encounters this year*

1.024

1.015–1.033

11+ non-inpatient (ER or clinic) health system encounters this year*

1.27

1.07–1.51

Behavioral health

Number of mental health outpatient days in the last 2 years*

1.013

1.010–1.015

Two or more mental health outpatient visits in the last 2 years*

1.40

1.23–1.59

Number of mental health inpatient admissions this year*

1.002

1.002–1.003

Two or more mental health inpatient admission in the last 2 years*

1.28

1.08–1.51

Substance abuse indicated by any recorded medical diagnosis or justice 

1.63

1.51–1.76

system charge* 

Number of drug abuse and alcohol service encounters in the last 2 years*

1.002

1.002–1.002

HUD-funded homeless services and county public assistance

Chronic homeless flag in any HUD-funded homeless service provider record*

1.28

1.17–1.39

Public assistance benefits received in the current year*

1.36

1.18–1.57

Two or more months of food stamp payments received in the past 2 years*

0.68

0.59–0.79

*  p  < .01. **  p  < .05. ***  p  < .10.

EMS = Emergency Medical Services. ER = emergency room. HUD = U.S. Department of Housing and Urban Development. 

ICD-9 = International Classification of Diseases, Ninth Revision. 

Cityscape 129

Toros and Flaming

The high-cost group also shows higher rates of engagement with health and emergency services. 

Group differences were large for EMS encounters (30 percent versus 7 percent), hospital inpatient admissions via emergency room admission or transfer from a psychiatric facility (20 percent versus 4 percent), and outpatient psychiatric emergency services or ambulatory surgery (41 percent versus 15 percent). The number of admissions and days of inpatient hospitalization and the number of outpatient encounters are also significantly higher for high-cost homeless persons.

Finally, behavioral health data show more frequent encounters for the high-cost group. Both mental health (inpatient and outpatient) and substance abuse service rates are higher. The prevalence of documented substance abuse, as indicated by any drug-related medical diagnosis or justice system charge, is twice as high for the high-cost group—61 percent versus 31 percent for the balance of the population. In contrast, the public assistance and homeless service participation rates differ only slightly.

Adjusted odds ratios presented exhibit 3 reflect the differences we observe from descriptive 

comparisons. Odds ratio for continuous variables are adjusted by controlling for all other variables. 

As a result, odds ratios for binary variables (for example, arrested or not) are generally higher than the odds ratios for continuous variables (for example, days in jail) and are interpreted differently. 

For example, the odds ratios show that persons who have been arrested in the past 2 years are 1.74 

times more likely to be in the high-cost group than those who have not been arrested. On the other hand, the odds ratio for each additional arrest is only 1.06, increasing the likelihood (or odds) of being in the high-cost group by 6 percent.

Odds ratios analysis reveals that being arrested in the last 2 years, higher jail security and substance abuse are among the strongest binary predictors of becoming a high-cost homeless resident, followed by being arrested for inebriation and released within 48 hours, heart disease, two or more EMS encounters, being admitted as a hospital inpatient via the emergency room, two or more 

mental health outpatient visits, and receiving public assistance benefits. All factors included in the model increase the likelihood of becoming a high-cost homeless person with adjusted ratios in the range of 1.05 and 1.28, with the exception of receiving 2 or more months of food stamp payments, which has an odds ratio of 0.68, indicating that receiving food stamps benefits makes it less likely to be in the high-cost group. The adjusted odds ratios for continuous variables all have values ranging from 1.002 (number drug abuse and alcohol services encounters) to 1.16 (number of 

hospital admissions), and all increase the likelihood of becoming a high-cost homeless person.

General performance of the model was evaluated using  C-statistic to assess the predictive ability of the model. The  C-statistic (sometimes called the “concordance” statistic or  C-index) is a measure of goodness of fit for binary outcomes in a logistic regression model. It gives the probability that a randomly selected subject who experienced an event (for example, became high-cost user) had a higher risk score than a subject who had not experienced the event. It is equal to the area under the ROC curve and has values ranging from 0.5 to 1.0.

The model achieved a very strong  C-statistic: 0.83.  C-statistic is the probability that predicting the outcome is better than chance. Models are typically considered reasonable when the  C-statistic is higher than 0.7 and strong when C-statistic exceeds 0.8 (Hosmer and Lemeshow, 2000). Overall, the model predicts high-cost homeless persons with a very good fit.

130 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach Exhibit 4 shows the predictive performance of the model for different scenarios: the top 1, 5, and 10 percent and the top 1,000 homeless persons with the highest risk of becoming a high-cost 

service user. The predictive performance measures were defined previously in the methods section.

If the 2,864 persons in the top 5 percent at greatest risk of becoming high-cost homeless service users are followed, the achieved sensitivity and specificity are 32.6 and 97.3 percent, respectively. 

These values suggest very reasonable predictive power, indicating that the model picks up 33 

percent of all high-cost service users and correctly identifies 97 percent of those who are not high-cost users. The PPV value of 51 percent and accuracy value of 92.3 percent for the top 5 percent are also very high. If we follow a subset within the top 5 percent, the 1,000 cases with the highest probability scores for being in the high-cost group (1.75 percent of all cases), we see even more accurate prediction outcomes. The model achieves a PPV result of 67 percent, meaning that out of 1,000 persons that the model identified as high-cost persons, two-thirds are true positives and the remaining one-third are false positives. PPV is an important measure for assessing the cost-effectiveness of the model.

Another measure of the effectiveness of a predictive model is the “lift,” which is calculated as the ratio between the results obtained with and without the predictive model for all thresholds. Exhibit 5 

illustrates the lift of the model, which is quite high for cases with a high probability of being in the high-cost group. For example, for the top 5 percent, the model generates a lift of 6.5. This means that model generates 6.5 times more correctly identified high-cost homeless persons (true positives) than random selection, which is presented as the baseline: a lift of 1 or 0. At slightly lower thresholds, such as the top 10 percent, lift drops to 4.7 because in order to capture more true positives, the model concurrently includes more false positives. Conversely, the number of false positives decreases as the probability of being in the high-cost group increases.

The most common way of assessing the predictive power of a model in the data mining literature is the area under the ROC curve. ROC shows the tradeoff between true positives (sensitivity) and false positives (1-specificity) at all possible thresholds. The ROC curve for the model is shown in exhibit 2. The accuracy of the model depends on how well it separates high-cost individuals from lower-cost individuals. Accuracy is measured by the AUC (Area Under the Curve, the ROC curve) or  C-statistic. The model generated a fairly high AUC of 0.83, indicating an 83-percent probability that a randomly selected homeless person with high future costs will receive a higher model score Exhibit 4

Predictive Performance of the Model

Measure

Top 1%

Top 5%

Top 10%

Top 1,000

Formula

Sensitivity (%)

9.3

32.6

47.7

14.9

True positive / (true positive + false 

negative)

Specificity (%)

99.7

97.3

93.2

99.4

True negative / (false positive + true 

negative)

PPV (%)

72.9

51.0

37.4

66.8

True positive / (true positive + false 

positive)

Accuracy (%)

92.6

92.3

89.5

92.7

(True positive + true negative) / number

PPV = positive predictive value.

Cityscape 131

Toros and Flaming

Exhibit 5

Lift Chart

than a randomly selected homeless person without high future service costs. In the predictive analytics literature, models with AUC exceeding 0.8 are accepted as models with good predictive power, and AUC values below 0.7 indicate poor model performance.

Because the model provides a probability score ranging from 0 to 1, we have to select a cutoff score or a threshold, above which homeless persons will be offered PSH. Choice of a cutoff level introduces the tradeoff between the correct identification of high-cost service users and false alarm rates. The ROC curve illustrates this tradeoff between true positives—finding as many homeless persons as possible who would be high-cost service users next year—and false positives— 

decreasing potential cost savings by including homeless persons who would not be high-cost 

service users next year.

Business Scenario and Cost Savings

Although the performance of the triage tool presented in this article is very high in statistical terms, it is still necessary to translate this performance into a pragmatic business scenario, showing how the tool contributes to the efficiency of PSH programs by prioritizing the population to be housed. The tradeoff to be weighed in using the triage tool is between, on the one hand, using lower selection thresholds in order to find as many high-cost homeless individuals as possible but accepting a substantial number of lower-cost individuals as part of the mix, and, on the other hand, using higher selection thresholds to identify a smaller population in which a higher proportion of individuals will be high-cost service users. This tradeoff is critical to the efficiency of a PSH 

program as elaborated previously. The model is highly accurate in distinguishing high-cost from 132 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach

low-cost users, however it is still necessary to calibrate the cutoff level based on goals for saving costs by offering PSH to the targeted population. The following analysis explores the cost efficiency of providing PSH to targeted high-cost homeless persons under different cutoff levels.

With 5 years of actual cost data, from 2008 through 2012, we used 2008 and 2009 data to 

produce probability scores for the likelihood of each individual being in the highest-cost group in 2010 and then track the accuracy and financial outcomes of these predictions over the next 2 years. Any placement decision has cost implications. If the homeless person predicted to be a high-cost user was correctly identified (true positive), the reduction in posthousing use of public services is likely to be roughly two-thirds. However, if the homeless person predicted to be a high-cost user was a false positive, then the expected cost savings would not be realized. The balance between the positive and negative savings generated by these two groups determines the efficiency of a PSH program.

One of the challenges the model must contend with is abrupt changes in costs in the scoring 

year—the year following the 2 years for which health conditions and service utilization are known. 

Some conditions are one-time events, resulting in costs that spike and then decline. Hence, the assessment of cost offsets should be done in the postplacement period, when the actual service utilization of true positives and false positives becomes evident. Some homeless persons who were true positives at the time of scoring year became low-cost users in subsequent years due to regression to the mean. On the other hand, some false positives that were predicted to be high-cost users but were low-cost users in the scoring year turned out to have higher costs in subsequent years.

These dynamics are shown in exhibit 6 for predictions of the top 5 percent. Looking at 2 years of post-scoring-year cost data (adjusted to 2014 U.S. dollars), the model successfully differentiates Exhibit 6

Average Annual Costs for Triage Tool Prediction Groups

Cityscape 133

Toros and Flaming

the highest cost cases from other cases, even though average costs decline because of regression to the mean. The low cost levels of true negatives verify the high specificity of the model. Another critical observation is that public costs for individuals experiencing homelessness vary significantly from one year to the next with important implications for the efficiency measure. False positives represent homeless persons with high service utilization prior to the scoring year of 2010, which led to high probability scores. However, in 2010, their service costs were low, making them false positives. On the other hand, their postprediction trend is positive, more than doubling between 2010 and 2012. If a person was predicted at the top 5 percent and had no service utilization at 2 years post scoring year, the person was labeled as false positive. If he or she was not predicted at the top 5 percent, the person was labeled as true negative, because he or she remained a low-cost user after the scoring year. 

Note also that false negatives, the group with low service utilization prior to 2010 and high costs in 2010, the scoring year, typically had one-time cost spikes. Their long-term trend is negative and subsequent to the scoring year their cost levels declined substantially. Hence, omitting them as high-cost users contributes to the efficiency of the program significantly as presented below exhibit 6 

suggests that cost savings should be assessed not at the year of scoring but rather in the postscoring years in order to capture the long-term service utilization of scored individuals.

The triage tool works to assign high scores to high-cost users, but different probability cutoff levels will exhibit different proportions of true positives with expected savings and false positives with no expected cost savings. Our estimation of net savings at different cutoff levels is based on the estimated cost savings for true positives after taking into account the housing and service costs for false positives. The results are sensitive to the probability score threshold, cost of housing, and the rate of anticipated reduction in service utilization and costs following placement in housing. As the probability score threshold increases, the ratio of true positives to false positives also increases, resulting in increased savings.

This analysis looks at financial outcomes based on two probability score thresholds, 0.37 and 0.53, for the predicted probability of having high costs in 2010, based on 2008 and 2009 information. 

The 0.37 cutoff level identifies approximately 5 percent of the test population as high-cost users. 

The 0.53 cutoff level identifies the top 1,000 high-probability service users in our test population. 

A different probability cutoff can be selected based on the requirements of specific initiatives to address homelessness. If the goal is to house a larger number of high-cost homeless persons, lower cutoff levels may be selected, resulting in lower savings per person. On the other hand, if the supply of housing is limited and a smaller number of high-cost homeless persons can be housed, than a higher cutoff level may be selected, resulting in higher savings per person.

It is assumed that the annual cost of PSH is $17,000 per person per year, based on rent subsidy and supportive service costs in Los Angeles. We used Los Angeles data for housing costs and 

posthousing cost savings because, at the time of our study, Santa Clara County did not have 

enough high-cost individuals who had been housed for a long enough interval to produce 

comparable data. This high-side estimate of housing costs is based on $11,000 annually for rental subsidy, including first-year costs for temporary housing and benefits advocacy, and $6,000 

annually for supportive services. Actual costs may be lower based on the level of subsidies built into different affordable housing projects and the level of long-term supportive services needed by tenants after they are stabilized in housing.

134 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach The posthousing reduction in service costs is assumed to be 68 percent for homeless persons in the 10th decile based on a study from Los Angeles (Economic Roundtable, 2009). Most other studies estimate service cost reductions for homeless persons in PSH for the whole population, rather than the top decile (Culhane, 2008; Culhane and Byrne, 2010). It is also assumed that there will not be any cost reduction for false positives—individuals below the 10th decile. This is a conservative assumption because an earlier study also found posthousing cost reductions among individuals in the 5th through 9th cost deciles (Economic Roundtable, 2011). However for purposes of the cost estimates shown here, net savings are -$17,000 for cost groups below the 10th decile because no cost savings are applied to them.

Exhibit 7 presents estimated cost savings for 2011 for the two selected cutoff levels (0.37 and 0.53). Posthousing costs are calculated as 32 percent of homeless costs for individuals in the 10th cost decile, and then $17,000 is added for each person in the group to cover the cost of housing and supportive services. Net savings are calculated by subtracting estimated posthousing costs from actual homeless costs for the year. All analysis was conducted in 2014 prices. Since actual costs in 2011 and 2012 were used, regression to the mean, that is, the tendency of extreme outcomes to be closer to the average when measured a second time, has been incorporated into the estimates.

Cost differences were estimated for four probability-cost groups, which each show different cost dynamics. If a score was above the selected cutoff (0.37 or 0.53) and 2010 costs were in the top decile, the record is a true positive. However, in subsequent years, true positives in 2010 may remain high-cost or become low-cost service users. The long-term cost status of individuals was Exhibit 7

Cost Savings for 2011 at the Cutoff Levels of 0.37 and 0.53

2010 Costs

2011 Costs

2011 Cost

2011 Net

2011 Total Number

Status

(Prediction  (1 Year After  Savings ($) Savings ($) Savings ($) of Cases

Year) ($)

Prediction) ($)

Cutoff level: 0.37

True positives— 

90,989

10,932

0

– 17,000

– 4,335,000

255

low-cost users

True positives— 

93,196

83,661

56,889

39,889

30,635,068

768

high-cost users

False positives— 

11,444

8,511

0

– 17,000

– 8,823,000

519

low-cost users

False positives— 

13,029

46,551

31,655

14,655

5,085,204

347

high-cost users

Total / average

11,944

22,562,272

1,889

Cutoff level: 0.53

True positives— 

111,580

11,496

0

– 17,000

– 2,074,000

122

low-cost users

True positives— 

96,892

86,947

59,124

42,124

22,367,823

531

high-cost users

False positives— 

12,427

8,829

0

– 17,000

– 3,094,000

182

low-cost users

False positives— 

13,579

43,560

29,621

12,621

2,082,432

165

high-cost users

Total / average

19,282

19,282,255

1,000

Cityscape 135

Toros and Flaming

evaluated based on their actual cost rankings in 2011 or 2012. If they were in the top decile in 2011 or 2012, they were identified as long-term high-cost users. Otherwise, they were identified as low-cost users.

If a score was above the selected cutoff (0.37 or 0.53) and 2010 costs were not in the top decile, the record is a false positive. False positives may also become high- or low-cost service users in the future. We tested this possibility by observing actual costs in 2011 and 2012 and identifying cases that moved into the true positive cost category. Exhibit 7 shows that, at the 0.37 cutoff level, out of the 1,123 individuals who were true positives, 255 became low-cost users in 2011. This cost shift was more than offset by 347 false positives that turned out to be high-cost users in 2011. In sum, out of 1,889 individuals, 1,115 (60 percent) were high-cost users in 2011.

If the 5 percent (0.37 cutoff level) with the highest probability of being high-cost service users were housed permanently with supportive services, savings of more than $22 million were estimated 

in 2011. Even though 40 percent of individuals were low-cost users in 2011 and would not be 

generating any cost savings, the net savings from the remaining 60 percent shows the feasibility of the intervention. The analysis shows a cost reduction of almost $12,000 per housed homeless person for the top 5 percent of the population identified by the triage tool as having the greatest probability of high future costs.

The results are even more positive when a higher cutoff level is selected, because the accuracy of the tool in predicting high-cost users improves as the probability level increases. The 2011 cost analysis for 1,000 persons in the test population with the highest probability scores, scores at or above 0.53, shows that almost two-thirds (653 individuals) were true positives. Evaluating actual costs in 2011, it is observed that 122 of them became low-cost users, whereas more than four-fifths (531) remained high-cost users. In addition, 165 false positives turned out to be high-cost users in 2011. In sum, out of 1,000 individuals, 696 (70 percent) were high-cost users in 2011. As expected, the feasibility of the intervention is higher at the 0.53 threshold than at the 0.37 threshold, with an estimated cost reduction for this group of more than $19,000 per person in 2011.

A separate analysis estimated savings in 2012 for both cutoff levels. Because lower cost levels were observed in 2012 due to the regression to the mean, lower cost savings were estimated. At the 0.37 

level, cost savings were estimated to be almost $16 million, which corresponds to more than $8,000 

per housed individual. At the 0.53 level, savings per individual were estimated to be $16,000, with cumulative savings for 2011 and 2012 estimated to exceed $35 million. Over the 2 years of postprediction data that we have for Santa Clara County, we see a year-to-year decline in actual costs for individuals with a high probability of having high costs. However, this may be the first phase of a longer-term cost cycle in which costs begin to increase again. This scenario is plausible considering that most individuals in this population have serious medical and mental health disorders that are likely to become more acute as they age. Indications of a longer-term cycle in which costs decline and then increase were found in an earlier cost study in Los Angeles (Economic Roundtable, 2009).

As noted previously, our cost savings analysis assumed that the annual cost of PSH is $17,000 per person per year and that the posthousing reduction in service costs is 68 percent for homeless persons in the 10th decile. Because both of these assumptions are made based on data and recent studies from Los Angeles, a separate sensitivity analysis was carried out to see how total net 136 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach cost savings estimates change if these cost assumptions change. The analysis showed that at the 0.37 cutoff level, the break-even point is reached when the annual cost of PSH is $29,000 or the posthousing reduction in service costs is 40 percent. These are the highest annual cost of PSH and the lowest percentage of service cost reduction that still yield net cost savings.

If instead of cost savings as a goal, a community seeks to break even against current or projected costs, a probability threshold of 0.20 is estimated to produce “break-even” cost outcomes, with cost savings from reduced service use equal to the cost of housing and services. An estimated 70 

percent of the population captured at this probability threshold is chronically homeless, and they represent an estimated 21 percent of all individuals who are chronically homeless in a given year.

Discussion

This study is the first attempt in Santa Clara County and one of the first studies to develop and validate a predictive model for identifying homeless persons who are likely to become high-cost users of public service. This model was developed using an integrated database built by linking seven agencies’ administrative records, which provided information on risk factors such as demographics, clinical variables, and service utilization variables for the current and previous years as well as cost of service data. The cost study that was used to develop the triage tool provided key evidence supporting Measure A, a $950 million affordable housing bond measure approved by voters in 2016.

The model is particularly strong when using high probability cutoff levels, generating small 

numbers of false positives and high numbers of true positives. For the top 1,000 high-cost users predicted by the model, two-thirds of them are true positives. A key strength of this study is that it assessed the overall effectiveness of predictions made by the tool, looking at costs over the 3 years following the 2 years that were the source of data used to make the prediction. This assessment showed that many false positives became high-cost or close-to-high-cost users in the second year after the prediction. In addition, a majority of the false negatives were actually true negatives over the next 2 years because their high-cost level in the scoring year represented a one-time cost spike. 

One of the challenges the model must contend with is abrupt changes in costs from one year to the next. Some conditions are one-time events, resulting in costs that spike and then decline. The tool performed very well by giving low scores to homeless persons with one-time cost spikes.

Another key strength of the study is information it provided for identifying distinctive attributes of high-cost individuals. Individuals in this group are the most likely to be diagnosed with a mental disorder, in particular, a disorder that takes the form of a psychosis, and a psychosis that takes the form of schizophrenia. They are also the most likely to be given a maximum or high-medium security jail classification because of the safety risk they are perceived to present. They are the most likely to have been continuously homeless for 3 years. They are most likely to be diagnosed with a skin disease such as cellulitis or an endocrine disease such as diabetes. They are most likely to be tri-morbid—diagnosed with a mental disorder, a chronic medical condition and to abuse drugs or alcohol. Demographically they are most likely to be male and to be in the middle of their lives—35 

to 44 years old. Also, they are most likely to frequent users of hospital emergency rooms and inpatient beds, emergency psychiatric facilities, mental health inpatient facilities, and to be incarcerated in a jail mental health cell block.

Cityscape 137

Toros and Flaming

This composite profile can help hospital and jail discharge planners and homeless service providers identify high-cost individuals. However, significant diversity is in the demographic attributes and types of crisis services needed by individuals in this population. The triage tool weighs the likely cost impact of each individual’s characteristics and uses this information to identify subgroups that fall outside this profile. For example, young women with acute mental illnesses and endocrine diseases who have ongoing high costs even though they are not substance abusers or involved in the justice system.

We further validated the model by developing a business analysis to assess its cost effectiveness. 

With 0.37 selected as the optimal cutoff level, which identifies the highest-cost 5 percent of the population that experienced homelessness over a 6-year period as the target group, the model 

assessed cost savings by comparing total housing and service costs ($17,000 annually) with the estimated 68 percent cost savings for true positives—those correctly identified as high-cost service users. The results confirmed that anticipated cost savings from true positives far exceed the total costs of housing, yielding net savings of $20,000 per person over the next 2 years, after the total population with a probability score of 0.37 or higher enters PSH. Using 0.53 as the minimum 

probability threshold for the target group, the estimated annual savings are $32,000 per person, after paying for housing and supportive services. On the other hand, using 0.20 as the probability threshold, we achieve break-even financial results, with the cost of providing housing and supportive services fully offsetting cost savings from reduced service use.

The optimal cutoff is not simply an empirical decision. In the context of PSH, it depends on the number of people who can be housed in available housing. However, in the context of a long-term strategy to address homelessness, the tradeoff between costs and savings in the population needing housing provides evidence that jurisdictions can use to validate local policy initiatives, such as affordable housing bond measures to expand the inventory of available housing.

It is often argued that the feasibility of prevention services, such as PSH, would not be attained without a strategy of balancing the costs with some degree of cost offsets. One of the most significant strengths of this study is its strong performance in identifying homeless persons with a high probability of having high ongoing public costs that will substantially exceed the cost of PSH.

The predictive performance of the Silicon Valley Triage Tool was compared to the performance of two earlier triage tools developed in Los Angeles by running all of the models on records of homeless persons from both Los Angeles and Santa Clara Counties. The tools were assessed based on the proportion of high-cost homeless persons correctly identified by each model and the proportion of persons predicted to be high-cost homeless who truly were high-cost persons. The Silicon Valley tool demonstrated comparable or higher accuracy when run on Los Angeles data and much higher 

accuracy when applied to the Santa Clara data. This comparison verifies that the Silicon Valley tool demonstrates strong predictive performance in multiple metropolitan regions.

Limitations

This analysis and the model developed in this study are also subject to some limitations that need to be acknowledged, and most of these limitations are inherent to analysis involving administrative datasets. Our study is limited by the usual shortcomings of research based on linked administrative 138 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach records, including errors in the underlying data sources, such as missing data and data entry errors. Matching inaccuracies prevented the use of the full homeless population for the analysis. The tool was developed using data for roughly 55 percent of the population that experienced homelessness, 57,259 persons. These were individuals with at least one record linked to an agency during our 6-year study window from 2007 through 2012. Since administrative databases usually are not designed to collect data for research, information about some critical risk factors is often missing. 

For example, in developing this tool, we did not have access to data about income and employ-

ment. Moreover, some service costs were missing for some years and had to be estimated. For some services, when individual-level costs were not available, average costs per unit of service were used.

Another shortcoming related to the use of administrative data is incomplete and sometimes inaccurate information about the timing of homeless episodes. Because complete information about the duration of homelessness was not available, the study population was assumed to be either homeless or at risk of homelessness while predicting high-cost users, assuming that individuals would use more services when they were experiencing homelessness. In addition, the administrative 

datasets did not show the mobility of homeless individuals in and out of the county, which would impact their utilization of services in county facilities.

The business scenario that estimated cost savings was also subject to some limitations. First, it assumed that PSH costs $17,000 a year, which needs to be verified when the county has a larger body of postsupportive housing cost data. Second, because posthousing costs of homeless persons were not available for this study, cost offsets were based on a saving factor of 68 percent, which was derived from an earlier study conducted in Los Angeles. Actual cost savings may be different after the implementation of the program. On the other hand, service reductions measured here 

represent a conservative assessment of the impact of the PSH on service use and costs because it was assumed that homeless persons with costs below the 10th decile would not experience any 

service reductions after being housed, so that PSH costs were not adjusted with any cost offsets for this group.

Finally, the Silicon Valley Tool is a system-based tool; that is, it requires detailed healthcare and justice system information about each individual that is available only from those institutional systems. This includes medical diagnoses, accurate details of encounters with healthcare providers, and details about stints of incarceration. Cooperation of both healthcare and justice system agencies is necessary to protect the privacy of personal information while providing the data required for the tool. Santa Clara County agencies agreed to authorize a research unit in the Behavioral Health Services Department to link records across county agencies and then to de-identify the linked records so that they could be used by the Economic Roundtable to develop this triage tool.

Because of the level of effort required to obtain and integrate the necessary data, the most efficient use of the tool is for regular, ongoing system-wide screening of linked records rather than screening clients individually. By predicting how likely each person in the entire identified population of homeless resident is to have high future costs, it is possible to prioritize individuals for access to the scarce supply of PSH. For example, targeted individuals can be flagged in client databases so that housing can be offered to them the next time they seek services.

Cityscape 139

Toros and Flaming

The Silicon Valley Tool can also be used to screen cases individually. A version of the tool for individual screening in Excel format as well as software code for screening entire client databases 

can be downloaded at https://economicrt.org/publication/silicon-valley-triage-tool/.

Because the tool does not correctly identify all high-cost individuals, the screening process for either individuals or groups should include an option to override the triage tool probability score based on the clinical judgment of healthcare professionals. For example, if a patient has recently been diagnosed with a high-cost, chronic medical condition, this would warrant overriding a 

negative result from the triage tool and including the patient in the high-cost group that receives access to PSH. Allowing overrides permits service providers to adapt to changing populations and conditions and to be responsive to unique circumstances.

The tool also has practical value for identifying patients served by health plans and private hospitals who have high ongoing costs, and whose health outcomes will improve and costs decrease if they are housed. Local government safety net resources can be augmented through collaborative care for frequent users who are also served by private hospitals.

Using the triage tool raises the broader ethical issue of making decisions about who gets into housing and who is left out. We see the tool as an interim means of prioritizing need in the context of an inadequate supply of affordable housing and insufficient human service interventions for reducing the flow of people into chronic homelessness. In this context, the tool prioritizes individuals based on public costs, which reflect frequency of service-intensive crises, and are closely linked to (but not identical with) level of distress. Use of the triage tool may be the approach that houses the greatest number of people because public agencies achieve the highest level of cost avoidance by housing high-cost individuals, opening the possibility of using those savings to pay for other crucial services.

Conclusion and Future Research

Needs within the homeless population vary significantly. Although the Silicon Valley Triage Tool is effective for prioritizing access to PSH for the small number of high-cost individuals who account for the majority of public costs, other tools are needed to target services for less disabled segments of the population. Less expensive interventions may be effective for individuals with less acute needs. This includes preventive care for children who have experienced homelessness, integrated outpatient healthcare, readily available and effective behavioral and mental health services, temporary affordable housing, and employment services. Without effective early intervention, the risk that individuals will become chronically homeless and that their problems will worsen to the extent that they become high-cost homeless is real. 

Acknowledgments

The authors thank the County of Santa Clara and Destination: Home for carrying out the cross-

agency linkage of records for homeless clients, for providing the funding that made this research possible, and also for their commitment to using data strategically for improving the lives of people experiencing homelessness.

140 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach Authors

Halil Toros is statistical analytics consultant at the Economic Roundtable.

Daniel Flaming is President of the Economic Roundtable.

References

Apicello, Jocelyn. 2010. “A Paradigm Shift in Housing and Homeless Services: Applying the 

Population and High-Risk Framework to Preventing Homelessness,”  The Open Health Services and Policy Journal 3: 41–52.

Ash, Arlene S., Yang Zhao, Randall P. Ellis, and Marilyn Schlein Kramer. 2001. “Finding Future High-Cost Cases: Comparing Prior Cost Versus Diagnosis-Based Methods,”  Health Services Research 36 (6 pt. 2): 194–206.

Billings, John. 2006.  Identifying High Cost Patients for Interventions to Improve Health and Social Care Services. New York: NYU Center for Health and Public Service Research. 

Billings, John, Theo Georghiou, Ian Blunt, and Martin Bardsley. 2013. “Choosing a Model To 

Predict Hospital Admission: An Observational Study of New Variants of Predictive Models for Case Finding,”  BMJ Open: 1–9. DOI: 10.1136/bmjopen-2013-003352.

Burt, Martha R. 2002. “Chronic Homelessness: Emergence of a Public Policy,”  Fordham Urban Law Journal 30: 1267–1279.

Burt, Martha R., Carol L. Pearson, and Ann Elizabeth Montgomery. 2005.  Strategies for Preventing Homelessness. Report prepared for the U.S. Department of Housing and Urban Development. 

Washington, DC: Urban Institute; Walter R. McDonald and Associates.

Byrne, Thomas, Jamison Fargo, Ann Elizabeth Montgomery, Ellen Munley, and Dennis P. Culhane. 

2014. “The Relationship Between Community Investment in Permanent Supportive Housing and 

Chronic Homelessness,”  Social Service Review 88: 234–263. DOI: 10.1086/676142.

Byrne, Thomas, Dan Treglia, Dennis P. Culhane, John Kuhn, and Vincent Kane. 2016. “Predictors of Homelessness Among Families and Single Adults After Exit From Homelessness Prevention 

and Rapid Re-Housing Programs: Evidence From the Department of Veterans Affairs Sup-

portive Services for Veteran Families Program,”  Housing Policy Debate 26: 1, 252–275. DOI: 10.1080/10511482.2015.1060249.

Caton, Carol, Carol Wilkins, and Jacquelyn Anderson. 2007. “People Who Experience Long-Term 

Homelessness: Characteristics and Interventions.” http://www.aspe.hhs.gov/hsp/homelessness/

symposium07/caton.

Caton, Carol L.M., Boanerges Dominguez, Bella Schanzer, Deborah S. Hasin, Patrick E. Shrout, 

Alan Felix, Hunter McQuistion, Lewis A. Opler, and Eustace Hsu. 2005. “Risk Factors for Long-

Term Homelessness: Findings From a Longitudinal Study of First-Time Homeless Single Adults,” 

American Journal of Public Health 95: 1753–1759. DOI: 10.2105/AJPH.2005.063321.

Cityscape 141

Toros and Flaming

Chechulin, Yuriy, Amir Nazerian, Saad Rais, and Kamil Malikov. 2014. “Predicting Patients With High Risk of Becoming High-Cost Healthcare Users in Ontario (Canada),”  Healthcare Policy 9: 68–79. DOI: 10.12927/hcpol.2014.23710.

Culhane, Dennis P. 2008. “The Cost of Homelessness: A Perspective From the United States,” 

European Journal of Homelessness 2: 97–114. http://repository.upenn.edu/spp_papers/148.

Culhane, Dennis P., and Thomas Byrne. 2010. Ending Chronic Homelessness: Cost-Effective Op-

portunities for Interagency Collaboration. Working paper. Philadelphia: University of Pennsylvania 

School of Social Policy and Practice. http://repository.upenn.edu/spp_papers/143.

Culhane, Dennis P., Stephen Metraux, and Thomas Byrne. 2011. “A Prevention-Centered Approach 

to Homelessness Assistance: A Paradigm Shift?”  Housing Policy Debate 21: 295–315. DOI:10.1080/1

0511482.2010.536246.

Culhane, Dennis P., Stephen Metraux, and Trevor Hadley. 2002. “Public Service Reductions As-

sociated With Placement of Homeless Persons With Severe Mental Illness in Supportive Housing,” 

Housing Policy Debate 13: 107–163. DOI: 10.1080/10511482.2002.9521437.

Economic Roundtable. 2015a.  All Alone: Antecedents of Chronic Homelessness. Los Angeles. DOI: 10.13140/RG.2.1.4067.9281.

———. 2015b.  Home Not Found: The Cost of Homelessness in Silicon Valley. Los Angeles. DOI: 10.13140/RG.2.1.4780.6327.

———. 2012.  Hospital to Home: Triage Tool II for Identifying Homeless Hospital Patient in Crisis. Los Angeles.

———. 2011.  Crisis Indicator: Triage Tool for Identifying Homeless Adults in Crisis. Los Angeles. DOI: 10.13140/RG.2.1.4788.8246.

———. 2009.  Where We Sleep: The Costs of Housing and Homelessness in Los Angeles. Los Angeles. 

DOI: 10.13140/RG.2.1.2624.0887.

Fleishman, John A., and Joel W. Cohen. 2010. “Using Information on Clinical Conditions 

To Predict High-Cost Patients,”  Health Services Research 45: 532–552. DOI: 10.1111/j.1475-6773.2009.01080.x.

Folsom, David P., William Hawthorne, Laurie Lindamer, Todd Gilmer, Anne Bailey, Shahrokh 

Golshan, Piedad Garcia, Jürgen Unützer, Richard Hough, and Dilip V. Jeste. 2005. “Prevalence and Risk Factors for Homelessness and Utilization of Mental Health Services Among 10,340 Patients With Serious Mental Illness in a Large Public Mental Health System,”  American Journal of Psychiatry 162: 370–376. DOI: 10.1176/appi.ajp.162.2.370.

Gilmer, Tod P., Willard G. Manning, and Susan L. Ettner. 2009. “A Cost Analysis of San Diego 

County’s REACH Program for Homeless Persons,”  Psychiatric Services 60: 445–450.

Gonen, Mithat. 2007.  Analyzing Receiver Operating Characteristics With SAS. SAS Press Series. Cary, NC: SAS Institute.

142 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach Greenwood, Ronnie Michelle, Ana Stefancic, and Sam J. Tsemberis. 2013. “Pathways Housing First for Homeless Persons With Psychiatric Disabilities: Program Innovation, Research, and Advocacy,” 

Journal of Social Issues 69: 645–663. DOI: 10.1111/josi.12034.

Gubits, Daniel, Marybeth Shinn, Michelle Wood, Stephen Bell, Samuel Dastrup, Claudia D. Solari, Debi McInnis, Tom McCall, and Utsav Kattel. 2016.  Family Options Study: 3-Year Impacts of Housing and Services Interventions for Homeless Families. Washington, DC: U.S. Department of Housing and 

Urban Development. huduser.gov/portal/publications/Family-Options-Study.html.

Henwood, Benjamin F., Howard Dichter, Robert Tynan, Christine Simiriglia, Krista Boermer, and Adam Fussaro. 2015. “Service Use Before and After the Provision of Scatter-Site Housing First for Chronically Homeless Individuals With Severe Alcohol Use Disorders,”  International Journal of Drug Policy 26: 883–886. DOI: 10.1016/j.drugpo.2015.05.022.

Hosmer, David W., and Stanley Lemeshow. 2000.  Applied Logistic Regression, 2nd ed. New York: John Wiley and Sons.

Kuhn, Randall, and Dennis P. Culhane. 1998. “Applying Cluster Analysis To Test a Typology of 

Homelessness by Pattern of Shelter Utilization: Results From the Analysis of Administrative Data,” 

American Journal of Community Psychology 26: 207–232. http://repository.upenn.edu/spp_papers/96.

Kuno, Eri, Aileen B. Rothbard, June Averyt, and Dennis Culhane. 2000. “Homelessness Among 

Persons With Serious Mental Illness in an Enhanced Community-Based Mental Health System,” 

Psychiatric Services 51: 1012–1016. DOI: 10.1176/appi.ps.51.8.1012.

Kushel, Margot B., Judith A. Hahn, Jennifer L. Evans, David R. Bangsberg, and Andrew R. Moss. 

2005. “Revolving Doors: Imprisonment Among the Homeless and Marginally Housed Population,” 

American Journal of Public Health 95: 1747–1752. DOI: 10.2105/AJPH.2005.065094.

Kushel, Margot B., Sharon Perry, David Bangsberg, Richard Clark, and Andrew R. Moss. 2002. “Emergency Department Use Among the Homeless and Marginally Housed: Results From a Community-

Based Study,”  American Journal of Public Health 92: 778–784. DOI: 10.1186/s13722-015-0038-1.

Larimer, Mary E., Daniel K. Malone, Michelle D. Garner, David C. Atkins, Bonnie Burlingham, 

Heather S. Lonczak, Kenneth Tanzer, Joshua Ginzler, Seema L. Clifasefi, William G. Hobson, and G. Alan Marlatt. 2009. “Health Care and Public Service Use and Costs Before and After Provision of Housing for Chronically Homeless Persons With Severe Alcohol Problems,”  Journal of American Medical Association 301: 1349–1357. DOI: 10.1001/jama.2009.414.

Larson, Eric B. 2013. “Building Trust in the Power of ‘Big Data’ Research To Serve the Public Good,”  Journal of American Medical Association 309: 2443–2444. DOI: 10.1001/jama.2013.5914.

Ly, Angela, and Eric Latimer. 2015. “Housing First Impact on Costs and Associated Cost Offsets: A Review of the Literature,”  Canadian Journal of Psychiatry 60: 275–287.

Martinez, Tia E., and Martha R. Burt. 2006. “Impact of Permanent Supportive Housing on the Use of Acute Care Health Services by Homeless Adults,”  Psychiatric Services 57: 1–8. DOI: 10.1176/

ps.2006.57.7.992.

Cityscape 143

Toros and Flaming

McLaughlin, Thomas Chalmers. 2011. “Using Common Themes: Cost-Effectiveness of Permanent 

Supported Housing for People With Mental Illness,”  Research on Social Work Practice 21: 404–411. 

DOI: 10.1177/1049731510387307.

McNiel, Dale E., Renee L. Binder, and Jo C. Robinson. 2005. “Incarceration Associated With 

Homelessness, Mental Disorder, and Co-Occurring Substance Abuse,”  Psychiatric Services 56: 840–846. DOI: 10.1176/appi.ps.56.7.840.

Metraux, Stephen, and Dennis P. Culhane. 2004. “Homeless Shelter Use and Reincarceration 

Following Prison Release: Assessing the Risk,”  Criminal Public Policy 3: 201–222. http://repository.

upenn.edu/spp_papers/116.

Metraux, Stephen, Dennis P. Culhane, Stacy Raphael, Matthew White, Carol Pearson, Eric Hirsh, Patricia Ferrell, Steve Rice, Barbara Ritter, and J. Stephen Cleghorn. 2001. “Assessing Homeless Population Size Through the Use of Emergency and Transitional Shelter Services in 1998: Results From the Analysis of Administrative Data From Nine U.S. Jurisdictions,”  Public Health Reports 116: 344–352. http://repository.upenn.edu/spp_papers/85.

Montgomery, Ann Elizabeth, Jamison D. Fargo, Thomas H. Byrne, Vincent R. Kane, and Dennis 

P. Culhane. 2013. “Universal Screening for Homelessness and Risk for Homelessness in the 

Veterans Health Administration,”  American Journal of Public Health 103: S210–S211. DOI: 10.2105/

AJPH.2013.301398.

Moturu, Sai T., William G. Johnson, and Huan Liu. 2010. “Predicting Future High-Cost Patients: A Real-World Risk Modeling Application,”  International Journal of Biomedical Engineering and Technology 3: 114–132. DOI: 10.1504/IJBET.2010.029654.

Poulin, Stephen R., Marcella Maguire, Stephen Metraux, and Dennis P. Culhane. 2010. “Service 

Use and Costs for Persons Experiencing Chronic Homelessness in Philadelphia: A Population-

Based Study,”  Psychiatric Services 61: 1093–1098. DOI: 10.1176/ps.2010.61.11.1093.

Rog, Debra J., Tina Marshall, Richard H. Dougherty, Preethy George, Allen S. Daniels, Sushmita Shoma Ghose, and Miriam E. Delphin-Rittmon. 2014. “Permanent Supportive Housing: Assessing 

the Evidence,”  Psychiatric Services 65: 287–294. DOI: 10.1176/appi.ps.201300261.

Rosenheck, Robert. 2000. “Cost-Effectiveness of Services for Mentally Ill Homeless People: The Application of Research to Policy and Practice,”  American Journal of Psychiatry 157: 1563–1570. 

DOI: 10.1176/appi.ajp.157.10.1563.

Sadowski, Laura S., Romina A. Kee, Tyler J. VanderWeele, and David Buchanan. 2009. “Effect of a Housing and Case Management Program on Emergency Department Visits and Hospitalizations 

Among Chronically Ill Homeless Adults: A Randomized Trial,”  Journal of the American Medical Association 301: 1771–1778. DOI: 10.1001/jama.2009.561.

Sarma, Kattamuri S. 2013.  Predictive Modeling With SAS Enterprise Miner: Practical Solutions for Business Applications. Cary, NC: SAS Institute.

144 Refereed Papers

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach SAS. 2013.  Getting Started With SAS Enterprise Miner 13.1. Cary, NC: SAS Institute.

Shinn, Marybeth, Jim Baumohl, and Kim Hopper. 2001. “The Prevention of Homelessness Revis-

ited,”  Analyses of Social Issues and Public Policy 1: 95–127. DOI: 10.1111/1530-2415.00006.

Shinn, Marybeth, Andrew L. Greer, Jay Bainbridge, Jonathan Kwon, and Sara Zuiderveen. 2013. 

“Efficient Targeting of Homelessness Prevention Services for Families,”  American Journal of Public Health 103: S324–S330. DOI: 10.2105/AJPH.2013.301468.

Tamang, Suzanne, Arnold Milstein, Henrik Toft Sørensen, Lars Pedersen, Lester Mackey, Jean-Ray-mond Betterton, Lucas Janson, and Nigam Shah. 2015.  Improving the Foundation of Population-Based Spending Arrangements by Predicting ‘Cost Blooms’ in Denmark: A Longitudinal Population-Based Study. 

Palo Alto, CA: Stanford University. http://statweb.stanford.edu/~ljanson/papers/Predicting_Pa-

tient_Cost_Blooms_In_Denmark-Tamang_ea-2016.pdf.

Toros, Halil, and Max Stevens. 2012.  Project 50: The Cost Effectiveness of the Permanent Supportive Housing Model in the Skid Row Section of Los Angeles County. Los Angeles: County of Los Angeles, CEO.

Tsemberis, Sam, and Ronda F. Eisenberg. 2000. “Pathways to Housing: Supported Housing 

for Street-Dwelling Homeless Individuals With Psychiatric Disabilities,”  Psychiatric Services 51: 487–493. DOI: 10.1176/appi.ps.51.4.487.

U.S. Department of Housing and Urban Development (HUD). 2016.  The 2016 Annual Homeless Assessment Report (AHAR) to Congress. Part 1: Point-in-Time Estimates of Homelessness. Report prepared for Congress. Washington, DC: U.S. Department of Housing and Urban Development. https://

www.hudexchange.info/resources/documents/2016-AHAR-Part-1.pdf.

———. 2017.  The 2017 Annual Homeless Assessment Report (AHAR) to Congress. Part 1: Point-in-Time Estimates of Homelessness. Report prepared for Congress. Washington, DC: U.S. Department 

of Housing and Urban Development. https://www.hudexchange.info/resources/documents/2017-

AHAR-Part-1.pdf.

———. 2016.  The 2016 Annual Homeless Assessment Report (AHAR) to Congress. Part 2: Estimates of Homelessness in the United States. Report prepared for Congress. Washington, DC: U.S. Department 

of Housing and Urban Development. https://www.hudexchange.info/resources/documents/2016-

AHAR-Part-2.pdf.

U.S. Interagency Council on Homelessness (USICH). 2016  The President’s 2016 Budget: Fact Sheet on Homelessness Assistance. Washington, DC. https://www.usich.gov/resources/uploads/asset_library/2016_

Budget_Fact_Sheet_on_Homelessness_Assistance.pdf.

———. 2015.  Opening Doors: Federal Strategic Plan to Prevent and End Homelessness. Washington, 

DC. https://www.usich.gov/opening-doors.

Cityscape 145

Toros and Flaming

———. 2010.  Opening Doors: Federal Strategic Plan To Prevent and End Homelessness. Washington, DC.

Zugazaga, Carole. 2004. “Stressful Life Event Experiences of Homeless Adults: A Comparison 

of Single Men, Single Women, and Women With Children,”  Journal of Community Psychology 32: 643–654. DOI: 10.1002/jcop.20025.

146 Refereed Papers

Scale in Housing Policy:  

A Case Study of the Potential of 

Small Area Fair Market Rents

Matthew Palm

University of Melbourne

Abstract

The U.S. Department of Housing and Urban Development (HUD) caps subsidies for 

Section 8 housing vouchers using limits known as the Fair Market Rents (FMRs). HUD 

recently implemented Small Area Fair Market Rents (SAFMRs), based on ZIP Code-

level rents, to improve options for voucher recipients in high-opportunity areas. I use a proprietary dataset of for-rent listings to test the ways in which SAFMRs would change 

the number of listings below FMR across five California HUD metropolitan FMR 

areas—Oakland-Fremont, Sacramento--Roseville--Arden-Arcade, San Diego-Carlsbad, 

San Francisco, and San Jose-Sunnyvale, Santa Clara. I examine local housing authori-

ties’ concerns regarding the SAFMRs. I find the SAFMRs will increase the number of 

listings below FMR in high-opportunity neighborhoods across each area studied except 

San Francisco. I confirm Oakland housing authorities’ concerns that the SAFMRs would 

reduce the number of units below FMR in areas with rapidly rising rents. I find that 

Sacramento and San Diego may benefit most from the SAFMRs among those studied. 

These findings validate HUD’s criteria for identifying areas in which to implement the 

SAFMRs, as Sacramento and San Diego are also the only two areas among the case 

studies in this article that HUD initially approved for SAFMRs implementation. The 

SAFMRs highlight the importance of geographic scale in housing policy implementation. 

Introduction

In select metropolitan areas, HUD recently implemented a new way of defining Fair Market Rent (FMR), the subsidy limits for Section 8 housing vouchers.1 This change, called the Small Area 1 “Establishing a More Effective Fair Market Rent System; Using Small Area Fair Market Rents in Housing Choice Voucher Program Instead of the Current 50th Percentile FMRs.” Final Rule. 24 CFR Parts 888, 982, 983, and 985.  Federal Register 81 (221) November 16, 2016.

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 147

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Palm

Fair Market Rent (SAFMR), shrinks the geographic scale at which HUD calculates these voucher 

maximums from the metropolitan scale, known as the HUD metro FMR area, to the smaller ZIP 

Code-related geography. HUD implemented this rule to encourage voucher recipients to relocate into high-opportunity areas. Recent research finds that children who grow up in high-opportunity communities are more likely to experience upward social mobility (Chetty, Hendren, and Katz, 

2016). The results of an SAFMR pilot program in Texas suggest that this policy scale change does improve voucher holders’ locations (Collinson and Ganong, 2013). However, advocates and public housing authorities (PHAs) in some areas objected to the SAFMRs, arguing the American Community Survey (ACS) data used to define FMRs did not keep pace with rapidly rising rents. They also noted the SAFMRs did not consider the lack of vacancies in some markets, or the ways SAFMRs could 

reduce housing options for residents in gentrifying communities (Johnson, 2016b; Levin, 2016).

In this study, I examine two aspects of the SAFMRs. First, I test the ability of SAFMRs to increase the number of listings below the voucher payment limits (henceforth, “below FMR”) in high-opportunity neighborhoods. Second, I test the concerns of some PHAs regarding the way in which SAFMRs may exacerbate the impacts of rising rents and reduce the number of below-FMR units in markets with tight vacancy rates. I utilize a proprietary database of for-rent listings covering five HUD metropolitan FMR areas in California—Oakland-Fremont (henceforth, Oakland), Sacramento--Roseville--Arden-Arcade (Sacramento), San Diego-Carlsbad (San Diego), San Francisco, and 

San Jose-Sunnyvale-Santa Clara (San Jose)—from 2012 and 2013 to model these potential impacts of the SAFMR. As some SAFMR critics’ concerns involve rapidly rising rents, I also draw on data from the same proprietary source for 2011 and 2014–2015 to contextualize my results within each area’s market trends. Because the use of web-based rental listings to study housing affordability is new, I also present a set of results comparing my analysis with those of the only other study to examine FMRs using web-based rental listings, conducted by Boeing and Waddell (Boeing and 

Waddell, 2016).

I find that the SAFMRs would increase the share of below-FMR listings in high-opportunity areas in four of the five HUD metropolitan FMR areas studied, the exception being San Francisco. I note that San Francisco’s unique results are a function of rents increasing faster than the SAFMRs in that area’s expensive ZIP Codes. I also identify a pattern unique to Oakland—the SAFMRs would exacerbate the loss of below-FMR units in lower rent ZIP Codes where rents are significantly increasing, reconfirming local PHAs’ concerns (Johnson, 2016b). I fail to find a relationship between ZIP Code vacancy rates and the potential impact of the SAFMRs, suggesting the SAFMRs may not reduce the overall share of units below FMR. 

This article contains five sections. I begin with a background to provide a more detailed explanation of FMRs, SAFMRs, and theoretical motivations behind the SAFMR approach. In the following section, a literature review examines the many obstacles facing voucher recipients, noting a lack of attention among scholars for the role that market rents play in limiting voucher holders’ choices. 

I also highlight the data limitations of the few studies that do consider market constraints. I then detail local agencies’ concerns with the SAFMRs and present several hypotheses based on this 

review. A data section and methods section follows in which I note the limitations of rental listings data. My results and conclusions highlight the implications of the SAFMRs for each of the metropolitan areas studied in this article.

148 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents Background

Vouchers enable participating households to pay only 30 percent of their income on rent, with the remaining rent subsidized by the federal government up to FMR limits (HUD, 2017a). HUD calculates FMRs for thousands of geographies known as HUD metropolitan FMR areas, geographies 

that can encompass between 50,000 and several million residents. HUD began using the ACS to 

measure FMRs after a report from the U.S. Government Accountability Office called for greater transparency and consistency in FMR implementation. The report detailed the need for a cost-effective approach to calculating FMRs that would be consistent across jurisdictions and over time and transparent enough that concerned local housing authorities could replicate and thus verify proposed FMRs (GAO, 2005). HUD responded to the report by adopting the ACS for its FMR 

calculations and by developing a completely open-access FMR documentation system available 

online (HUD, 2017a).

To calculate the FMRs, HUD starts by estimating the 40th percentile of the distribution of gross rents in each metropolitan FMR area, by number of bedrooms, using the latest available ACS data (HUD, 2017b). Gross rents include both the contracted rent for housing and the utilities paid by the household, meaning that HUD intends for FMRs to cover utilities and rent. HUD relies on the latest 5-year rolling sample of the ACS to produce these estimates (U.S. Census Bureau, 2016a). If recent movers in the ACS report significantly higher rents than the 40th percentile, then HUD adjusts the 40th percentile rent proportionally upward. HUD uses the latest 1-year sample of the ACS to make these recent mover adjustments, bringing the data inputs closer to the time of policy application. 

HUD must publicly post FMRs for thousands of jurisdictions several months prior to the year of their implementation to provide time for PHAs and other stakeholders to comment on the proposed limits. HUD usually posts the FMRs in September of the prior year (NLIHC, 2016a). By that time, however, the latest ACS data available are usually 2 years old. To account for this lag, HUD 

applies trend and inflation factors to the initial ACS-derived limits. Although this method accounts for time factors, spatial challenges remain. The traditional FMRs are frequently insensitive to inter-neighborhood variations in rental market conditions. HUD’s calculation of the FMRs covering large geographies creates a mismatch between local markets and regional statutory voucher limits. 

The SAFMRs address this problem. To calculate the SAFMRs, HUD first calculates a ratio—the 

median rent of a ZIP Code more than the median rent of its respective metropolitan statistical area (MSA). Suppose this ratio equals 110 percent for a given ZIP Code, because this ZIP Code is 10 

percent more expensive than its overall MSA. HUD multiplies this ratio to the existing FMRs to calculate the SAFMR for that specific ZIP Code. This SAFMR now better reflects the rental market in this more expensive ZIP Code. Voucher recipients in that ZIP Code can now rent units 10 

percent more expensive than those they could previously rent. HUD caps SAFMRs at 150 percent 

of their respective FMRs. 

Geographic theory suggests this change should improve the ability of housing voucher recipients to disperse evenly across metropolitan areas. Tobler’s First Law of Geography holds that near things are more related than far things (Miller, 2004). Housing is no exception. Rental and land markets are highly autocorrelated across space, with spatially oriented approaches like geographic fixed Cityscape 149

Palm

effects and geographically weighted regression often better predicting rents and home prices than nonspatial statistical techniques in multiple studies (Löchl and Axhausen, 2010; McCord et al., 2014; Tu, Sun, and Yu, 2007). This spatial autocorrelation suggests that the rents of a unit’s neighbors can more easily predict that unit’s rent than similar units much farther away. The SAFMRs embodies this principle.

HUD recently rescinded the mandatory implementation of the SAFMRs in response to mounting 

uncertainty about its programmatic costs (NLIHC, 2017). Advocates successfully challenged that decision in court, and it appears the SAFMRs will take effect (Cohen, 2018).2 Some critics argued that the SAFMRs would reduce the number of households supported by vouchers, because it 

would raise the cost of the program without providing additional revenue to offset those costs (NAHB, 2017). When considering the FMR, HUD must also contend with the fact that “flat rents” 

for public housing tenants are also derived from the FMR (HUD, 2015). Flat rents are alternative affordable rents for public housing tenants that are higher than income-based rents but do not rise if the tenants’ income rises. Some jurisdictions have implemented flat rents as an incentive for public housing tenants to find employment (SAMHSA, 2016). The link between the flat rent and 

FMRs creates a separate set of stakeholders, public housing tenants in flat rent jurisdictions who would be adversely impacted by rising FMRs. 

Unfortunately, rental prices and FMRs are only one of many factors constraining voucher holder’s location choices. The following literature review details these challenges.

Literature Review

Researchers first identified the failure of housing vouchers to induce recipients to move into high-opportunity neighborhoods early in the life of the program (Newman and Schnare, 1997). More 

recent work confirms that this pattern continues in voucher location outcomes (Galvez, 2011), with one study suggesting voucher holders actually began reconcentrating in low-opportunity 

areas in the past decade (Metzger, 2014). 

The literature identifies several causes of these patterns. One recent review of the evidence finds voucher holders lack support in the process of searching for housing, face lawful landlord discrimination against the use of vouchers as a source of income, and have limited social networks that constrain their capacity to relocate (Galvez, 2010). Perhaps as a result of these factors, voucher recipients who move involuntarily generally move to within 3 miles of their prior homes (Goetz and Chapple, 2010; Kleit and Galvez, 2011). The factors identified by Galvez (2010) may also 

help explain the reasons voucher holders who do relocate to the suburbs sometimes do so by 

renting in affordable housing sites built by the country’s supply-side housing programs, such as the Low-Income Housing Tax Credit Program (Wang and Varady, 2005). If social networks are linked 

to one’s race and class, the social network factor may explain the persistently unequal locational outcomes between White and non-White voucher recipients, although these trends appear to have improved slightly (McClure, Schwartz, and Taghavi, 2014). Reliance on public transportation 

may also limit the relocation options of voucher recipients who lack vehicles (Ruel et al., 2013). 

2 As of January 30, 2018.

150 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents The culmination of these challenges is so severe that even offering voucher recipients additional financial support if they agree to “move to opportunity” often fails to induce relocations into areas of higher opportunity (Schwartz, Mihaly, and Gala, 2016). 

This body of literature largely ignores the role of market forces, or the role of the FMRs, in curtail-ing voucher recipients’ ability to penetrate higher opportunity neighborhoods. This literature offers no obvious insight into the ramifications of the SAFMRs, except to suggest it does not directly address many of the challenges voucher recipients face. However, the literature does demonstrate that even if the SAFMRs make voucher use in high-opportunity neighborhoods a financial possibility for participating families, they may not make it more likely to happen. This pattern will be particularly evident in states where source-of-income discrimination is legal, and landlords can lawfully refuse to rent to voucher holders (Tighe, Hatch, and Mead, 2016). As I discuss in the next section, the thin body of work that tackles market constraints relies on data that may be insufficient for properly assessing the role of market constraints.

Data Challenges in Unpacking Market Constraints

Researchers studying FMRs consistently find units renting below FMRs exist in nearly every census block group across the United States (McClure, 2014; McClure, Schwartz, and Taghavi, 2014). 

McClure also finds, however, that units below statutory FMRs are disproportionately in higher poverty neighborhoods. HUD references this finding in its rationale for the SAFMRs.3 

Studies relying on the ACS, or long-form census, must contend with the challenge of data lag. How relevant and representative can years-old data be to voucher recipients who face rental markets in constant flux? Little research considers the means by which vacancies, seasonal factors, and rapidly changing market conditions might bias ACS-reliant analysis. Rapidly changing markets matter 

greatly in areas with rent control. For example, a unit captured in the ACS data or long-form census by McClure (2014) may be voucher accessible, simply because rent control applies and the rent observed is far lower than it would be if the unit returned to the market. Some researchers have attempted to measure the impact of market restraints on voucher relocation decisions with ACS median rents (Kleit and Galvez, 2011). This approach includes an additional drawback. The ACS tract-level median must represent market forces for units of all sizes, when in fact the FMRs vary based on the number of bedrooms, and these variations can fluctuate by metropolitan area. 

Several papers have addressed the challenge of estimating the effect of the FMR by selecting a unit size with an FMR threshold that closely mirrors an ACS cross tabulation. For example, in a case study area where the FMR for a two-bedroom was $979, researchers used ACS tabulations 

that count the number of units in a tract renting up to $999 a month as a proxy for below-FMR 

counts (Cunningham and Droesch, 2005; Horn, Ellen, and Schwartz, 2014). This improvement 

comes with the trade-off of smaller ACS samples representing market conditions in these tracts. 

This method relies not on the small ACS tract samples but on even smaller sub-samples within 

them. Unfortunately, the switch from a long-form census to the 5-year ACS wave complicates this approach for the future, as 5-year wave estimates are highly unstable and potentially biased across space at the tract scale (Bazuin and Fraser, 2013; Folch et al., 2014).

3 Final rule.

Cityscape 151

Palm

One recent study offers insights by examining the FMR eligibility of rental listings posted on Craigslist (Boeing and Waddell, 2016). Compared with previous studies and HUD expectations, Boeing and Waddell found fewer below-FMR units in many of the country’s most expensive markets. This significant difference suggests the need for further analysis of market constraints utilizing different kinds of data. Market studies conducted in Oakland, for example, led to significant upward revisions in FMRs there (Johnson, 2016a). Those findings, placed alongside Boeing and Waddell’s estimates, imply that Craigslist may be more accurate than the ACS in capturing present market conditions.

Another recent study by Geyer (2017) also models the role of the market in explaining vouchers recipients’ locational patterns. Geyer relies on listings of below-FMR units to build a model of voucher holders’ residential location choices. She uses this model to predict the way various policy changes will affect the locations of voucher recipients. She finds that neighborhood-scaled housing voucher limits would be “both more effective at moving households to neighborhoods with lower poverty rates, and less expensive to implement, than a policy that increases the maximum voucher amount by 20 percent” (Geyer, 2017: 58). 

Data and Local Pushback Against the SAFMR

When HUD proposed the SAFMR, numerous local PHAs and advocacy organizations opposed the 

proposal, often citing data concerns in their criticisms. 

A coalition of the PHAs in Oakland, one of my case study areas, commented that the existing 

evidence on the SAFMRs is limited to one HUD metropolitan FMR area, Dallas, TX, where vacancy rates are significantly higher when compared with rates in California (Johnson, 2016b). The 

letter argues that the SAFMRs would limit the purchasing power of vouchers in less expensive 

neighborhoods and thus reduce the overall number of units listed below FMRs. In a separate 

comment submitted to HUD, the Oakland Housing Authority also argued that the inadequacies of 

the ACS would worsen the situation for voucher recipients in rapidly gentrifying neighborhoods (Johnson, 2016a). A coalition of East Bay Housing Organizations, also in Oakland, emphasized 

their concerns about the lagging nature of the ACS preventing FMRs from keeping pace with 

the market (Levin, 2016). Lastly, the Santa Clara County Housing Authority also questioned the accuracy and relevance of ZIP Code-scaled FMRs. They argued it would harm voucher recipients 

who found housing in some high-income census tracts that are sandwiched inside lower rent ZIP 

Codes (Harsasz, 2016).

HUD responded to these and other concerns by allowing the SAFMRs to apply only in areas that 

met two conditions.

1. The metropolitan FMR area has a vacancy rate above 4 percent.

2. The metropolitan FMR area has at least 20 percent of its below-FMR stock in ZIP Codes with SAFMRs above 110 percent of their respective metropolitan FMRs.4

4 Final rule.

152 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents These stipulations might ensure a sufficient supply of below-FMR stock in these high-end ZIP 

Codes can offset the loss of below-FMR units in less expensive ZIP Codes. These changes address concerns related to overall vacancies and availability issues. They do not eliminate the possibility that lower vacancies in expensive ZIP Codes might mean the SAFMRs will reduce the total number of below-FMRs vacancies across areas. They also do not address the underlying issue of HUD’s 

reliance on time-lagged ACS data and its potential impacts to voucher users in gentrifying communities. These criticisms and HUD’s response inform the hypotheses presented in the following section. No prior empirical work with rental listings data explores these challenges with FMR 

formulation and their implications for enabling voucher recipients to penetrate the market. 

Hypotheses

The primary purpose of this article is to test the potential for the SAFMRs to increase the number of units below FMRs in high-opportunity areas. The article utilizes a proprietary dataset of for-rent listings derived from online sources that do not face the drawbacks of the ACS at small geographies. To that end, the first hypothesis is—

1.  The SAFMRs will increase the percentage of listings that are below FMR in high-opportunity areas.

Critics of the SAFMRs suggest the policy will not increase the number of below- FMR units in 

markets with tight vacancy rates. They argue it could reduce the overall share of units below FMR. 

Critics also note the SAFMRs may exacerbate the impacts of gentrification on voucher recipients due to the time lag between data collection and policy implementation. To assist in clarifying the ways these issues interact with the SAFMRs, I test three additional hypotheses.

2. A switch to the SAFMRs will reduce the overall share of listings below FMR in each HUD 

metropolitan FMR area.

3. At the ZIP Code level, vacancy rates will correlate positively and significantly with SAFMR-induced increases in listings below FMR. 

4. Areas that lose below- FMR listings under the SAFMRs are also those experiencing the most 

rapid loss in below-FMR units under the existing FMR system (presumably, from gentrification). 

The fourth hypothesis tests the idea that areas that saw below-FMR units vanishing due to gentrification were also those which would see a reduction of below-FMR listings under the SAFMRs, as suggested by several comments to HUD (Johnson, 2016a; Levin, 2016).

I selected proprietary rental listings data for this study in light of the many pitfalls of relying on ACS data noted in the literature review. Concerns do exist about the value of rental listings for policy analysis, however. The only study utilizing listings to measure FMRs in the market found strikingly different results than ACS-based estimates in the areas studied in this article (Boeing and Waddell, 2016). To further our understanding of this emerging source of data, I compare my data’s findings in these areas with both Boeing and Waddell’s and the ACS to test a final hypothesis.

5. Proprietary rental listings data will more closely reflect Boeing and Waddell’s (2016) Craigslist findings on the share of units below FMR than findings derived from the ACS.

Cityscape 153

Palm

Data and Methods

I use a rental database prepared by Rent Jungle. The database contains information on rental 

listings gathered from internet sources such as Craigslist, as well as the web-listings provided by newspapers, other proprietary gatherers, property management companies, and community web 

pages. Rent Jungle uses a web-based tool to extract the data from each of these sources once a week on an automated basis and compiles them into a database on a central server. The dataset thus provides a weekly snapshot of all rental listings posted online or in print in each market. This snapshot also provides the location, number of bedrooms, and number of bathrooms for each of 

these listings. In total, this comprehensive for-rent database has more than 150,000 listings across the five HUD metropolitan FMR areas for 2012 and 2013. 

Limitations

The potential statistical power of such a large sample does not make this dataset immune from criticism. This dataset faces several limitations. First, it measures contract rent, although HUD sets FMRs using gross rent, which includes utilities. The Rent Jungle data do not specify which listings include utilities and which do not. Second, the data do not capture the vacancy rate of listings in multifamily buildings. This missing information means the dataset may underestimate rental availability on very large multifamily properties. For example, the data might show a large apartment complex listing a single, two-bedroom unit every week from June through October. This listing will appear in the dataset as one consistently listed two-bedroom unit. In reality, however, the building may have three or four different two-bedroom units becoming available throughout that time. Reflected in the data, however, is only that one consistent listing at the same address with the same number of bedrooms during that time span. This example illustrates the third and final drawback, which is that the dataset does not uniquely identify each available unit. Because of this disadvantage, this study can only comment on the general impacts of the SAFMRs across each area studied but not the precise magnitudes of such impacts. 

These limitations also illustrate why these data, although still helpful for academic research and this particular study, may be inappropriate for use by HUD in setting FMRs. Beyond these technical limitations, proprietary data may not fit the criteria set by GAO (2005) that the FMR documentation system be transparent, reproducible, and consistent across geographies. 

Identifying Unique Listings

Boeing and Waddell (2016) take advantage of Craigslist’s use of a unique identifier to pinpoint and track individual listings, thereby removing duplications of the same listing in their dataset. My proprietary listings do not offer such an easy solution. However, my interest in the rental market is relatively straightforward—I need the inventory of available rentals from which aspiring voucher tenants could choose in a given year. Toward this goal, I set up a series of heuristics to identify unique listings. First, I assigned a unique observation identification number for every listing with a unique combination of the following—an address, number of bedrooms, and year of listing 

(2012 versus 2013). If the data listed one of these unique observations twice within a minimum 6-month time span, I split it into two unique listings, 6 months apart, to account for the extended 154 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents availability and the possibility of multiple listings therein. In those cases in which a unit was listed in both 2012 and 2013, with its availability consistent but shifting from 2012 into 2013, I allowed for the unit to count once in each year if the total span of its availability was greater than 6 months (for example, if it was available from September 2012 through April 2013). These heuristics 

reduced the total sample from 150,000 to 95,868 units. The final culled listings dataset provides an average of 240 observations per ZIP Code, still far higher than the number of new renters most likely provided by the ACS, roughly an average of 44 observations per census tract.5

Addressing hypotheses four and five requires contextualizing the 2012–2013 analysis within 

broader trends of rental market change. To identify these trends, I drew on the same Rent Jungle database to procure listings from 2011 through 2015 and applied the same heuristics to these data to ensure consistency. 

The dataset covers five HUD metropolitan FMR areas in California—Oakland, Sacramento, San 

Diego, San Francisco, and San Jose—which are all consistently ranked among the most expensive rental markets in the United States (NLIHC, 2016b). 

Applying FMRs and SAFMRs

I classified each rental listing as above or below established FMRs based on tables provided on HUD’s website (HUD, 2017a). I then repeated the same step using the hypothetical SAFMRs HUD 

also posts online. I revised the hypothetical SAFMRs to be no less than 90 percent of the previous year’s FMR, per HUD’s implementation of the rule.6 This step required estimating the revised 

SAFMRs for 2012 to produce the final SAFMRs for the observations in 2013. Thus, the results of this study present an average impact of the SAFMRs during 2 years of consecutive implementation. 

Defining High Opportunity

HUD’s final rule issuing the SAFMRs refers specifically to increasing voucher holder presence in high-opportunity neighborhoods. HUD characterizes high-opportunity neighborhoods as those 

with low-poverty rates and access to good schools.7 I define opportunity in terms of neighborhood poverty rate, drawing on tract poverty rates from the 2010–2014 ACS. I define school quality 

using an average of the California Academic Performance Index (API) score of the three elementary schools nearest to each tract. I select this source as a previous study on the ability of voucher recipients to access high-performing schools also draws on the API (Basolo, 2014).

5 Although the ACS does not publish sample sizes for small scales, using state-level sample sizes, we can deduce that the 2014–2010 ACS contains roughly 390 responses per ZIP Code in California to represent the total housing stock (U.S. 

Census Bureau, 2016b). Because roughly one-half of California households rent, the ACS thus probably averages 195 rental units surveyed per ZIP Code. As roughly 12 percent of renters move annually (U.S. Census Bureau, 2015), then we can estimate the ACS contains around 24 new movers per ZIP Code in California compared with our sample of nearly 240 

listings per ZIP Code.

6 Final rule.

7 Final rule.

Cityscape 155

Palm

Results

This section presents results by hypothesis, starting with the first—that the SAFMRs will increase the percentage of listings that are below FMRs in high-opportunity areas. Exhibit 1 presents the results with respect to tract-level poverty rates.

In tracts with poverty rates of less than 10 percent, the SAFMRs increase the percent of listings below FMR in every HUD metropolitan FMR area except San Francisco, where they decline slightly. 

Increases are the most dramatic in Sacramento, with a 34-percentage-point increase in below-FMR 

listings, and San Diego, with a 28.1-percentage-point increase in these low-poverty tracts. In tracts with poverty rates of more than 40 percent, in contrast, a sharp drop in below-FMR units occurs in the Oakland market of 18 percentage points. In the other four areas, however, the below-FMR 

share in high-poverty tracts declines between 0 and 11 percentage points. With the exception of San Francisco, these results suggest the SAFMRs will improve the number of listings below FMR in low-poverty neighborhoods. Exhibit 2 presents the results for the average API score of nearby schools.

In tracts near schools with the highest API scores, the percent of listings below FMR increases in every jurisdiction. These increases range from 1 percentage point in San Francisco to 38 percentage points in San Diego. In tracts near schools with the lowest API scores, the percent of listings below FMR decline modestly, from 0.6 percentage points in San Diego to 11 percentage points in San Jose. 

Exhibit 1

Percent of Listings Below FMR by Area, Tract Poverty Rate, and FMR System

Tract Poverty  Current FMR 

SAFMR  

Change  

HUD Metropolitan FMR Area

Rate (%)

(%)

(%)

(%)

Oakland-Fremont, CA

0–10

23.2

43.1

19.9

10–20

37.3

49.5

12.2

20–30

72.5

71.4

– 1.1

30–40

77.6

68.0

– 9.6

40 or more

64.2

45.3

– 18.9

Sacramento--Roseville-- 

0–10

40.1

74.3

34.2

Arden-Arcade, CA

10–20

73.4

82.5

9.1

20–30

90.3

83.5

– 6.9

30–40

93.1

86.0

– 7.0

40 or more

89.0

82.1

– 6.9

San Diego-Carlsbad, CA

0–10

31.0

59.1

28.1

10–20

56.6

64.8

8.2

20–30

79.2

71.9

– 7.3

30–40

68.5

74.2

5.7

40 or more

82.7

71.6

– 11.2

San Francisco, CA

0–10

16.5

13.8

– 2.7

10–20

17.7

10.6

– 7.2

20–30

16.7

6.2

– 10.6

30–40

16.2

10.0

– 6.2

40 or more

38.8

35.9

– 2.9

San Jose-Sunnyvale-Santa Clara, CA

0–10

19.8

29.5

9.7

10–20

40.8

34.8

– 6.0

20–30

59.1

40.9

– 18.2

30–40

68.3

46.2

– 22.1

40 or more

75.0

75.0

0.0

FMR = Fair Market Rent. HUD = U.S. Department of Housing and Urban Development. SAFMR = Small Area Fair Market Rent.

156 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents Exhibit 2

Percent of Listings Below FMR by Area, School API Score, and FMR System

School API 

Current FMR 

SAFMR  

Change  

HUD Metropolitan FMR Area

Scores

(%)

(%)

(%)

Oakland-Fremont, CA

Less than 770

68.7

65.6

– 3.0

770–820

50.4

60.9

10.5

820–870

36.2

47.7

11.4

870–915

14.6

36.9

22.4

915 or more

12.5

40.8

28.3

Sacramento--Roseville-- 

Less than 770

87.6

79.4

– 8.2

Arden-Arcade, CA

770–820

74.9

80.8

5.9

820–870

68.2

83.7

15.5

870–915

37.9

77.1

39.2

915 or more

28.6

61.1

32.5

San Diego-Carlsbad, CA

Less than 770

71.3

70.8

– 0.6

770–820

74.8

71.5

– 3.3

820–870

62.0

67.1

5.1

870–915

40.9

61.7

20.8

915 or more

16.4

54.7

38.3

San Francisco, CA

Less than 770

15.0

9.2

– 5.8

770–820

17.8

8.3

– 9.4

820–870

16.6

11.8

– 4.8

870–915

18.6

17.4

– 1.3

915 or more

13.8

14.8

1.0

San Jose-Sunnyvale-Santa Clara, CA Less than 770

38.4

27.2

– 11.2

770–820

47.0

42.3

– 4.7

820–870

22.0

28.2

6.2

870–915

33.4

47.2

13.8

915 or more

14.5

22.5

8.0

API = Academic Performance Index. FMR = Fair Market Rent. HUD = U.S. Department of Housing and Urban Development. 

SAFMR = Small Area Fair Market Rent.

The overall rate of declines in exhibit 2 are lower than in exhibit 1, as many neighborhoods with significant losses of below-FMR listings under SAFMRs were near schools that did not contain API scores in the dataset at the time of this analysis, a surprising finding. Taken together, however, the results on poverty rates and school quality demonstrate the SAFMRs will increase the percentage of listings that are below FMRs in high-opportunity areas, as expected, with the notable exception of San Francisco.

Expanding on Hypothesis One: Why Is San Francisco Different?

The SAFMRs fail to raise the share of below-FMR units in San Francisco in this time period, 

because the SAFMRs did not fully match stratospheric rents in that area’s expensive ZIP Codes. To assess this failure properly, I limited the dataset to units in ZIP Codes with SAFMRs at 110 percent or above traditional FMRs. I then measured the percentage of listings in these high SAFMR ZIP 

Codes that were below their related SAFMRs and plotted the difference across HUD metropolitan FMR areas, shown in exhibit 3. Thus, the percentages in exhibit 3 represent the percent of listings a voucher recipient could afford under the SAFMRs in only the high-SAFMR ZIP Codes.

In San Francisco, 25 percent of listings in expensive ZIP Codes would have fallen below FMRs 

under the hypothetical SAFMRs in 2012. This decrease means that the SAFMRs would have 

enabled voucher holders to afford only 25 percent of listings in the ZIP Codes in which maximum Cityscape 157

Palm

Exhibit 3

Percent of Listings in High-SAFMR ZIP Codes Below the SAFMRs, 2012 and 2013

SAFMR = Small Area Fair Market Rent.

rents would have risen under SAFMR policy. Most listings in expensive ZIP Codes in Sacramento and San Diego, in contrast, would have fallen below FMRs under the same scenario, suggesting the SAFMRs would have significantly increased the number of listings affordable to voucher holders in expensive ZIP Codes in those cities. In all three Bay Area markets, a temporal pattern appears to emerge. These percentages would have declined by roughly one-half across the Bay Area from 2012 to 2013 under the SAFMRs, implying the SAFMRs would not have kept pace with rising 

rents in these ZIP Codes.

As the dataset acquired for this project spans 5 years, I use the same 5-year timeframe to plot the ways in which the statistics presented in exhibit 3 change across all five HUD metropolitan FMR 

areas during 5 years. Exhibit 4 shows the changes. During this period, the SAFMR system would Exhibit 4

Percent of Listings in High-SAFMR ZIP Codes Below the SAFMRs, 2011–2015

SAFMR = Small Area Fair Market Rent.

158 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents not have kept pace with observed rents in the San Francisco Bay Area. Exhibit 4 illustrates the concerns of the East Bay PHAs that FMRs do not rise at the same pace as rents increases in that area (Johnson, 2016b). These findings could result from HUD’s decision to limit SAFMRs to no 

more than 150 percent of traditional FMRs. I replicated this analysis under hypothetical SAFMRs that could rise above the 150 percent cap and found very similar results. Only 3 percent of listings in more expensive ZIP Codes saw their SAFMRs increase with the 150 percent cap removed.

The data suggest that the Bay Area is experiencing rapid rent increases that are particularly puni-tive to voucher recipients, regardless of the FMR system adopted. The failure of these higher-end SAFMRs to make a difference may explain the results for hypothesis two, discussed in the next subsection.

Hypothesis Two: SAFMRs Will Reduce the Overall Share of Listings Below FMR

Only in San Francisco do the SAFMRs reduce the overall share of listings below FMR, from 16.5 

to 11.9 percent. In the other four areas, the share of listings below FMR increases, as shown in exhibit 5. In Oakland, Sacramento, and San Diego, a sufficient number of listings become below FMR in high-cost ZIP Codes to cancel out the loss of below-FMR units in less expensive ZIP 

Codes. In San Jose, the SAFMRs break even against the existing FMRs.

The increases range from only 2 percentage points in San Jose to nearly 15 percentage points in Oakland. These results suggest the SAFMRs as implemented in the HUD rule will not cause a 

cataclysmic loss of below-FMR units in its first 2 years of implementation. Given that HUD’s ruling allows for annual stepwise reductions in SAFMRs in low-rent areas, however, these results do not offer insights on the long-term impact of the SAFMRs on the overall availability of below-FMR units. 

A major potential drawback of the analysis thus far is the potential for real vacancy rates to bias the picture painted in this dataset. I address this problem in the next subsection.

Exhibit 5

Share of Listings Below FMR by Area and FMR System, 2012–2013

HUD Metropolitan FMR Area 

Current FMR (%)

SAFMR (%)

Change (%)

Oakland-Fremont, CA 

34.9

49.7

14.8

Sacramento--Roseville--Arden-Arcade, CA 

69.6

80.6

10.9

San Diego-Carlsbad, CA 

52.9

64.7

11.8

San Francisco, CA 

16.5

11.9

– 4.6

San Jose-Sunnyvale-Santa Clara, CA 

30.6

32.6

2.0

FMR = Fair Market Rent. HUD = U.S. Department of Housing and Urban Development. SAFMR = Small Area Fair Market Rent.

Hypothesis Three: Vacancy Rates Will Correlate Positively With the SAFMR’s Effect

To test this hypothesis, I limited my analysis to only those ZIP Codes with rental vacancy rates significantly different from zero in the ACS, which reduced the data 50 percent. I also limited the analysis to ZIP Codes with more than 40 rental listings in my dataset, which did not further alter the sample size. With or without these adjustments, however, my results are the same. No apparent relationship exists between SAFMR shift and ZIP Code vacancy rates. I plot the relationship of these two variables across the 243 ZIP Codes with sufficient data in exhibit 6. 

Cityscape 159

Palm

Exhibit 6

ZIP Code Vacancy Rates Plotted Against the Percent Change in Below-FMR Listings 

From the SAFMR

FMR = Fair Market Rent. SAFMR = Small Area Fair Market Rent.

Although regression analyses failed to specify a significant relationship, visual inspection of exhibit 6 offers some insights. All the ZIP Codes that had a 60–percentage-point increase or greater in below-FMR listings under the SAFMRs reported vacancy rates of less than 5 percent. ZIP Codes with vacancy rates of greater than 10 percent either saw a modest decline in below-FMR units 

under the SAFMRs or were unaffected. 

If a significant relationship exists between vacancy rates and the impact of the SAFMRs at these smaller geographies, the SAFMRs could severely restrict the number of units voucher recipients could afford. Fortunately, nothing in this study suggests this result is a major possibility, although vacancies from rigorous market studies could paint a different picture than the flawed ACS 

estimates used here. Regardless, HUD’s decision to limit implementation of the SAFMRs to areas with vacancy rates above 4 percent offers some assurance that vacancy rates may not be a major challenge for voucher recipients in areas using the SAFMRs.

Hypothesis Four: Areas That Will Lose Below-FMR Listings Under the SAFMRs 

Are Those Experiencing the Most Rapid Loss in Below-FMR Units 

To test this hypothesis, I categorized all units into two geographic groups—those in ZIP Codes in which the SAFMRs were above the FMR, thus prospectively increasing voucher holders’ rental opportunities in that ZIP Code, and those in ZIP Codes in which the SAFMRs dropped below the 

FMR, and voucher holders would have fewer options. I then measured the actual loss of below-

FMR listings under the existing statutory FMRs over time and bifurcated these results into these two geographic groups, producing exhibit 7.

160 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents

Exhibit 7

Percentage-Point Shift in Share of Below-FMR Listings, 2011–2015, Under Existing 

FMRs, by Impact of SAFMRs 

FMR = Fair Market Rent. SAFMR = Small Area Fair Market Rent.

Exhibit 7 shows that in Oakland, those ZIP Codes that would lose below-FMR listings under the SAFMRs were already experiencing an 11-percentage-point drop in below-FMR listings under 

existing policy. In contrast, ZIP Codes that would see uplift in below-FMR units under the SAFMRs witnessed only a 6-percentage-point drop in below-FMR listings. These results only reconfirm the concerns of activists and PHAs in Oakland (Johnson, 2016a, 2016b; Levin, 2016). Exhibit 7 suggests this dynamic of the SAFMRs aggravating existing market trends would not have occurred in San Diego, San Francisco, and San Jose. 

The other remarkable finding presented in exhibit 7 concerns Sacramento. Between 2011 and 

2015, the ZIP Codes in Sacramento in which SAFMRs would have increased below-FMR listings 

are also those that witnessed drops in the number of below-FMR listings under existing policy. This outcome suggests Sacramento will benefit from the SAFMRs if this trend continues past 2015. 

Hypothesis Five: Proprietary Rental Listings Data Will More Closely Reflect 

Boeing and Waddell’s (2016) Estimation of the Share of Units Below FMR Than 

the ACS

As Boeing and Waddell utilized data in 2014, I use 2014 listings from my database to compare 

measurements of the share of units below FMR in exhibit 8. I disaggregate the results by number of bedrooms and the MSA designations that Boeing and Waddell used, which are different from the HUD metropolitan FMR area designations used elsewhere in this article. 

In line with my hypothesis, Rent Jungle and Craigslist are more closely aligned to each other than the ACS in 8 of the 12 statistics examined here (66 percent). In all eight of these cases, these two datasets show fewer listings below FMR than the ACS. In 6 of the 12 comparisons, the Rent Jungle and Craigslist results are less than 5 percentage points apart. 

Discrepancies between the two rental listings sources may be the results of my heuristics for identifying unique listings and my approach may need refinement. Alternatively, Rent Jungle’s Cityscape 161

Palm

Exhibit 8

Percent of Units Below FMR According to Craigslist, Rent Jungle, and the ACS

Craigslist  

Rent Jungle   2014 PUMS 1-Year 

MSA

Bedrooms

(%)

(%)

Recent Movers (%)

Sacramento--Roseville-- 

1

58

67

70

Arden-Arcade, CA

San Diego-Carlsbad, CA

1

17

28

51

San Francisco-Oakland-

1

18

16

41

Hayward, CA

 Sacramento--Roseville-- 

2

62

63

73

Arden-Arcade, CA

San Diego-Carlsbad, CA

2

24

27

36

San Francisco-Oakland-

2

26

20

27

Hayward, CA 

 Sacramento--Roseville-- 

3

69

73

62

Arden-Arcade, CA

San Diego-Carlsbad, CA

3

39

34

55

San Francisco-Oakland-

3

37

30

62

Hayward, CA

 Sacramento--Roseville-- 

4

53

66

71

Arden-Arcade, CA

San Diego-Carlsbad, CA

4

36

36

51

San Francisco-Oakland-

4

47

57

61

Hayward, CA

ACS = American Community Survey. FMR = Fair Market Rent. MSA = metropolitan statistical area. PUMS = Public Use Microdata Sample.

Sources: Craigslist data from Boeing and Waddell (2016); Rent Jungle data from author’s calculation proprietary listings might also include some higher-end rentals that property managers do not post to Craigslist, presenting a market that appears more expensive than the market viewed via Craigslist. The variation in findings among the two similarly organized listing sources demonstrates their limitations as suitable replacements to the ACS for FMR documentation. Proprietary providers may not be willing to make their assumptions, methods, and techniques publicly available, as HUD is required to do. 

Conclusions

Switching to the SAFMRs increases the share of rental listings below FMR in low-poverty neighborhoods in the Oakland, Sacramento, San Diego, and San Jose HUD metropolitan FMR areas. It 

does the opposite in San Francisco. Across all five areas, shifting to the SAFMRs increases the share of rental listings below FMR in neighborhoods with high performing schools, although data limitations prevent conclusions about below-FMR shifts in less expensive areas. This article reveals that in San Francisco, the hypothetical SAFMRs in high-rent areas are simply not high enough 

to increase the below-FMR rate in these neighborhoods. Although it might seem appropriate to 

recommend even higher FMRs for these neighborhoods, the cost of providing vouchers in the most expensive parts of the nation’s most expensive metropolitan areas may not be worth the trade-off of reducing the number of families the program could serve, as some critics of the SAFMRs point out 162 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents (NAHB, 2017). Regardless, this article finds the SAFMRs may increase voucher holders’ options in high-opportunity neighborhoods, even in some of the most expensive metropolitan areas in the United States. This benefit does not guarantee voucher recipients will be able to use vouchers in these areas, however, as the SAFMRs do not address source of income discrimination.

I also explored several concerns expressed by local activists and PHAs in response to HUD’s 

SAFMR ruling. San Francisco would see an overall drop in below-FMR listings, although the other four areas would see overall increases. However, given the trends tracked in exhibit 4, Oakland and San Jose might see a net drop in below-FMR listings if they adopted the SAFMRs at present. 

The results also illustrate the way areas that would lose below-FMR listings under the SAFMRs experienced a faster drop in below-FMR rates under existing policy in Oakland. This outcome 

implies the SAFMRs might only worsen the challenges of voucher recipients in that area. These results are broadly consistent with concerns expressed by advocates and agencies in Oakland, who are concerned with unprecedented upward swings in the local rental market. 

In contrast to Oakland, Sacramento may likely benefit from the SAFMR. This study finds that 

the SAFMRs may work most effectively in Sacramento and San Diego. That HUD only initially 

implemented the SAFMRs in these two markets among the five studied here inspires confidence in HUD’s criteria for applying the SAFMRs to metropolitan areas. 

This study also compared estimates of the share of units below FMRs across three sources of 

data, finding the ACS varied significantly from real-time listings data. Unfortunately, proprietary listings data do not meet the criteria established by GAO (2005) that the FMR system be entirely consistent, transparent, and replicable. Changes to the FMR system’s use of the Consumer Price Index and trend-adjustment factors may be the most effective way to alleviate the challenges faced by voucher recipients in tight markets while maintaining methodological transparency. 

Lastly, this study demonstrates the importance of scale when implementing housing policies and programs. Rents and land values are highly autocorrelated across space. The finer the scale at which market-oriented policy thresholds are determined, the more even policy outcomes are likely to be. This principle should be explored further, as it may have implications for other aspects of urban housing policy, from supply-side subsidy allocations to rent control.

Acknowledgments 

The author thanks Deb Niemeier and Carolyn Whitzman for helpful feedback on this work. The 

author also thanks five anonymous reviewers and the Cityscape editorial team for helpful com-

ments and review that enhanced this paper. This work was supported by California Department of Transportation (Caltrans) [Grant 65A0527 TO 017].

Author 

Matthew Palm is a research fellow at the University of Melbourne.

Cityscape 163

Palm

References

Basolo, Victoria. 2014. “Examining Mobility Outcomes in the Housing Choice Voucher Program: 

Neighborhood Poverty, Employment, and Public School Quality,”  Cityscape 15 (2): 135–153.

Bazuin, Joshua T., and James C. Fraser. 2013. “How the ACS Gets It Wrong: The Story of the 

American Community Survey and a Small, Inner City Neighborhood,”  Applied Geography 45: 292–302. https://doi.org/10.1016/j.apgeog.2013.08.013.

Boeing, Geoff, and Paul Waddell. 2016. “New Insights Into Rental Housing Markets Across the 

United States: Web Scraping and Analyzing Craigslist Rental Listings,”  Journal of Planning Education and Research. https://doi.org/10.1177/0739456X16664789.

Chetty, Raj, Nathaniel Hendren, and Lawrence Katz. 2016. “The Effects of Exposure to Better 

Neighborhoods on Children: New Evidence From the Moving to Opportunity Experiment,”  American Economic Review 106 (4): 855–902. https://doi.org/10.1257/aer.20150572.

Cohen, Rachel. 2018. “Court Forces Ben Carson To Be A Civil Rights Champion For A Day,”  The Intercept, January 2. https://theintercept.com/2018/01/02/ben-carson-hud-housing-voucher/.

Collinson, Robert, and Peter Ganong. 2013. Incidence and Price Discrimination: Evidence From 

Housing Vouchers. Working paper. Cambridge, MA: Joint Center for Housing Studies of Harvard 

University.

Cunningham, Mary K., and Audrey Droesch. 2005. “Neighborhood Quality and Racial Segrega-

tion.” http://webarchive.urban.org/UpLoadedPDF/411248_neighborhood_quality.pdf.

Folch, David C., Daniel Arribas-Bel, Julia Koschinsky, and Seth E. Spielman. 2014. Uncertain 

Uncertainty: Spatial Variation in the Quality of American Community Survey Estimates. Working paper 01. Tempe: Arizona State University, GeoDa Center for Geospatial Analysis and Computation.

Galvez, Martha. 2011. Defining Choice in the Housing Choice Voucher Program. Unpublished.

Galvez, Martha M. 2010.  What Do We Know About Housing Choice Voucher Program Location Outcomes? A Review of Recent Literature. Washington, DC: Urban Institute.

Geyer, Judy. 2017. “Housing Demand and Neighborhood Choice With Housing Vouchers,”  Journal of Urban Economics 99: 48–61. https://doi.org/10.1016/j.jue.2016.12.002.

Goetz, Edward G., and Karen Chapple. 2010. “You Gotta Move: Advancing the Debate on the 

Record of Dispersal,”  Housing Policy Debate 20 (2): 209–236.

Harsasz, Katherine. 2016. “Re: Docket No. FR-5855-P-02, Establishing a More Effective Fair Market Rent System: Using Small Area Fair Market Rents in Housing Choice Voucher Program Instead of the Current 50th Percentile FMRs.” San Jose, CA: Housing Authority of Santa Clara.

Horn, Keren M., Ingrid G. Ellen, and Amy E. Schwartz. 2014. “Do Housing Choice Voucher Hold-

ers Live Near Good Schools?”  Journal of Housing Economics 23: 28–40. https://doi.org/10.1016/j.

jhe.2013.11.005.

164 Refereed Papers

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents Johnson, Eric. 2016a. “Re: Establishing a More Effective Fair Market Rent (FMR) System: Using Small Area Fair Market Rents (SAFMRs) in Housing Choice Voucher Program Instead of the Current 50th percentile FMRs.” Docket No. FR-5855-P-02. Oakland, CA: Oakland Housing Authority.

———. 2016b. “Re: Docket No. FR-5855-P-02 (Title: Establishing a More Effective Fair Market 

Rent System: Using Small Area Fair Market Rents in Housing Choice Voucher Program Instead of 

the Current 50th Percentile FMRs).” Oakland, CA: Oakland Housing Authority.

Kleit, Rachel G., and Martha Galvez. 2011. “The Location Choices of Public Housing Residents 

Displaced by Redevelopment: Market Constraints, Personal Preferences, or Social Information?” 

Journal of Urban Affairs 33 (4): 375–407. https://doi.org/10.1111/j.1467-9906.2011.00557.x.

Levin, Jeff. 2016. “Re: Docket No. FR 5855-P-02, (Title: Establishing a More Effective Fair Market Rent System: Using Small Area Fair Market Rents in Housing Choice Voucher Program Instead of 

the Current 50th Percentile FMRs).” Oakland, CA: East Bay Housing Organizations. 

Löchl, Michael, and Kay W. Axhausen. 2010. “Modeling Hedonic Residential Rents for Land Use 

and Transport Simulation While Considering Spatial Effects,”  The Journal of Transportation and Land Use 3 (2): 39–63. https://dx.doi.org/10.5198/jtlu.v3i2.117.

McClure, Kirk. 2014. “The Prospects for Guiding Housing Choice Voucher Households to High-

Opportunity Neighborhoods,”  Cityscape 12 (3): 101–122.

McClure, Kirk, Alex F. Schwartz, and Lydia B. Taghavi. 2014. “Housing Choice Voucher Location Patterns a Decade Later,”  Housing Policy Debate 25 (2): 37–41. https://doi.org/10.1080/10511482.2

014.921223.

McCord, Michael, Peadar T. Davis, Martin Haran, David McIlhatton, and John McCord. 2014. “Un-

derstanding Rental Prices in the UK: A Comparative Application of Spatial Modelling Approaches,” 

International Journal of Housing Markets and Analysis 7 (1): 98–128. https://doi.org/10.1108/

IJHMA-09-2012-0043.

Metzger, Molly W. 2014. “The Reconcentration of Poverty: Patterns of Housing Voucher Use, 2000 

to 2008,”  Housing Policy Debate 24 (3): 544–567. https://doi.org/10.1080/10511482.2013.876437.

Miller, Harvey J. 2004. “Tobler’s First Law and Spatial Analysis,”  Annals of the Association of American Geographers 94 (2): 284–289. https://doi.org/10.1111/j.1467-8306.2004.09402005.x.

National Association of Home Builders (NAHB). 2017. “HUD Announces 2-Year Suspension of 

Small Area Fair Market Rents.” http://nahbnow.com/2017/08/hud-announces-2-year-suspension-

of-small-area-fair-market-rents/.

National Low Income Housing Coalition (NLIHC). 2017. “HUD Suspends Mandatory Small 

Area FMR Implementation.” http://nlihc.org/article/hud-suspends-mandatory-small-area-fmr-

implementation.

———. 2016a. “HUD Publishes FY17 Fair Market Rents.” http://nlihc.org/article/hud-publishes-

fy17-fair-market-rents.

Cityscape 165

Palm

———. 2016b. “Out of Reach 2016: No Refuge for Low Income Renters.” http://nlihc.org/sites/

default/files/oor/OOR_2016.pdf.

Newman, Sandra, and Ann B. Schnare. 1997. “‘… And a Suitable Living Environment’: The Failure of Housing Programs To Deliver on Neighborhood Quality,”  Housing Policy Debate 8 (4): 703–741.

Ruel, Erin, Deirdre A. Oakley, Chandra Ward, Reneé Alston, and Lesley W. Reid. 2013. “Public 

Housing Relocations in Atlanta: Documenting Residents’ Attitudes, Concerns and Experiences,” 

Cities 35: 349–358. https://doi.org/10.1016/j.cities.2012.07.010.

Schwartz, Heather L., Kata Mihaly, and Breann Gala. 2016. “Encouraging Residential Moves to Opportunity Neighborhoods: An Experiment Testing Incentives Offered to Housing Voucher Recipients,” 

Housing Policy Debate 27 (2): 230–260. https://doi.org/10.1080/10511482.2016.1212247.

Substance Abuse and Mental Health Services Administration (SAMHSA). 2016. “Impact of 

Increased Income on Housing Voucher Programs.” https://soarworks.prainc.com/sites/soarworks.

prainc.com/files/Income_Housing_Vouchers_052616.pdf.

Tighe, J. Rosie, Megan E. Hatch, and Joseph Mead. 2016. “Source of Income Dis-

crimination and Fair Housing Policy,”  Journal of Planning Literature 32 (1): 3–15. https://doi.

org/10.1177/0885412216670603.

Tu, Yong, Hua Sun, and Shi-Ming Yu. 2007. “Spatial Autocorrelations and Urban Housing Market 

Segmentation,”  The Journal of Real Estate Finance and Economics 34 (3): 385–406. https://doi.

org/10.1007/s11146-007-9015-0.

U.S. Census Bureau. 2016a. “American Community Survey (ACS).” https://www.census.gov/

programs-surveys/acs/technical-documentation/table-and-geography-changes.html.

———. 2016b. “Sample Size.” http://www.census.gov/acs/www/methodology/sample-size-and-

data-quality/sample-size/index.php.

———. 2015. “U.S. Mover Rate Remains Stable at About 12 Percent Since 2008, Census Bureau 

Reports.” http://www.census.gov/newsroom/press-releases/2015/cb15-47.html.

U.S. Department of Housing and Urban Development (HUD). 2017a. “Final FY 2016 Fair Market 

Rent Documentation System.” huduser.gov/portal/datasets/fmr.html#2016.

———. 2017b. “The Final FY 2016 San Diego-Carlsbad-San Marcos, CA MSA FMRs for All 

Bedroom Sizes.” huduser.gov/portal/datasets/fmr/fmrs/fy2013_code/2013summary.odn.

———. 2015.  Changes to Flat Rent Requirements—FY 2015 Appropriations Act PIH-2015-1. https://

www.hud.gov/sites/documents/PIH201513FLATRENTNOTICE.PDF.

U.S. Government Accountability Office (GAO). 2005. “Rental Housing: HUD Can Improve Its 

Process for Estimating Fair Market Rents.” http://www.gao.gov/products/GAO-05-342.

Wang, Xinhao, and David P. Varady. 2005. “Using Hot-Spot Analysis to Study the Clustering of 

Section 8 Housing Voucher Families,”  Housing Studies 20 (1): 29–48. https://doi.org/10.1080/0267

303042000308714.

166 Refereed Papers

Can a Car-Centric City 

Become Transit Oriented? 

Evidence From Los Angeles

Jenny Schuetz    

Brookings Institution

Genevieve Giuliano  

University of Southern California

Eun Jin Shin  

Yale-NUS College

Abstract

The urban built environment develops over decades around fixed infrastructure. Los 

Angeles began its major growth at the dawn of the automobile era and became a 

low-density, dispersed metropolis organized around a vast freeway system. Since the 

1990s, local governments have sought to restructure Los Angeles, shifting toward higher 

density, mixed-use housing and commercial development. A large investment in new 

rail transit lines is seen as critical to achieving these land use goals, mainly through 

promotion of transit-oriented development. In this article, we examine how employment 

patterns have changed around newly built Los Angeles rail stations. Results suggest that 

employment did not increase near stations immediately before or after station opening, 

but a few stations saw increased employment 5 to 10 years after opening.

Introduction

For most of the 20th century, Los Angeles was the quintessential car-oriented city. Over the past 20 

years, however, local and regional governments have invested significant resources in building rail transit infrastructure that connects major employment centers, including downtown Los Angeles, Long Beach, Pasadena, and the eastern Wilshire Corridor. One goal of transit infrastructure is to catalyze high-density, mixed-use housing and commercial development within walking distance 

of rail stations, known as transit-oriented development (TOD). By increasing the accessibility of 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 167

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Schuetz, Giuliano, and Shin

station areas, the building of new stations should increase surrounding land values, leading to higher-density development. In this article, we examine changes in employment patterns around Los Angeles County Metropolitan Transit Authority (LA Metro) rail stations from 1990 to 2010. 

The analysis examines whether station areas have experienced changes in the density or composition of employment following station opening, and explores the time frame in which such changes may happen.

Standard urban economics models yield several hypotheses for how and why economic activity 

might change in areas where new rail stations are built. Following the standard monocentric city model, land values are highest at the central business district (CBD) and decline moving outward in proportion with increasing travel costs (Alonso, 1964; Brueckner, 1987; Mills, 1967; Muth, 1969). Building a rail station that connects the station’s neighborhood with the CBD or employment subcenters should increase the accessibility of that neighborhood, thereby increasing land values and encouraging higher density development nearby (Anas, 1995; Glaeser and Kohlhase, 

2004). Neighborhoods around rail stations should be relatively more attractive both to firms and households. Firms can attract more workers due to increased accessibility, as well as more consumers to convenient locations, particularly in household-serving industries such as retail, food service, and healthcare. Households will be willing to pay higher housing prices in exchange for lower transit costs.

How much land values and economic activity increase near stations depends on the extent of 

improved accessibility to the location; for instance, stations that link to larger and denser rail networks should have greater impacts on land values. Rail lines that simply replace existing bus transit service have little impact on accessibility. Because most passengers access rail stations by walking, station effects will be highly localized. Prior research has also posited some potential negative impacts of rail stations on nearby areas. Rail stations may increase noise, traffic congestion, or crime in the adjacent area; these nuisances are likely stronger disamenities for households than for commercial uses. Land values around stations may fluctuate in the short run, both prior to and immediately after opening, before reaching long-run equilibrium. The relationship between short-run and long-run land values is somewhat ambiguous. For instance, anticipation of increased demand may cause short-run spikes in land values, beyond prices that developers are willing to pay, which can deter or delay development. This is particularly likely if small parcel owners become “holdouts” (Brooks and Lutz, 2016). Conversely, developers may perceive untested locations as excessively risky and delay undertaking projects until some first-mover demonstrates actual profits (essentially underestimating long-run land values in the short run).

A broad empirical literature has attempted to identify the impacts of rail transit investments on outcomes such as transit ridership, land values, housing prices, population and housing density, employment composition, population characteristics, and crime (Baum-Snow and Kahn, 2005; 

Billings, 2011; Billings, Leland, and Swindell, 2011; Boarnet and Crane, 1997; Bollinger and Ihlanfeldt, 1997; Bowes and Ihlanfeldt, 2001; Cervero and Landis, 1997; Debrezion, Pels, and Rietveld, 2007; Dubé, Thériault, and Des Rosiers, 2013; Giuliano and Agarwal, 2010; Handy, Cao, and 

Mokhtarian, 2005; Kahn, 2007; Lin, 2002; Mathur and Ferrell, 2013; McMillen and McDonald, 

2004; Poister, 1996; Renne and Ewing, 2013; Winston and Maheshri, 2007). As well as measuring different outcomes, these studies cover different cities, time periods, and transit types (heavy rail, 168 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles light rail, and streetcar). Results from these studies are somewhat mixed, although one relatively consistent finding is that the extent of changes in property values, employment, and related 

economic outcomes depends on the level of transit ridership; low ridership on average produces smaller impacts.

Only a few prior studies have examined the LA Metro system, which is new relative to “legacy” 

systems such as those in New York City, Boston, and Chicago, or even second-wave subways such as the Bay Area Rapid Transit (BART) in the San Francisco Bay Area and Washington, D.C.’s Metro. 

Kolko (2011) and Schuetz (2015) examined employment near newly opened rail stations in Los 

Angeles and several other large California metropolitan areas; both find little change in employment levels near stations. Redfearn (2009) found no average change in housing prices near Los Angeles rail stations but furthermore found that the average conceals substantial variation in housing price changes across stations. Similarly, in a qualitative study of physical redevelopment near five LA Metro stations, Schuetz, Giuliano, and Shin (2017) found that TOD is emerging unevenly across station neighborhoods. Areas that experienced changes in land use or buildings have strong localized real estate markets, have zoning that allows high-density residential and commercial development, and benefited from highly targeted local government engagement. Weak property 

values and incompatible zoning both contribute to lack of redevelopment near some stations.

This article makes several contributions to the existing literature. First, relatively few studies have examined the impacts of rail transit on employment or commercial activity, although retail, services, and related activities are key components of TOD. Second, we are able to conduct 

longitudinal analysis of treatment and control areas over a 20-year period, which allows us to test for prestation anticipation effects and lagged changes. Third, impacts of transit in Los Angeles have been less studied than in many other cities. Los Angeles’ history as a car-centered city makes this a particularly interesting empirical setting to determine whether introduction of a rail system has the capacity to change land use patterns. This research is particularly relevant in light of ongoing rail and streetcar investment in many U.S. cities, including Charlotte, Cincinnati, Denver, and Washington, D.C.

In this analysis, we combine data on the location and opening dates of 28 rail stations in Los Angeles County with establishment-level employment data. We measure the level and industrial composition of employment within 0.25- and 0.5-mile catchment areas of newly opened rail stations, before and after opening. As a comparison group, we identify a set of major road intersections in the same neighborhoods as stations but outside direct catchment areas. We use a difference-in-differences approach to compare changes in employment outcomes before and after opening for 

station and control areas. Results indicate that the areas selected for new stations had unusually high employment density prior to station opening. No evidence suggests that employment near 

stations changed within 5 years before or after station opening, but some results suggest that a few stations experienced increased employment within a 5- to 10-year period after opening. One possible explanation for the long lag is that most stations were built in already densely developed areas, where redevelopment is costly and slow. Alternatively, proximity to stations may become more valuable as the network size expands through additional lines.

Our results offer two key insights to transit planners who are building or expanding rail networks in other metropolitan areas, particularly car-oriented cities with multiple employment centers Cityscape 169

Schuetz, Giuliano, and Shin

and a dense urban fabric. First, transit infrastructure is more likely to deliver long-run benefits than short-run stimulus. Second, planners should be clear about the primary goal of building rail systems—weighing access to existing jobs versus stimulating new residential development—when 

choosing station locations and coordinating housing or land use policies.

The remainder of this article is organized as follows. Section 2 provides more context on Los Angeles’ rail network. Section 3 discusses the data sources and empirical methods. Results of the analysis are presented in Section 4. Section 5 discusses policy implications and concludes.

Background on Los Angeles Rail Network

Even after roughly $9 billion (nominal) of public investment in rail infrastructure, Los Angeles remains a car-oriented city (Nelson and Weikel, 2016). As of the late 2000s, 84 percent of the city’s residents commuted to work by car, with fewer than 7 percent using mass transit (exhibit 1). Even among transit riders, over 90 percent of commuters relied on buses rather than rail; these market shares have not changed appreciably since rail service began in 1990. The relatively low ridership raises questions about whether proximity to rail stations is highly valued by residents, workers, and firms, and thus whether station access will be capitalized into higher land values and increased employment. The utility of a rail network is determined by how much it increases accessibility, that is, to what extent it facilitates passengers’ ability to reach desirable locations. LA Metro stations are relatively thinly spread across a large geographic area (exhibits 2 and 3); on average, each station is 1.25 miles from its nearest station (Schuetz, 2015). The existing rail lines link several large employment centers to one another, but many residential areas, and a large share of the population, are too far from any rail station to make using the system practical for daily commuting even when considering using bus service to transfer to the nearest rail station.

One means of illustrating the demand for rail stations is the number of daily boardings (exhibit 4). 

Across all study-area stations, daily boardings averaged about 6,700 in 2013, the most recent year for which data are available. Boardings vary widely across stations and lines; the Purple and Red Line stations in downtown and central Los Angeles draw the most riders, with over 13,500 average boardings per day, compared to about 1,700 boardings at the Gold Line stations in Pasadena and the Arroyo Seco corridor north of downtown. Connectivity to the broader network is correlated Exhibit 1

Mode Share for Daily Journey to Work, Selected U.S. Counties (2006–2010)

County

Rail 

Bus

Car

Bike or Walk

Other

New York City, New York

39.1 

12.2 

30.0 

10.6 

8.1 

Suffolk, Massachusetts

17.4 

11.6 

51.0 

14.5 

5.6 

San Francisco, California

9.7 

20.7 

47.4 

12.4 

9.8 

Cook, Illinois

6.2 

7.4 

73.1 

4.8 

8.6 

Los Angeles, California

0.4 

5.7 

84.3 

3.5 

6.1 

Dallas, Texas

0.4 

2.2 

90.6 

1.5 

5.3 

King, Washington

0.1 

9.9 

77.7 

5.2 

7.2 

Notes: Rail includes subway, elevated, streetcar, and trolley car. Car includes truck and van. New York City includes five constituent counties (Bronx, Kings, New York, Queens, and Richmond).

Source: Calculations based on Ruggles et al. (2015), 2006–2010 Integrated Public Use Microdata Series sample of American Community Survey

170 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles

Exhibit 2

Study Area Metro Stations

Note: Map shows only Metro stations included in study (excludes Gold Line stations that opened after 2009 and all Blue Line stations). 

Source: Data assembled from Los Angeles County Metropolitan Transit Authority

Exhibit 3

Station Areas and Comparison Neighborhoods

Source: Data assembled from Los Angeles County Metropolitan Transit Authority

Cityscape 171

Schuetz, Giuliano, and Shin

Exhibit 4

Average Daily Boardings at Metro Stations, 2013

Line(s)

Boardings

Gold

1,709

Purple and Red

13,555

Red

7,448

All sample stations

6,733

Notes: Only stations included in the study are shown. Purple and Red Lines include stations that serve both lines, as well as the two stations that serve only the Purple Line (Wilshire/Western and Wilshire/Normandie). Union Station, which serves all three lines, is included in the Purple and Red group.

Source: Calculations based on data provided by Los Angeles County Metropolitan Transit Authority with ridership. The stations with the three highest number of daily boardings are Union Station (over 34,000), which serves the three Metro lines, as well as the Metrolink commuter rail system and Amtrak; 7th St/Metro Center (27,000), serving the three Metro lines and several major bus lines; and North Hollywood (17,000), which connects the Red Line rail to a bus rapid transit 

system serving the San Fernando Valley. The most used station on the Gold Line is Sierra Madre Villa (2,900), also the line’s final station at the time of the study and which (like North Hollywood) has a large adjacent park-and-ride lot. These stations likely attract riders from a larger area than the typical 0.5-mile catchment estimated for walking.1 Unfortunately we do not have time-series ridership data by station and so cannot determine how much current ridership reflects changes that have taken place since station opening versus original population density or land use.

In an economically efficient world, in order to maximize the value of infrastructure, rail stations should be located in areas with the greatest potential for ridership—based on the density of nearby population and jobs—and with potential for high-density development surrounding stations. 

In reality, the nearly three-decade-long planning for Metro routes was influenced by numerous competing political factions, including the Los Angeles Mayor and City Council members, Los 

Angeles County supervisors, members of Congress, city and county taxpayers, neighborhood 

residents, local business leaders, as well as civic, cultural, and economic institutions throughout the region.2 The general direction of each line, as well as the placement of some stations, reflect compromises along multiple dimensions. For instance, the Blue Line between Los Angeles and 

Long Beach was built first because of several political and fiscal advantages. Including Long Beach brought additional local tax revenues into the deal, and using existing rail rights-of-way reduced development costs. The route ran through the district of a highly influential Los Angeles County Supervisor, Kenneth Hahn, and through a largely industrial corridor with mostly low-income 

residents who generally supported transit, or at least were not organized in opposition to the route (Elkind, 2014). The Green Line was built down the middle of Interstate 105 as part of a consent decree resolving a lawsuit over the freeway’s construction (Elkind, 2014). The subway lines from downtown Los Angeles to Hollywood and the San Fernando Valley were the most controversial 

1 Commuters who bike to rail stations may also originate from a larger catchment area.

2 The lengthy and complex planning and development process was minutely documented in Elkind (2014). Taylor, Kim, and Gahbauer (2009) examined political influences for the Red Line. In this article, we briefly summarize a few of the general factors and examples that illustrate why rail station areas are systematically different than control areas.

172 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles routes. Initially, the subway was planned to run along Wilshire Boulevard from downtown Los 

Angeles in the east to Fairfax Avenue on the west, one of the densest employment and housing 

corridors in the U.S. However, political opposition from residents of several affluent Westside neighborhoods, and their representatives, Congressman Henry Waxman and Los Angeles City 

Council member Zev Yaroslovsky, effectively forced the subway to turn north from Wilshire much farther east than originally planned (Elkind, 2014; Taylor, Kim, and Gahbauer, 2009). The final route along Vermont Avenue was selected because it had fewer residential areas to raise opposition and because the subway was supported by several large health and educational institutions along the route (Elkind, 2014). Similarly, the stations in Hollywood were supported by the local chamber of commerce, which welcomed the potential revitalization of a declining area (Elkind, 2014). In general, well-organized opposition by affluent homeowners blocked proposed routes that would 

have directly connected some of the largest and densest employment centers, resulting in routes through less dense, lower-end commercial and industrial corridors.

Besides the overall level of ridership, composition of Metro rail passengers may affect the potential for economic development near new stations. Higher income riders will have greater potential 

purchasing power and so increase the demand for housing and other goods and services near rail stations. According to Census data, the median household income of rail transit commuters living in Los Angeles and Pasadena is around $61,000, about $14,000 below incomes for car commuters 

and well above the $42,000 median income for bus riders. Many of LA Metro’s rail passengers 

had previously relied on buses as a primary means of transportation, prior to the opening of the rail system, so rail represents not an increase in total mass transit share but a switch across modes within transit. In some instances, rail stations were built at locations with important bus connections (for instance, all the Purple Line stops along Wilshire Boulevard are served by the heavily used Metro Rapid 720 express bus). For such station areas, the site’s accessibility through public transit may already have been capitalized into land values and development patterns well before the rail stations opened.3

Data Sources and Empirical Approach

We analyze changes in employment density and composition around 28 rail stations that opened 

in Los Angeles County between 1992 and 2003. As a comparison group, we identify a set of 

intersections located more than 0.5 mile but within 3 miles of the rail stations. The analysis uses several variations on a difference-in-differences framework to test whether employment changed near rail stations after station opening, relative to control areas. We test for differences before and after opening, as well as variation over time before and after opening.

Data Sources

The location and opening dates of rail transit stations were assembled from the LA Metro website and supplemental documentation. The street address of stations has been geocoded and matched 

to latitude-longitude coordinates and census geographies. Information on which rail lines serve 3 Unfortunately, we do not have time-varying data on bus station locations and service lines.

Cityscape 173

Schuetz, Giuliano, and Shin

each station was also assembled. The research focuses on 28 stations along the Red, Gold, and Purple Lines, for which we have sufficient data on preopening and postopening outcomes.4

Data on business establishments come from the National Establishment Time Series, or NETS, database, which contains the business name, address, North American Industry Classification System (NAICS) industry code, and number of employees for all business establishments from 1992 to 2009. Outcomes of interest are the total number of jobs near stations and the mix of jobs by industry category.

General economic and demographic characteristics on station and control areas are assembled from tract-level data from the 1990 and 2000 decennial census and the 2005–2009 American Community Survey. Treatment areas around stations are defined as circles with radius of either 0.25 or 0.5 mile, while control areas are similar-sized circles around major intersections, described in more detail in the following section. To match census tract characteristics to station and control areas, we use GIS to determine the percent of land in each study area drawn from each census tract, and created weighted averages of census variables using these percentages. Variables included in the analysis include population density and median household income. 

Empirical Approach

The research design compares changes in housing and employment outcomes near newly opened 

rail stations, before and after opening. As shown in exhibit 5, study-area stations offer sufficient variation in timing to allow analysis of employment changes prior to and after development. The stations vary along a number of other dimensions that are likely to affect employment outcomes. 

Some stations are below ground while others are above grade, and they are located in neighborhoods of varying economic, demographic, and physical characteristics. The density and mix of 

prior development around the station sites also varies. The Red and Purple Lines run through 

predominantly commercial parts of Los Angeles, as well as some residential areas near North 

Hollywood, while the Gold Line goes through both residential and commercial areas. About three-fourths of the stations are located within the city of Los Angeles, with six in the city of Pasadena and one in the city of South Pasadena. Treatment areas are defined as circles of either 0.25- or 0.5-mile radius from the rail station, which prior literature has shown is the typical catchment area for rail transit ridership (Guerra and Cervero, 2013; Hess and Almeida, 2007; Kolko, 2011; McDonald and Osuji, 1995). We use 0.25-mile radius for Red and Purple Line stations, because 

these stations are located closely together, and 0.25 mile yields mostly nonoverlapping treatment areas. The Gold Line stations and Red Line stations in North Hollywood are located farther from one another, so we use 0.5-mile radius as the treatment area for those stations.5  

4 The Blue Line stations opened in 1990, before our employment data are available, whereas the Expo Line and some Gold Line stations are too recent for us to observe poststation outcomes. The Green Line is excluded because most stations are in the freeway median, making development immediately adjacent to the stations impossible. Descriptive statistics include all 28 stations, but regression analysis excludes the 5 stations that opened prior to 1996, because we do not observe at least 3 years of preopening employment.

5 The 0.5-mile treatment areas around three downtown Pasadena stations do overlap, but the 0.5-mile catchment area was deemed more appropriate, given the presence of onsite station parking. The overlapping areas are in a sense doubly treated, which could introduce upward bias into the estimated impact of those stations. A few control areas have small overlaps with the station areas, which may bias results downward for those pairs, but the small number of overlapping control areas is unlikely to influence aggregate regression results.

174 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles Exhibit 5

Station Opening Dates

Year  Number of 

Station Name(s)

Lines

Open

Stations

1990

1

7th St/Metro Center

Blue, Purple, Red

1992

1

Union Station 

Gold, Purple, Red

1993

3

Civic Center, Pershing Square, Westlake/Macarthur Park

Purple, Red

1996

3

Wilshire/Normandie, Wilshire/ Vermont, Wilshire/Western

Purple, Red

1999

5

Hollywood/Vine, Hollywood/ Western, Vermont/Beverly,  

Red

Vermont/ Sunset, Vermont/ Western

2000

3

Hollywood/Highland, North Hollywood, Universal City

Red

2003

12

Allen, Chinatown, Del Mar, Fillmore, Heritage Square, Highland  Gold

Park, Lake, Lincoln/Cypress, Memorial Park, Mission, Sierra 

Madre Villa, Southwest Museum

Total

28

Note: When the 7th St/Metro Center station opened in 1990, only the Blue Line was in operation.

Source: Data assembled by authors from Los Angeles County Metropolitan Transit Authority

The key challenge in determining whether new rail stations lead to changes in nearby economic activity is identifying plausible comparison areas: geographic areas that had similar characteristics to station areas prior to station opening and would have had similar trajectories over time but which were not affected by the new stations. As summarized in Section 2, historical evidence reveals that LA Metro station locations were selected based largely on political and fiscal compromises, which may not correspond to the most economically or geographically efficient sites. Nonetheless, station locations likely differ from all nonstation areas in Los Angeles County in ways that can affect subsequent development. Therefore, we defined comparison areas based on several criteria designed to control for important preopening differences. First, comparison areas should be more than 0.5 mile from any rail station (new, existing, or future) so they will not directly be affected by the station. 

Second, they should be located within 3 miles of at least one newly opened station, so that they share general place-specific attributes, such as proximity to large employment centers or school districts. Third, because rail stations are almost always located at intersections of major streets, which will have relatively high volumes of car and pedestrian traffic, control areas are selected from among the intersections of similarly sized streets. In practice, we attempted to define control areas as intersections that shared one or more streets with rail stations (for instance, the intersection of Western Avenue and West 3rd Street is a comparison site for the rail station located directly south at Western Avenue and Wilshire Boulevard). 

This approach offers two advantages over other matching methods, such as propensity score 

matching (PSM). First, our treatment areas—circular areas within walking distance of transit 

stations—do not correspond to conventionally defined geographic areas such as census tracts or block groups. (Indeed, most stations are located at the intersection of multiple census tracts, so any single tract or block group captures only a fraction of the relevant area.) Therefore, no set of predefined geographies not affected by stations could serve as potential control areas to be used in an automated matching process. Second, the underlying rationale for why station areas should see increased economic activity is that they benefit from particularly high accessibility to a larger transit network. The major intersections where stations are located tend to have greater access for Cityscape 175

Schuetz, Giuliano, and Shin

automobiles, buses, and pedestrians, as well as trains. Choosing control areas that share similar road access allows us to control for the nonrail access of the comparison sites, in a way that would be difficult to capture accurately using PSM or similar methods.

Exhibits 2 and 3 show the location of the 28 station areas and 48 comparison areas in the study. 

The stations form a rough triangle among the North Hollywood Station (northwest corner), Sierra Madre Villa Station in Pasadena (northeast corner, approximately 20 miles apart), and the 7th Street/Metro Center Station in downtown Los Angeles (approximately 13 miles southeast of North Hollywood and 15 miles southwest of Sierra Madre Villa). Stations and control areas form several spatial clusters, assigned to five geographic submarkets: Arroyo Seco, Central Los Angeles, Downtown Los Angeles, North Hollywood, and Pasadena.

We begin with a set of graphs and descriptive statistics, illustrating the levels and changes in employment during the study period. We then use a modified difference-in-differences framework to compare station outcomes and comparison area outcomes, as illustrated in equation (1).

(1)

In this equation,  i indexes the study area,  t indexes the year.  Y is a measure of employment.  Station is a dummy indicating station areas. We look at both employment density (employees per acre) 

across all industry sectors as well as share of employment in each of four broad industry categories: commercial, industrial, public-institutional, and miscellaneous (see the appendix for NAICS 

two-digit sectors assigned to the four industry categories).  Post is a dummy variable that equals one after station opening (for comparison areas, this is based on the opening date of the nearest station). The coefficient of interest is  β , on the interaction between  Station and  Post, indicating 3

whether employment near station areas changes after station opening.  X is a vector of control variables that could influence employment outcomes in study areas and change over time, such as population density and household income. Models also include polynomial terms for year (year 

and year-squared), to control for larger economic time trends such as labor market conditions.6 

Fixed effects for geographic submarkets described previously are also included.

The before-and-after opening framework may obscure an important question: do employment pat-

terns vary differently across years, either before or after station opening? Several of the hypotheses about how outcomes might vary over time would not be captured by a simple before-and-after 

analysis. Some studies of new rail lines in other cities have found an “anticipation effect,” in which real estate prices near stations increase after the locations have been announced but well before the stations begin operating (Billings, 2011; McMillen and McDonald, 2004). For impacts to 

begin appearing soon after the announcement, it is necessary that landowners or developers have reasonably certain expectations that stations will indeed be built at the announced locations, in a time frame that justifies current investment.7 However, in the case of Los Angeles, it is unclear 6 We include time trends as polynomial terms rather than a set of year fixed effects to avoid collinearity with years of station opening. Robustness checks using linear year and higher order polynomials suggest a squared term is the appropriate functional form.

7 In Chicago, the line in question was an expansion of the already well-utilized system, adding a connection from downtown to Midway airport (McMillen and McDonald, 2004). In Charlotte, the city government revised the zoning and land use planning to maximize growth potential around a new light rail system (Billings, 2011). In both cases, the announcement of specific locations was followed relatively soon by appropriation of funds and the start of construction.

176 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles

whether the conditions for anticipatory investment were present. The earliest plans suggesting rail line pathways emerged in the early 1970s, but federal and local funding for construction remained highly uncertain until the mid-1980s. The location of stations along the Red and Purple Lines was highly contentious, with multiple plans proposed and political jockeying for and against, until shortly before construction began (Elkind, 2014). In practice, we are unable to test for changes in employment before and after the announcement date, or around the date that funding was secured, because our employment data do not extend back far enough. Given the demonstrated reliance of Los Angeles commuters on cars, demand for rail transit—particularly for early stations—will be particularly uncertain. Thus it seems plausible that employers or real estate developers may be re-luctant to expand employment or construct buildings near a planned station until a few years after operation to observe the volume of transit riders and effectiveness of the new rail line. In this case, there may be a substantial delay before aggregate economic patterns change. To test for varying employment patterns over time, we estimate the following regressions, shown in equation (2). 

(2)

In this equation,  YrsPre is a continuous numeric variable indicating the number of years prior to station opening (equal to 0 for all years after opening),  YrsPost is the count of years after station opening (equal to 0 for all years prior). The interaction term,  Station*YrsPost, gives the coefficient of interest, indicating the difference in employment associated with each year after opening for station areas, relative to control areas. Regressions include the same control variables, year polynomial terms, and fixed effects for geographic submarkets.8 

The regression analysis implicitly tests the hypothesis that increases in land values due to station areas’ improved accessibility will result in higher density of economic activity. However, localized public policy interventions, particularly land use regulation, have the potential either to enhance or constrain market pressures on economic outcomes near stations. For instance, if new stations are opened in areas zoned for low-density, exclusively residential land use, then it is unlikely that new housing or employment could emerge near the station, even if firms and developers wished 

to locate nearby. Alternatively, if zoning grants developers density bonuses or other incentives to locate near stations, relative to equivalent sites not near transit, then the regulation could result in more economic activity near the station than markets alone would have provided. Because zoning and other public interventions may either constrain or enhance development, and prior research has found that zoning differs substantially across LA Metro stations (Schuetz, Giuliano, and Shin, 2017), it seems likely that not controlling for local policies will introduce measurement error but will not consistently bias our results.

Results

The locations in which new rail stations were built during the 1990s and 2000s had unusually 

high employment densities prior to station opening. Employment densities around station and 

8 As a robustness check, we also estimate regressions with a full set of dummy variables for each year before and after opening. Results of the fully interacted model are substantively similar to the simpler interactions with continuous number of years; results available from authors on request.

Cityscape 177

Schuetz, Giuliano, and Shin

control areas fluctuated somewhat over time with macroeconomic cycles, but there is no clear time trend. Descriptive statistics and regressions both indicate that station areas did not see employment growth within 3 years before or 5 years after station opening. Regression results suggest that a few stations that opened between 1996 and 1999 saw significant employment gains between 5 and 10 

years after stations opened.

Descriptive Statistics: Employment Metrics

A substantial difference between the rail system in Los Angeles and those in older cities such as New York City and Boston is that land use and employment patterns were well-established before Los Angeles’ rail stations were built. As noted in the second section, rail lines were intended to connect existing employment centers, enhancing access of potential workers to job-rich areas. An analysis of preopening station area characteristics confirms that areas where rail stations opened during the 1990s and 2000s already had high employment densities well before the rail network was built (exhibit 6). The average station area had nearly 70 employees per acre as of 1992, four times the employment density in control areas; excluding stations and control areas in downtown Los Angeles, station areas had on average 34 employees per acre, compared to 11 employees per acre in control areas. Both station and control areas had much higher employment density than Los Angeles County overall, suggesting that the selected control areas form a better counterfactual to station areas than the remainder of the county. Establishments near future stations were, on average, nearly 50 percent larger than establishments in control areas, measured by employees per establishment. Station and control areas share two prominent employment sectors: retail (NAICS 

codes 44 and 45) and healthcare and social assistance (NAICS code 62) each make up 10 to 12 

percent of employment. Beyond those sectors, employment near stations was more weighted 

Exhibit 6

Station and Control Areas Prior to Rail System Opening, 1990

Station Areas

Control Areas

Los Angeles County

Employment characteristics

Employees per acre

66.6

15.8

1.5

Establishments per acre

3.49

1.57

0.1

Employees per establishment

21.3

14.6

11.6

Employment mix

Commercial (%)

47.0

41.0

38.1

Industrial (%)

22.7

33.9

37.2

Public/institution (%)

20.1

18.5

19.0

Miscellaneous (%)

7.2

7.8

5.8

Population characteristics

Population per acre

111.8

102.3

3.4

Household income ($)

44,017

58,187

75,908

Bachelor's/graduate degree (%)

22.5

23.6

22.3

Black (%)

9.0

9.4

11.2

Hispanic (%)

42.9

42.2

37.3

Asian (%)

16.4

14.4

10.5

Younger than 18 (%)

20.31

22.98

26.2

Notes: All numbers for station and control areas are averages per study area. Housing and census variables are measured as of 1990, employment variables as of 1992–1994. Prices and incomes reported in constant 2009 dollars.

Sources: Calculations based on National Establishment Time Series database, DataQuick, and American Community Survey 2005–2009

178 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles

toward commercial sectors, including professional, scientific and technical services, and accommodation/food services, which are typical users of retail and office buildings. Control areas leaned more toward industrial sectors, mostly wholesale trade and manufacturing, which tend to be 

located in buildings with lower floor-to-area ratios.

Station and control areas differed somewhat in population characteristics, prior to development of the rail network, but these differences are less pronounced than the prestation differences in employment. 

Both station and control areas had higher population densities than Los Angeles County overall. 

As of 1990, residents near station areas had lower incomes than the population in control areas and the county overall. The populations in both station and control areas tended to be slightly more Hispanic and Asian than Los Angeles County, with slightly lower African-American population shares.

The implications of these differences for future job growth are not immediately obvious. It is possible that the more industrially oriented control areas will be less desirable for additional development, or may not be zoned for standard commercial uses. Alternatively, areas with more industrial uses might offer more large-scale land parcels for redevelopment, or face less opposition from existing landowners and tenants at the prospect of new, higher-density development. Lower incomes in station areas may suggest that those areas were initially less attractive sites for new development, or that residents would welcome additional jobs and services. Thus, it is unclear whether and in what direction preexisting differences might bias regression results.

Exhibit 7 shows average employment density near station and control areas over time, indicating years in which groups of stations opened. Because stations opened intermittently over a relatively long period that includes several business cycles, we try to distinguish the effect of the stations Exhibit 7

Employment in Study Areas, 1992–2009

easarol tron

e, c

yees per acr

Emplo

Source: Calculations based on National Establishment Time Series database

Cityscape 179

Schuetz, Giuliano, and Shin

from changes in general economic conditions. Average employment densities in both station and control areas show some cyclical movements between 1992 and 2009, decreasing during the reces-sions of the early 1990s, early 2000s, and in the Great Recession from 2007 to 2009 (exhibit 7). These cyclical variations generally match time trends in employment density for Los Angeles County as a whole. However, there is no clearly apparent time trend among the study areas, nor does the graph show clear visual evidence of employment changes around station opening dates.

To focus more clearly on the time periods of interest, exhibit 8 shows average annual employment density, beginning 3 years before station opening and ending 5 years after station opening. The employment analysis includes only the 23 stations and matched control areas for which at least 3 years of preopening employment data are available.9 The year of opening is defined for each station/

control area, so that  t  represents different years for each cluster of stations/controls. Although 0

average employment density levels differ substantially between stations and control areas, the time trends before opening are quite similar; employment is virtually flat during the prestation years and for 1 year afterward (exhibit 8). Station areas show a modest increase between years 2 and 5, from about 32 employees per acre to about 34 employees per acre. Control areas have flat employment Exhibit 8

Employment Density, Before and After Station Opening

CI = confidence interval. 

Notes: Average values and 95-percent confidence intervals for station and control areas. Excludes three stations that opened in 1993 and matched control areas.

Source: Calculations based on National Establishment Time Series database

9 Dropping the three earliest Red and Purple Line stations reduces the average employment density among stations by roughly one-half, from about 60 employees per acre to about 30, because the earliest stations include the highest density employment centers in downtown Los Angeles. 

180 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles

density through year 3 after station opening, then an increase of slightly less than 1 employee per acre from years 3 through 5. Both station and control areas have fairly wide 95-percent confidence intervals around the estimated line, suggesting that the slopes are not significantly different than 0.

The three rail lines in our sample differ by opening year, and run through different parts of the city, so the averages for all stations may conceal differences in time trends across lines. Exhibits 9 

and 10 show employment density before and after opening for stations and controls along the Red and Gold Lines.10 Employment densities around Red Line stations and controls are trending slightly upward during the 8-year window around station opening, but the slopes are not significantly 

different from one another, and there is no indication of a change in slope after opening (exhibit 9). 

Gold Line stations have employment densities close to their control areas, and show different time trends; employment is trending upward around station areas and downward near control areas 

(exhibit 10). However, the confidence intervals for both station and control areas are quite wide and almost completely overlap, so we cannot infer significant differences between them from the graphs. Neither stations nor controls show changes in slope after station opening. Replicating exhibits 8 through 10 for longer time intervals show similar patterns (results available from authors by request).

Exhibit 9

Employment Density Around Red Line Areas, Before and After Station Opening

CI = confidence interval. 

Notes: Average values and 95-percent confidence intervals for station and control areas. Excludes three stations that opened in 1993 and matched control areas.

Source: Calculations based on National Establishment Time Series database

10 Only three stations on the Purple Line have 3 years of preopening employment data. 

Cityscape 181

Schuetz, Giuliano, and Shin

Exhibit 10

Employment Density Around Gold Line Areas, Before and After Station Opening

CI = confidence interval. 

Notes: Average values and 95-percent confidence intervals for station and control areas. Excludes three stations that opened in 1993 and matched control areas.

Source: Calculations based on National Establishment Time Series database

Before estimating regressions, we compare our main employment outcomes for the 3 years before and after station opening. We calculate average employment density and share of employment 

in each of the four industry categories for station and control areas over 3 years, before and after opening (exhibit 11). Using a 3-year window allows for the possibility that employment 

patterns might begin changing prior to opening due to anticipation, or that it changes might take several years after opening to become evident.11 None of the five employment outcome variables show significant changes from the 3 years prior to station opening to the 3 years after opening, either in station or control areas. Among station areas, there are small increases in employment density, commercial employment share, and public/institutional employment share, but none of 

these differences are statistically different from 0 or substantively large in magnitude. The largest change is a nearly 3-percentage-point decrease in industrial employment share, but this is also not statistically significant. Among control areas, overall employment density is essentially the same before and after opening years of the matched station areas, and there are no significant changes in employment composition. Consistent with exhibits 7 through 10, results indicate substantially higher employment levels around stations than in control areas, but do not indicate changes in employment levels shortly after station opening.

11 We have examined annual data for these intervals separately for each station and for groups of stations that open in the same year, because the impact of opening might vary across points in the economic cycle. No observable time trends are within the 3-year windows, nor does apparent variation occur in time trends across stations. The annual data are reasonably smooth, not displaying large year-over-year variations that would raise concerns about short-term noise-to-signal ratios. Therefore, the remaining analysis will use annual employment metrics to allow for clean identification of before-and-after periods. 

182 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles Exhibit 11

Employment Changes, Before and After Station Opening 

Station Areas

Control Areas

Preopening Postopening Difference Preopening Postopening Difference

Employment density, 

31.5 

32.0 

0.47

10.6

10.6

– 0.07

all sectors

(4.0)

(4.0)

(1.0)

(0.9)

Commercial (%)

47.5 

49.5 

1.97

46.6 

47.2 

0.55

(2.4)

(2.1)

(1.9)

(1.9)

Public/institution (%)

21.8 

22.7 

0.82

22.0

22.3

0.31

(2.4)

(2.2)

– (1.7)

– (1.8)

Industrial (%)

22.5 

19.7 

– 2.77

22.9 

22.3 

– 0.62

(1.3)

(1.2)

(1.4)

(1.1)

Miscelleaneous (%)

8.1 

8.1 

– 0.03

8.5

8.2

– 0.24

(0.5)

(0.6)

– (0.8)

– (0.7)

n

69

69

117

117

Notes: Excludes five stations that opened before 1996 and nine matched control areas, because authors cannot observe 3 

years of preopening employment. Standard errors shown in parentheses. None of the differences are statistically significant at the 10-percent level or above.

Regression Results: Employment Changes

As a more rigorous test of whether employment around stations changed after stations opened, 

we estimate a series of regressions summarized in exhibit 12. We estimate preopening and postopening differences for three time windows: (1) 3 years before station opening to 5 years after, (2) 5 years before to 10 years after, and (3) using the entire set of years available in the dataset. 

Stations that opened prior to 1996 are excluded from the regression analysis, because we do not observe preopening employment.12

Regression results generally confirm the findings from graphs and descriptive statistics: station areas had higher initial employment density than control areas, and saw no immediate changes in employment following station openings. However, there is some evidence that employment may 

increase in the 5- to 10-year window after stations open. In the simple before-and-after analysis (columns 1 through 3), the coefficient on  Post*Station increases in magnitude as the time window around station opening expands, only becoming statistically significant when including all years (column 3). The magnitude suggests a 34-percent increase in employees per acre (from an average of 67 prior to opening) over the entire duration of poststation years. However, we can only observe 10 years of postopening employment for eight stations, up to 9 years of postopening employment for another three stations, while we observe at most 6 years of postopening employment for the 12 Gold Line stations. This suggests that the employment gains discerned in the regression occur for the stations that opened from around 1996 to 1999, and became evident toward the latter 

part of the study period. Because that period coincides with the Great Recession, it may in fact be that those station areas lost less employment during the downturn than control areas, rather than experienced absolute employment gains.

12 Estimating the regressions for variations on these time windows, including 3 years prior to opening to 5 or 10 years after opening yields very similar results. Including stations that opened prior to 1996 does not alter the estimated coefficients but is conceptually less clean.

Cityscape 183

Schuetz, Giuliano, and Shin

Exhibit 12

Regression Results on Employment Density, Before and After Station Opening

Dependent  

ln(Employees/Acre)

ln(Employees/Acre)

Variable

(1)

(3)

(4)

(5)

(7)

(8)

Time window

– 3 ≤ t ≤ 5

– 5 ≤ t ≤ 10

All yrs

– 3 ≤ t ≤ 5

– 5 ≤ t ≤ 10

All years

Station

0.906***

0.857***

0.663**

0.883***

0.842***

0.850***

(0.254)

(0.250)

(0.257)

(0.251)

(0.261)

(0.259)

Post

0.079

– 0.002

– 0.100

(0.171)

(0.150)

(0.110)

Post*station

– 0.030

0.112

0.344**

(0.056)

(0.084)

(0.143)

YrsPre

– 0.052

-0.038

– 0.007

(0.089)

(0.086)

(0.085)

Station*YrsPre

0.014

– 0.003

– 0.0511**

(0.016)

(0.015)

(0.021)

YrsPost

0.036 

0.038 

0.046 

(0.089)

(0.085)

(0.083)

Station*YrsPost

– 0.004

0.0411**

0.0563***

(0.011)

(0.020)

(0.018)

Group fixed effects

Yes

Yes

Yes

Yes

Yes

Yes

Time controls

Yes

Yes

Yes

Yes

Yes

Yes

Observations

558

872

1,116

558

872

1,116

R-squared

0.296

0.321

0.299

0.297

0.327

0.312

***  p  < 0.01. **  p  < 0.05. *  p  < 0.1.

Notes: All models include year and year-squared, log of population density and income, and group fixed effects. Robust standard errors, clustered by study area, in parentheses.

Next we estimate a parallel set of regressions using continuous number of years before and after station opening (exhibit 12, columns 4 through 6). Results are very similar to those using a binary indicator for after opening: the coefficient on  Station*YrsPost does not become positive and significant until the study window includes up to 10 years after opening (column 5), and increases in both magnitude and significance when using the full set of years (column 6). The magnitudes of the coefficients suggest a 4- to 6-percent increase in employment per acre each year after opening, with most of this coming from the early-opening stations in the 5 to 10 years afterward. These annual numbers are roughly consistent with the 34-percent increase for all poststation years from column 3. 

The example of the Hollywood and Vine station on the Red Line illustrates why development may substantially lag station opening. The Los Angeles Community Redevelopment Agency used eminent domain to assemble parcels near the station, enabling LA Metro to undertake a large-scale redevelopment project, complete with high-density multifamily housing, a hotel, and ground-floor commercial uses (Schuetz, Giuliano, and Shin, 2017). Even with unusually concerted efforts by public agencies, the redevelopment project was completed in 2009, 10 years after the station opened. This example raises questions about how quickly redevelopment may become apparent in aggregate data. Because we only observe 10 years of postopening data for a few stations, we cannot infer whether the lag reflects true redevelopment times or some unobserved characteristics for the particular set of stations. 

Similar regressions that estimate employment density separately for the Red and Gold Lines, and by geographic submarket, yield no significant results on  Post*Station, even among the oldest station clusters in Central Los Angeles and North Hollywood (results available from authors upon request).13

13 These regressions were estimated for Central Los Angeles, North Hollywood, Arroyo Seco, and Pasadena submarkets. 

Only one station in downtown Los Angeles opened after 1996, so we exclude the DTLA cluster.

184 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles While employment levels may adjust slowly because of the time needed to construct or reconfigure buildings, the composition of employment across industries could adjust more rapidly using 

existing space. Therefore we estimate a set of regressions on the employment shares across four industry categories, over 5- and 10-year windows after station opening (exhibit 13). The coefficients from 3 years before to 5 years after opening (columns 1 to 4) show similar results to the difference-in-means tests shown in exhibit 11. During the immediate 5-year period after station opening, employment in station areas shifted toward commercial and public/institutional jobs, away from industrial and miscellaneous sectors, although the changes are not significantly different from control areas. Over the longer time period, up to 10 years after station opening, there were significant gains in public/institutional employment shares relative to control areas, at the expense of employment in the other three industry categories (although none of the negative coefficients are statistically significant). One possible explanation for this shift in overall employment composition is that public sector organizations near stations, including medical facilities and schools, had relatively smaller employment losses during the Great Recession than private sector firms. These results also may indicate greater public investment around stations; for example, new buildings for the California Transportation Department and the Los Angeles Police Department were constructed around the Civic Center station after station opening.

Exhibit 13

Employment Density, by Industry Category

Dependent 

Percent of Employees in  

Variable

Industry Category

Time Window

t-3 to t+5

t-5 to t+10

Industry  

Commer- Public/ 

Miscella- Commer-

Public/ 

Miscel-

Industrial

Industrial

Category

cial

Institution

neous

cial

Institution

laneous

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

Station

5.56

– 1.62

– 2.84

– 1.10

8.324*

– 4.09

– 3.06

– 1.18

(4.85)

(5.71)

(3.55)

(1.93)

(4.65)

(5.44)

(3.37)

(1.69)

YrsPre

0.86

– 1.94

0.74

0.35

0.49

– 2.03

1.32

0.22

(1.57)

(1.58)

(1.10)

(0.85)

(1.57)

(1.57)

(1.02)

(0.85)

Station*YrsPre

0.26

– 0.37

0.17

– 0.06

– 0.50

0.60

– 0.15

0.05

(0.50)

(0.58)

(0.47)

(0.28)

(0.41)

(0.48)

(0.40)

(0.20)

YrsPost

– 0.90

1.37

– 0.05

– 0.42

– 0.65

0.96

0.04

– 0.35

(1.53)

(1.56)

(1.09)

(1.21)

(1.57)

(1.62)

(0.98)

(0.85)

Station*YrsPost

0.66

0.07

– 0.55

– 0.04

– 0.61

1.171**

– 0.49

– 0.07

(0.59)

(0.54)

(0.49)

(0.33)

(0.44)

(0.57)

(0.38)

(0.21)

Observations

558

558

558

558

872

872

872

872

R-squared

0.237

0.158

0.129

0.111

0.226

0.154

0.122

0.118

***  p  < 0.01. **  p  < 0.05. *  p  < 0.1.

Notes: All models include controls for population density, income, year and year-squared, and group fixed effects. Robust standard errors, clustered by study area, in parentheses.

Conclusions and Policy Implications

The Los Angeles metropolitan area is one of several U.S. regions that have recently made substantial public investments in subway or light rail systems. Developing new transit infrastructure can have multiple goals, including increasing access to existing job centers or public facilities, encouraging high density housing near transit and retail, reducing the growth of vehicle traffic and road Cityscape 185

Schuetz, Giuliano, and Shin

congestion, and spurring physical and economic development. In this article, we examine how 

employment patterns have changed around newly opened rail stations in Los Angeles during the 

last two decades. Although this study focuses on Los Angeles, the results are likely to be relevant to transit planners who are building or expanding rail networks (including streetcars) in cities with similar urban environments, particularly with low transit ridership, multiple employment subcenters, and a densely built urban core.

Results indicate that employment densities in station and control areas fluctuated somewhat over time with regional economic cycles, but there are few clear time trends among study areas. Station areas did not see stronger employment growth within the first 5 years after station opening, but a small group of stations that opened between 1996 and 1999 saw significant employment gains 

between 5 and 10 years after stations opened. 

The relatively scattershot and delayed employment gains near stations most likely reflect two features of the LA Metro system. First, rail transit ridership in Los Angeles is quite low, relative to other large U.S. cities. Due to complicated political considerations that drove the route planning—and perhaps the need to avoid established residential neighbors who opposed the rail—Los Angeles’s rail stations were located in areas with high initial job density, although the system did not create direct connections to important job centers on the city’s west side. The polycentric employment structure in the Los Angeles metropolitan area makes it difficult for most households to complete the home-to-work journey entirely by rail, therefore it is unclear that most rail stations increase neighborhood accessibility and will result in higher land values. The employment gains around Red and Purple Line stations 5 to 10 years after opening also coincides roughly with the opening of the Gold Line. It is possible that proximity to the older stations became more valuable once the Metro system expanded. 

Second, many of the stations are located in densely developed areas with highly fragmented land ownership, so that large-scale redevelopment will require complex and costly land assembly, which adds to the uncertainty and time needed for development. Relative to the legacy systems in New York City and Boston, or even systems like Washington, D.C.’s Metro and San Francisco’s BART, LA Metro stations may be too new for land use patterns to have adjusted. The example of the Hollywood and Vine station suggests that, even in areas with strong market demand and TOD-friendly zoning, it may take a decade or more for changes to land use patterns and physical development to emerge.

One potentially important factor our study cannot address is the role of zoning or other localized policy interventions in facilitating development around stations. A parallel qualitative study reveals that the type and density of buildings allowed under zoning varies substantially across sample stations (Schuetz, Giuliano, and Shin, 2017). High-density residential and commercial uses consistent with TOD are allowed near all stations in downtown Pasadena and some parts of downtown 

and central Los Angeles, but many stations have complex or ambiguous zoning that may hinder 

redevelopment. Los Angeles and Pasadena also demonstrate fundamentally different approaches to land use planning near transit stations. Pasadena adopted new, density-friendly zoning around all its downtown station areas around the time that Gold Line service began. By contrast, Los Angeles has conducted only piecemeal rezoning or granted variances around selected stations, and those changes were not always implemented when stations opened. More recently, LA Metro has begun 

a TOD Planning Grant program to help local governments revise their land use regulations around 186 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles stations in ways that can accommodate and encourage development.14 This offers one alternative way to coordinate zoning and infrastructure development across multiple agencies; evaluating its effectiveness will be an interesting area for future research.

The experience of Los Angeles offers two key lessons for policymakers in other regions. First, even if rail networks generate long-run economic spillovers to surrounding areas, short-run impacts may be quite limited, especially in regions without strong public transportation usage. Second, transit infrastructure may be intended to serve multiple goals, each of which implies different strategies for station location, coordinating policies, and metrics of success. For instance, if the primary goal is to facilitate access of workers to existing jobs, then stations should be located near large employment centers and near dense residential areas with high proportions of workers who commute to those employment centers. However, residential and commercial areas with high prior density may offer less potential (or require more time) for additional development. If the primary goal is to encourage more or denser residential development, then placing stations in greenfields areas and revising the nearby zoning to allow dense mixed-use development may be a more effective location strategy. Real-istically, however, Los Angeles’ example suggests that political feasibility may be at least as important as economic efficiency in driving both station placement and coordinating land use policies.

Appendix

Exhibit A-1

Industry Categories, by Two-Digit NAICS Code

Category

NAICS Sector

NAICS2

Commercial

Retail trade

44

Information

51

Finance and insurance

52

Real estate and rental and leasing

53

Professional, scientific, and technical services

54

Management of companies and enterprises

55

Arts, entertainment, and recreation

71

Accommodation and food services

72

Industrial

Mining

21

Utilities

22

Construction

23

Manufacturing

31

Wholesale trade

42

Transportation and warehousing

48

Administrative and support and waste management and remediation

56

Public/Administrative Educational services

61

Health care and social assistance

62

Public administration

92

Miscellaneous

Agriculture, forestry, fishing and hunting

11

Other services

81

NAICS = North American Industry Classification System.

14 https://www.metro.net/projects/tod/.

Cityscape 187

Schuetz, Giuliano, and Shin

Acknowledgments

This research was supported by a grant from the John Randolph and Dora Haynes Foundation. 

The analysis and conclusions set forth are solely the responsibility of the authors. The authors benefited from thoughtful comments from Steve Billings, Ed Coulson, Mark Shroder, participants at the Lincoln Institute for Land Policy’s Urban Economics and Public Finance Meetings, and several anonymous reviewers.

Authors

Jenny Schuetz is a David M. Rubenstein Fellow at the Brookings Institution.

Genevieve Giuliano is the Margaret and John Ferraro Chair in Effective Local Government at the University of Southern California.

Eun Jin Shin is an assistant professor at the Yale-NUS College.

References

Alonso, William. 1964.  Location and Land Use. Cambridge, MA: Harvard University Press.

Anas, Alex. 1995. “Capitalization of Urban Travel Improvements Into Residential and Commercial Real Estate: Simulations With a Unified Model of Housing, Travel Mode and Shopping Choices,” 

Journal of Regional Science 35 (3): 351–375.

Baum-Snow, Nathaniel, and Matthew Kahn. 2005.  Effects of Urban Rail Transit Expansions: Evidence From Sixteen Cities, 1970–2000. Brookings-Wharton Papers on Urban Affairs. Washington, DC: Brookings Institution Press.

Billings, Stephen. 2011. “Estimating the Value of a New Transit Option,”  Regional Science and Urban Economics 41: 525–536.

Billings, Stephen, Suzanne Leland, and David Swindell. 2011. “The Effects of the Announcement and Opening of Light Rail Transit Stations on Neighborhood Crime,”  Journal of Urban Affairs 33 

(5): 549–566.

Boarnet, Marlon, and Randall Crane. 1997. “LA Story: A Reality Check for Transit-Based Housing,” 

Journal of the American Planning Association 63 (2): 189–204.

Bollinger, Christopher R., and Keith Ihlanfeldt. 1997. “The Impact of Rapid Rail Transit on Economic Development: The Case of Atlanta’s MARTA,”  Journal of Urban Economics 42: 179–204.

Bowes, David, and Keith Ihlanfeldt. 2001. “Identifying the Impacts of Rail Transit Stations on Residential Property Values,”  Journal of Urban Economics 50 (1): 1–25.

Brooks, Leah, and Byron Lutz. 2016. “From Today’s City to Tomorrow’s City: An Empirical Investigation of Urban Land Assembly,”  American Economic Journal: Economic Policy 8 (3): 69–105.

188 Refereed Papers

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles Brueckner, Jan. 1987. “The Structure of Urban Equilibria: A Unified Treatment of the Muth-Mills Model.” In  Handbook of Regional and Urban Economics, Vol. 2, edited by Edwin S. Mills. Amsterdam: North Holland: 821–845.

Cervero, Robert, and John Landis. 1997. “Twenty Years of the Bay Area Rapid Transit System: Land Use and Development Impacts,”  Transportation Research Part A 31 (4): 309–333.

Debrezion, Ghebreegziabiher, Eric Pels, and Piet Rietveld. 2007. “The Impact of Railway Stations on Residential and Commercial Property Values: A Meta-Analysis,”  Journal of Real Estate Finance and Economics 35: 161–180.

Dubé, Jean, Marius Thériault, and François Des Rosiers. 2013. “Commuter Rail Accessibility and House Values: The Case of the Montreal South Shore, Canada, 1992–2009,”  Transportation Research Part A 54: 49–66.

Elkind, Ethan. 2014.  Railtown: The Fight for the Los Angeles Metro Rail and the Future of the City. 

Berkeley: University of California Press.

Giuliano, Genevieve, and Ajay Agarwal. 2010. Public Transit as a Metropolitan Growth and Development Strategy. Working paper. Washington, DC: Brookings Institution.

Glaeser, Edward L., and J.E. Kohlhase. 2004. “Cities, Regions and the Decline of Transport Costs,” 

Papers in Regional Science 83 (1): 197–228.

Guerra, Erick, and Robert Cervero. 2013. “Is a Half-Mile Circle the Right Standard for TODs?” 

ACCESS Magazine 42 (Spring). https://www.accessmagazine.org/spring-2013/half-mile-circle-right-

standard-tods/.

Handy, Susan, Xinyu Cao, and Patricia Mokhtarian. 2005. “Correlation or Causality Between 

the Built Environment and Travel Behavior? Evidence From Northern California,”  Transportation Research Part D 10: 427–444.

Hess, Daniel B., and Tangerine M. Almeida. 2007. “Impact of Proximity to Light Rail Rapid Transit on Station-Area Property Values in Buffalo, New York,”  Urban Studies 44 (5–6): 1041–1068.

Kahn, Matthew. 2007. “Gentrification Trends in New Transit Oriented Communities: Evidence 

From Fourteen Cities That Expanded and Built Rail Transit Systems,”  Real Estate Economics 35 (2): 155–182.

Kolko, Jed. 2011. Making the Most of Transit: Density, Employment Growth, and Ridership 

Around New Stations. Working paper. San Francisco: Public Policy Institute of California.

Lin, Jeffrey. 2002. “Gentrification and Transit in Northwest Chicago,”  Transportation Quarterly 56 

(4): 175–191.

Mathur, Shishir, and Christopher Ferrell. 2013. “Measuring the Impact of Suburban Transit Oriented Developments on Single-Family Home Values,”  Transportation Research Part A 47: 42–55.

McDonald, John, and C.I. Osuji. 1995. “The Effect of Anticipated Transportation Improvement on Residential Land Values,”  Regional Science and Urban Economics 25 (3): 261–278.

Cityscape 189

Schuetz, Giuliano, and Shin

McMillen, Daniel, and John McDonald. 2004. “Reaction of House Prices to a New Rapid Transit 

Line: Chicago’s Midway Line, 1983–1999,”  Real Estate Economics 32 (3): 463–486.

Mills, Edwin. 1967. “An Aggregative Model of Resource Allocation in a Metropolitan Area,”  American Economic Review 57: 197–210.

Muth, Richard. 1969.  Cities and Housing. Chicago: University of Chicago Press.

Nelson, Laura, and Dan Weikel. 2016. “Billions Spent, but Fewer People Are Using Public Transportation in Southern California,”  Los Angeles Times, January 27.

Poister, Theodore. 1996. “Transit-Related Crime in Suburban Areas,”  Journal of Urban Affairs 18 

(1): 63–75.

Redfearn, Christian. 2009. “How Informative Are Average Effects? Hedonic Regression and Ame-

nity Capitalization in Complex Urban Housing Markets,”  Regional Science and Urban Economics (39) 3: 297–306. 

Renne, J., and Reid Ewing. 2013.  Transit-Oriented Development: An Examination of America’s Transit Precincts in 2000 and 2010. UNOTI Publications Paper No. 17. New Orleans: University of New Orleans.

Ruggles, Steven, Katie Genadek, Ronald Goeken, Josiah Grover, and Matthew Sobek. 2015.  Integrated Public Use Microdata Series: Version 6.0 [Machine-readable database]. Minneapolis: University of Minnesota.

Schuetz, Jenny. 2015. “Do Rail Stations Encourage Neighborhood Retail Activity?”  Urban Studies 52 

(14): 2699–2723.

Schuetz, Jenny, Gen Giuliano, and Eun Jin Shin. 2017. “Does Zoning Help or Hinder Transit-

Oriented (Re)Development?”  Urban Studies. DOI: https://doi.org/10.1177/0042098017700575.

Taylor, Brian, Eugene Kim, and John Gahbauer. 2009. “The Thin Red Line: A Case Study of Political Influence on Transportation Planning Practice,”  Journal of Planning Education and Research 29: 173–193.

Winston, Clifford, and Vikram Maheshri. 2007. “On the Social Desirability of Urban Rail Transit Systems,”  Journal of Urban Economics 62: 362–382.

190 Refereed Papers

Departments

In this issue—

• Data Shop

• SpAM

• Evaluation Tradecraft

• Correction

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 191

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

192 Departments

Data Shop

Data Shop, a department of  Cityscape , presents short articles or notes on the uses of data in housing and urban research. Through this department, the Office of Policy Development 

and Research introduces readers to new and overlooked data sources and to improved 

techniques in using well-known data. The emphasis is on sources and methods that 

analysts can use in their own work. Researchers often run into knotty data problems 

involving data interpretation or manipulation that must be solved before a project can 

proceed, but they seldom get to focus in detail on the solutions to such problems. If you have an idea for an applied, data-centric note of no more than 3,000 words, please send 

a one-paragraph abstract to david.a.vandenbroucke@hud.gov for consideration. 

First-Time Homebuyers: 

Toward a New Measure

Arthur Acolin    

University of Washington

Paul Calem  

Federal Reserve Bank of Philadelphia

Julapa Jagtiani  

Federal Reserve Bank of Philadelphia

Susan Wachter  

University of Pennsylvania

Abstract

Existing data sources show divergent estimates of the number of homes purchased by 

first-time homebuyers as a share of all home purchases. In this article, we use a new 

dataset to construct a time series of the share of first-time homebuyers. This series, 

based on the Federal Reserve Bank of New York Equifax Consumer Credit Panel, 

shows a significant decline in the share of first-time homebuyers, particularly among 

young households, consistent with the decline in homeownership in this age cohort since 

the early 2000s.

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 193

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Acolin, Calem, Jagtiani, and Wachter

Introduction

The rate of homeownership in the United States has declined since the financial crisis and recession of 2007 through 2009. As of the third quarter of 2017, the U.S. homeownership rate was 63.9 

percent, substantially below its peak of 69 percent in 2004 and near a 50-year low. The decline in homeownership is particularly pronounced among young and minority households (Acolin, 

Goodman and Wachter, 2016). No disagreement exists on the measurement of this decline, which 

is based on census data reported quarterly (U.S. Census Bureau, 2017).1 Disagreement does exist, however, on an important component of aggregate homeownership: first-time homebuyers as a 

share of all purchasers.

The estimated number of first-time homebuyers is an important statistic because it provides data on initial access to homeownership for those who may be renters or living with their parents. It also has implications for future economic activity in the housing sector because households typically move from rentals to starter homes and then to larger homes over the life cycle. Nonetheless, no comprehensive data source is currently available on the number or share of first-time homebuyers in the United States. These statistics are not measured in the decennial census, the annual American Community Survey (ACS), or the monthly Current Population Survey (CPS). A measure 

of first-time homebuyers exists in the biennial American Housing Survey (AHS) but is limited by the AHS sampling frame and thus cannot be used to identify changes over time for the U.S. market in aggregate.2 In the absence of census data, two alternative data sources have been developed and used to provide updates on first-time homeownership trends. These data sources report divergent trends in the share of first-time buyers.

The first source is a measure developed by the American Enterprise Institute (AEI) and the Urban Institute (UI) based on mortgage origination data, which shows no decrease since the subprime mortgage crisis. The second source, a survey developed by the National Association of Realtors® 

(NAR), finds a substantial decrease in the share of first-time homebuyers over time. The AEI/UI measure provides data on the share of first-time homebuyers among users of agency debt, and the NAR measure uses the share among all purchasers, including cash purchasers. In this article, we develop a new measure of first-time homebuyers, using all sources of mortgage funding. Using this measure, we compare, over time, the use of mortgage debt for first-time purchases of homes by age group and by credit score characteristics of borrowers.

We make use of the Federal Reserve Bank of New York Equifax Consumer Credit Panel (CCP), 

which has been developed with the goal of tracking credit usage, including the use of mortgages in the aftermath of the 2007 subprime crisis (Lee and Van der Klaauw, 2010). This dataset consists of a nationally representative 5-percent sample of credit records.

Using the CCP, we measure the number of first-time homebuyers during the 2002-through-2015 

period by identifying borrowers who previously did not have a mortgage and combine this 

1 Available at http://www.census.gov/housing/hvs/data/histtabs.html.

2 Including a question about first-time homebuyers in the ACS, the CPS/Housing Vacancy Survey, or both would provide a source of reliable estimates. In its absence, the challenges in estimating the number of first-time homebuyers are similar to those faced in obtaining a reliable measure of household formation (Masnick, Giordmaina, and Belsky, 2010; McCue, Masnick, and Herbert, 2015).

194 Data Shop

First-Time Homebuyers: Toward a New Measure

measure with U.S. Department of Housing and Urban Development (HUD) data on all home 

purchases to derive a measure of the trend in first-time homebuyers as a share of overall purchases over time.

The rest of the article is organized as follows. In the following section, we review homeownership shares and first-time homebuyer shares among all home purchasers, based on measures developed in previous studies. The subsequent section applies the new measure of first-time homebuyers using the CCP, to reconcile the different measures of first-time homebuyers, and relates the findings to changes in the funding sources for access to homeownership. The conclusion follows.

First-Time Homeownership Trends: Measurement Over Time

Exhibit 1 provides time series data on the share of first-time homebuyers from the two widely used existing measures, (1) the NAR survey-based data and (2) the composite data from the government-sponsored enterprises (GSEs) Fannie Mae and Freddie Mac and from the Federal Housing 

Administration (FHA), which AEI and UI have used to report on first-time homeownership trends (American Enterprise Institute, 2015; 2017; Bai, Zhu, and Goodman 2015).

The two series provide broadly similar measures from 2001 to 2005, with first-time homebuyers relatively constant at about 40 percent of overall homebuyers. The NAR measure shows a decrease in first-time homebuyer share in 2006. Both measures exhibit increases after 2006, which continue until 2009 (AEI/UI) or 2010 (NAR), although the NAR trend line increases less. After that, major Exhibit 1

Comparison of the Measures of the Share of First-Time Homebuyers

70  

 %) 60  

50  

40  

30  

20  

10  

Share  of  first-‐-me  homebuyers  (

0  

2001   2002   2003   2004   2005   2006   2007   2008   2009   2010   2011   2012   2013   2014   2015   2016  

NAR  

GSEs  and  FHA  

FHA = Federal Housing Administration. GSE = government-sponsored enterprise. NAR = National Association of Realtors®.

Sources: NAR; Federal Housing Finance Agency; FHA

Cityscape 195

Acolin, Calem, Jagtiani, and Wachter

differences emerge in the reported trends of first-time homebuyer share. The AEI/UI estimate 

shows the share of first-time homebuyers currently far above the 40-percent rate that prevailed before the housing market downturn, to rates of about 60 percent, with peaks of 63 percent in 2009 and 58 percent in 2016. The NAR measure shows the share of first-time homebuyers peak 

at 50 percent in 2010, but then decrease to a near 30-year low of 32 percent in 2015, increasing to 35 percent in 2016. As of 2016, the difference between the NAR and the AEI/UI measures is 23 

percentage points.

These measures diverge due to differences in data sources and methodology used to identify first-time homebuyers. As noted, the NAR measure relies on survey results of all buyers. NAR sends a questionnaire to a random sample of primary resident homebuyers within the previous year (for example, home purchases between June 2014 and June 2015 for the 2015 results). The survey 

based on all purchase transactions includes both cash transactions and mortgage transactions.3

The measure AEI and UI use relies on historical monthly data from Fannie Mae, Freddie Mac, 

and FHA. The data include mortgages sold to the GSEs and mortgages covered by FHA mortgage 

insurance. Purchases made with mortgages held in lenders’ portfolios, cash purchases, and noninstitutional privately transacted mortgages are not included.4 

With the disappearance of subprime lending and withdrawal of private securitization from the 

mortgage market after 2006, first-time homebuyers became concentrated in the FHA, Fannie Mae, and Freddie Mac segments of the mortgage market (with jumbo loans concentrated in the bank 

portfolio segment). Thus, the FHA, Fannie Mae, and Freddie Mac segments became less represen-

tative of the market as a whole and would be apt to overstate the share of first-time homebuyers. 

In addition, as Urban Institute (2016) pointed out, the exclusion of cash transactions in the AEI/

UI measure increases the reported share of first-time buyers relative to the NAR estimate because first-time homebuyers are less likely to be cash buyers.

The CCP Data on First-Time Homeownership

Almost all first-time homebuyers use debt. A comprehensive measure of the share of first-time homebuyers using mortgage debt can be constructed using the Federal Reserve Bank of New York 

Equifax CCP. The CCP provides a broad measure of first-time homebuyers who use mortgage debt. 

The CCP dataset also includes individual characteristics such as age, credit score, and information about household-level credit and debt—including second-lien mortgages, credit cards, automobile loans, and student loans—and thus can be used to analyze the relationship among different uses of debt and access to homeownership using a mortgage.

3 The survey is restricted to principal residence purchases only; it does not include investor or vacation homes. For the 2016 “Profile of Home Buyers and Sellers,” NAR mailed a survey to a random sample of 93,171 recent homebuyers who had purchased a home between July 2015 and June 2016. Of the responses received, 5,465 responses were from primary residence buyers, resulting in an adjusted response rate of 5.9 percent after accounting for undeliverable addresses (National Association of Realtors®, 2016).

4 The Federal Housing Finance Agency first made these historical databases available to the public in 2013. In the data, first-time homebuyers are defined as those individuals who did not own a property with a mortgage within the past 3 years (FHFA, 2013).

196 Data Shop

First-Time Homebuyers: Toward a New Measure

The CCP provides quarterly detailed credit report data dating back to 19995 for a 5-percent representative random sample of individuals with at least one credit record. The random selection of the database is based on consumers with a Social Security number (randomizing the last two digits of the Social Security number). The database follows the characteristics and credit performance for these 5 percent of randomly selected consumers (so-called Primary consumers) for the entire data period (until their death or bankruptcy filing, resulting in no credit record for at least 6 months). 

In addition to data on the Primary consumers, the CCP also reports characteristics and credit performance of all the consumers who live in the same household (same address) as the Primary consumers (although their characteristics would stop being reported once they moved out of 

the Primary consumer’s household). The dataset of the consumer-quarter panel is not a balanced panel—exits occur due to factors such as young adults acquiring a credit record and death.

This longitudinal panel of individuals and households makes it possible to identify households that have taken out a mortgage and did not have one previously. We focus on the household level because the Primary consumers in the CCP may not be the head of the household (and thus would not have mortgage debt), but others in the household who may not be the Primary consumer 

would have mortgage debt. In addition, when the head of the household passes homeownership 

on to another household member, we also capture that it is not a new homebuyer. Using the CCP 

dataset, we identify first-time homeownership with a household that obtains a mortgage in a 

given quarter and shows no mortgages reported in its credit file during the previous 3 years.6 The 3-year prior observation window is similar to the one used in the AEI/UI measure. Note that some households identified as first-time homeowners by this criterion may have owned a home but did not have a mortgage (originally having purchased the home with cash or having already paid it off), but such cases should be relatively infrequent.

The CCP provides information about the number of new homebuyers using a mortgage but not 

about the total number of home purchase transactions regardless of whether they involved a 

mortgage. Therefore, we use HUD estimates about new and existing homes sold (HUD, 2017), 

which aim to capture all transactions as the denominator.7 Exhibit 2 shows the total number of home sales from HUD and the number of households that take out a first-lien purchase mortgage in a given year and did not have one in the previous 3 years, the CCP-based measure of first-time homebuyers.8

Given that the CCP data series starts in 1999, the measure of first-time homebuyers, with a 3-year lag, begins in 2002 and goes until 2015. The number of first-time homeowners is estimated to be more than 3 million each year during the period 2002 to 2005, peaking at almost 3.5 million in 5 Equifax has provided this dataset to the Federal Reserve System since 2010 (Lee and Van der Klaauw, 2010). A drawback of this dataset is that it is not available to the public.

6 Gyourko, Lee, and Tracy (2015) also used this method.

7 We use HUD data for the denominator due to the difficulty of identifying owner-occupied home purchase mortgages in Equifax. This approach is limited due to the potential biases and noise introduced by relying on two different sources for numerator and denominator. However, it should capture the universe of home purchases in a given year.

8 The total number of first-time homebuyers is very sensitive to various sample construction methods in the CCP; however, the trend of declining first-time homebuyers is robust across multiple possible definitions. Appendix A provides details and assumptions of the steps followed to identify household members in the CCP and limits associated with the CCP household definition.

Cityscape 197

Acolin, Calem, Jagtiani, and Wachter

Exhibit 2

Total Number of Home Sales Versus First-Time Homebuyers (2002–2015)

9,000,000  

8,000,000  

7,000,000  

6,000,000  

s  

5,000,000  

e  sale

4,000,000  

Hom

3,000,000  

2,000,000  

1,000,000  

0  

2002   2003   2004   2005   2006   2007   2008   2009   2010   2011   2012   2013   2014   2015  

First-‐3me  homebuyers  

Total  transac3ons  from  HUD  

HUD = U.S. Department of Housing and Urban Development.

Sources: Federal Reserve Bank of New York Equifax Consumer Credit Panel; HUD database

2004. It then declines to 1.9 million in 2011 and remains very stable around 2 million through 2015, which is well below the historical levels. As of 2015, the number of first-time homebuyers is estimated to be roughly one-third below the level experienced during the housing boom.9

Using the HUD data to measure total home purchases, first-time homebuyers comprised about 40 

percent of homebuyers between 2002 and 2004 (exhibit 3), a share consistent with the two existing measures.10 The share decreases to 37 and 38 percent of all homebuyers in 2005 and 2006, 

respectively, directionally similar to the NAR series. That decline may reflect that, during the peak of the housing boom, many households were trading up and investors represented an increasing 

share of purchases (Haughwout et al., 2011). Subsequently, through the 2007-through-2010 

downturn period, the first-time homebuyer share is elevated, likely reflecting the reduced mobility of existing homeowners (due to the drop in home prices) and a drop in first-time, all-cash buyers with a greater investment motive. Between 2010 and 2013, the share of first-time buyers declines sharply in this series and has remained low since then. The share of first-time homebuyers represents about 35 percent of all transactions in 2015.

9 Given the natural exit from homeownership of households transitioning from owning to renting or passing away, and added exit due to foreclosures, the decline in the number of first-time homeowners resulted in no net change in the number of homeowners in the 2006-to-2016 period. As of 2016, the number of homeowner households was 75.0 million compared with 75.4 million in 2006 (U.S. Census Bureau, 2017).

10 This share would be lower if we used never having had a mortgage instead of not having had one within the last 3 years.

198 Data Shop

First-Time Homebuyers: Toward a New Measure

Exhibit 3

Share of First-Time Homebuyers (2002–2015)

Sources: Federal Reserve Bank of New York Equifax Consumer Credit Panel for first-time homebuyers; U.S. Department of Housing and Urban Development for home sales

Further, we break down the total number of first-time homebuyers by age group. Across all age groups, we see significant declines in the number of first-time homebuyers from 2004 to 2011 

(exhibit 4). After 2011, the number of first-time homebuyers stabilizes, for each of the age groups, at lower levels than the precrisis highs. Overall, from 2004 to 2015, the number of first-time homebuyers has declined roughly 40 percent for all three age groups. This suggests that the decline in first-time homebuyers is not simply due to younger people delaying home purchases by a few years; rather, purchase behavior seems to have shifted across all age groups.

Exhibit 4

Number of First-Time Homebuyers by Age Group (2002–2015)

1,400,000  

1,200,000  

1,000,000  

800,000  

600,000  

400,000  

200,000  

First-‐'me  homebuyers  

0  

2002   2003   2004   2005   2006   2007   2008   2009   2010   2011   2012   2013   2014   2015  

25–34  

35–44  

45–64  

Source: Federal Reserve Bank of New York Equifax Consumer Credit Panel

Cityscape 199

Acolin, Calem, Jagtiani, and Wachter

We also estimate the share of individuals with a mortgage (as a ratio to all individuals with a credit record) by age groups. The CCP data show that it is younger people, under the age of 35, who 

have experienced the largest drop in the share of individuals with a mortgage (exhibit 5).11 

The CCP allows us to examine the credit characteristics of mortgage debt holders. Exhibit 6 shows the share of first-time homebuyers by credit score level and the share of all households with a mortgage. It shows that the decline in the share of individuals with mortgage debt is most pronounced among lower-credit-score borrowers and that they represent a smaller share of first-time homebuyers in the postcrisis period. This finding is consistent with access to credit playing a role in the decrease in first-time homebuyers, as reported in Bhutta (2015). Another factor that might contribute to delayed access to homeownership is student debt (Elliott, Grinstein-Weiss, and Nam, 2013; Mezza et al., 2016).12 

Exhibit 5

Share of Individuals With Mortgage Debt, First Liens Only, by Age Group (2001–2016)

50  

45  

%)

40  

35  

30  

25  

20  

15  

10  

5  

Individuals  with  mortgage  debt  (

0  

2001   2002   2003   2004   2005   2006   2007   2008   2009   2010   2011   2012   2013   2014   2015   2016  

25–34  

35–44  

45–64  

65+  

Source: Federal Reserve Bank of New York Equifax Consumer Credit Panel

11 These results on the decline in first-time homebuyers by age group imply that the drop in aggregate homeownership is not only the result of the forced transition from owning to renting by households experiencing foreclosure or short sales but also the result of fewer households, particularly younger households, gaining access to homeownership. These results are supported by findings reported in Bhutta (2015), who decomposed the entry and exit of mortgage debtors and found that the decrease in the volume of new entrants far outpaced the volume of exits. 

12 The role of student debt in postponing access to mortgages is consistent with findings that attribute the changes in the homeownership rate to the increased impact of credit constraints (Acolin et al., 2016a; 2016b).

200 Data Shop

First-Time Homebuyers: Toward a New Measure

Exhibit 6

Share of First-Time Homebuyers by Credit Score Level (a) and Share of Individuals 

With a Mortgage by Credit Score Level (b)

Source: Federal Reserve Bank of New York Equifax Consumer Credit Panel

Cityscape 201

Acolin, Calem, Jagtiani, and Wachter

Conclusion

In this article, we present a new measure of first-time homebuyers as a share of all purchasers. 

Existing measures yield divergent assessments of this ratio. NAR provides a measure based on 

a survey, whereas AEI and UI provide measures based on administrative data and include only 

purchases with a mortgage that involved the GSEs or FHA. In this article, we use a new dataset to construct a time series of the share of first-time homebuyers. This series, based on the Federal Reserve Bank of New York Equifax CCP, shows a significant decline in the share. We also find a decline in the number of first-time homebuyers across all age groups relative to the early 2000s.

Appendix A: Data Steps

The Federal Reserve Bank of New York Equifax Consumer Credit Panel is a 5-percent sample of 

all individuals with at least one credit record. These individuals form the principal sample and are given an identification number (ID) that remains constant over time. In addition, in any quarter, a household ID is attributed to all credit records sharing the same address as individuals in the principal sample. Credit records for the members of that household are also available over time (until they move out of the Primary consumer’s household).

We use the following steps to identify the number of first-time homebuyers defined as not having had a first lien mortgage in the previous 3 years and having one in that quarter—

•  Extract all individuals in a given quarter and their household IDs.

•  Drop duplicate household IDs so that each household has only 1 record.

•  Draw a 1-percent sample of households.

•  Match the household IDs from the household sample back to the individual IDs.

•  Drop households with more than 10 records (likely to be multiunit addresses rather than real households). This sample is of individuals in the households used to determine the number of 

first-time homebuyers.

•  Use the individual IDs to extract mortgage history for all individuals in the sample of 

households.

•  Identify whether each individual had a mortgage in any of the previous 12 quarters. If an 

individual had no mortgage in any of these quarters and has one now, identify the individual as a first-time homebuyer, even if missing data during one or more quarters.

•  If any member of the household is a new homebuyer and none of the household members had 

mortgages in the previous 3 years, identify the household as first-time homebuyer.

•  Sum the number of first-time homebuyer households.

•  Extrapolate the number of first-time homebuyer households identified from the sample to the entire population.

202 Data Shop

First-Time Homebuyers: Toward a New Measure

Acknowledgments

The authors thank Raman Quinn Maingi for his research assistance. The views expressed in this paper are those of the authors and do not necessarily reflect those of the Federal Reserve Bank of Philadelphia or the Federal Reserve System. 

Authors

Arthur Acolin is an assistant professor of real estate at the University of Washington.

Paul Calem is a vice president in the Supervision, Regulation and Credit Department at the Federal Reserve Bank of Philadelphia.

Julapa Jagtiani is a special advisor in the Supervision, Regulation and Credit Department at the Federal Reserve Bank of Philadelphia.

Susan Wachter is Albert Sussman Professor of Real Estate and professor of finance at The Wharton School of the University of Pennsylvania.

References

Acolin, Arthur, Jesse Bricker, Paul Calem, and Susan Wachter. 2016a. “Borrowing Constraints and Homeownership,”  The American Economic Review 106 (5): 625–629.

———. 2016b. Borrowing Constraints and Homeownership Over the Recent Cycle. Working 

paper. Philadelphia: Wharton Real Estate Center.

Acolin, Arthur, Laurie S. Goodman, and Susan M. Wachter. 2016. “A Renter or Homeowner Na-

tion?”  Cityscape 18 (1): 145–159.

American Enterprise Institute. 2017. “February 2017 First-Time Buyer Index From AEI’s ICHR.” 

http://www.aei.org/publication/february-2017-first-time-buyer-index-from-aeis-ichr/. 

———. 2015. “June 2015 Mortgage Risk Index From AEI’s ICHR.” https://www.aei.org/publica-

tion/june-2015-mortgage-risk-index-from-aeis-ichr/.

Bai, Bing, Jun Zhu, and Laurie Goodman. 2015.  A Closer Look at the Data on First Time Homebuyers. 

Washington, DC: Urban Institute. http://www.urban.org/sites/default/files/publication/49876/2000210-

A-Closer-Look-at-the-Data-on-First-Time-Homebuyers.pdf.

Bhutta, Neil. 2015. “The Ins and Outs of Mortgage Debt During the Housing Boom and Bust,” 

Journal of Monetary Economics 76: 284–298.

Elliott, William, Michal Grinstein-Weiss, and Ilsung Nam. 2013. Is Student Debt Compromising 

Homeownership as a Wealth Building Tool? Working paper No. 13-33. St. Louis: Center for Social Development.

Cityscape 203

Acolin, Calem, Jagtiani, and Wachter

Federal Housing Finance Agency (FHFA). 2013. “Mortgage Market Note 13-01: A Study of 

First-Time Homebuyers.” https://www.fhfa.gov/PolicyProgramsResearch/Research/Pages/Mortgage-

Market-Note-13-01.aspx.

Gyourko, Joseph, Donghoon Lee, and Joseph Tracy. 2015. “First-Time Homebuyers: The Role of 

Thin Credit Files.” https://www.frbsf.org/economic-research/files/S01_P1_JoeTracy.pdf.

Haughwout, Andrew, Donghoon Lee, Joseph S. Tracy, and Wilbert Van der Klaauw. 2011.  Real 

Estate Investors, the Leverage Cycle, and the Housing Market Crisis. Staff Report No. 514. New York: Federal Reserve Bank of New York.

Lee, Donghoon, and Wilbert Van der Klaauw. 2010.  An Introduction to the FRBNY Consumer Credit Panel. Staff Report No. 479. New York: Federal Reserve Bank of New York.

Masnick, George S., Diane Giordmaina, and Eric S. Belsky. 2010.  Updated 2010–2020 Household and New Home Demand Projections. Cambridge, MA: Joint Center for Housing Studies of Harvard University.

McCue, Daniel, George Masnick, and Chris Herbert. 2015. Assessing Households and Household 

Growth Estimates With Census Bureau Surveys. Working paper W15-5. Cambridge, MA: Joint 

Center for Housing Studies of Harvard University.

Mezza, Alvaro, Daniel R. Ringo, Shane Sherland, and Kamila Sommer. 2016. On the Effect of 

Student Loans on Access to Homeownership. Federal Reserve Working Paper No. 2016-10. Wash-

ington, DC: Board of Governors of the Federal Reserve System.

National Association of Realtors®. 2016. “First-Time Buyers, Single Women Gain Traction in NAR’s 2016 Buyer and Seller Survey.” https://www.nar.realtor/news-releases/2016/10/first-time-buyers-

single- women-gain-traction-in-nar-s-2016-buyer-and-seller-survey.

U.S. Census Bureau. 2017 “Housing Vacancies and Homeownership.” Dataset. https://www.census.

gov/housing/hvs/data/histtabs.html.

U.S. Department of Housing and Urban Development (HUD). 2017. “New and Existing Home 

Sales.” Dataset. huduser.gov/portal/ushmc/hd_home_sales.html.

Urban Institute. 2016. “Do We Have a Generation Stuck in Starter Homes?” www.urban.org/urban-

wire/do-we-have-generation-stuck-starter-homes.

204 Data Shop

Do It Yourself: Obtaining Updated 

Transit Stop and Route Shapefiles 

in Urban and Nonurban Areas

Seva Rodnyansky    

University of Southern California

Abstract

Research that combines housing and transportation aims to jointly understand the 

elements of neighborhood accessibility, affordability, and sustainability. Access to high-quality public transit and nonmotorized transportation helps reduce emissions and 

transportation costs for all households, including those with lower incomes. Transit 

access also expands the range of community destinations and shopping opportunities 

for those without cars. However, researchers often struggle to obtain accurate, geo-

coded data—especially in suburban and nonurban areas—on transit station locations, 

routes, and schedules. This article highlights a newer tool, the General Transit Feed 

Specification (GTFS) from Google, which provides an open source database of updated 

transit data. This free data source combines static and dynamic transit data and can 

be incorporated into analysis using geographic information system, or GIS, software. 

It also significantly eases cross-sectional, rural, and metropolitan-areawide analyses of housing using transportation as a key input. This article summarizes the GTFS data 

type, gives an overview of methods for using the data, explores current uses of the data, and suggests future applications.

Introduction

Accessibility to employment and amenities is a primary input to a household’s choice of residential location. In the monocentric city model, households commute to jobs in the central business 

district and select housing locations by trading off the cost of commuting longer distances versus the higher cost of housing closer to the city center (Alonso, 1964; Brueckner, 1987). Although many U.S. metropolitan areas are less monocentric today than they were in the middle of the 20th century, many households still commute to downtowns or to local employment centers and must 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 205

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Rodnyansky

thus factor transportation into their location decision (Giuliano and Small, 1991; Redfearn, 2007). 

The same is true of other amenities households use—shopping, entertainment, educational, medical, and so on—which tend to cluster in particular locations, necessitating transportation.

Location accessibility depends on the transportation mode available and chosen. For example, a housing development near an interstate highway exit, but with no public transit, may have high accessibility for households with car access but low accessibility for those with no or low car access. 

However, if this highway is routinely congested, the traffic may lower the site’s relative accessibility. If the same housing development had nearby bus access, its relative accessibility would still depend on the distance to the bus stop and on the frequency of bus service. Thus, while both the highway and the bus service may appear nominally accessible, in reality, traffic and transit service constraints may decrease the location’s accessibility. Hence, when making residential location choices, households must optimize housing cost, job and amenity accessibility, mode choice, traffic, and transit service.

Researchers and policymakers who design and evaluate housing policies also need to take ac-

cessibility into account. Combined housing and transportation research aims to understand the combined elements of accessibility, affordability, and sustainability at both the household and neighborhood levels (Haas et al., 2013).

Measuring accessibility to account for transit service and traffic can be complex. Prior measures of location or neighborhood accessibility (for example, Alonso, 1964), used linear (Euclidean) distance, which gives only an approximation and is less suitable for cities with irregular topogra-phy or with grid street layouts, which includes many U.S. cities. More recent commercial Global Positioning System (or GPS)-based tools, such as Google Maps, take street network and congestion into account and have improved accessibility measurements for automobile, public transit, biking, and walking modes. Although Google Maps and similar tools work well for individuals, researchers and policymakers need data to be aggregable and analyzable over various time periods. For driving modes, this goal has been accomplished by using street network analyses in geographic information system (GIS) packages and by aggregating road sensor data, such as the Archived Data Management System (Giuliano, Chakrabarti, and Rhoads, 2016) in Los Angeles, California, but 

this method has helped enhance accessibility measurements for only automobiles, not other modes like public transit, walking, and biking.

Data on public transit stop location, service, and performance on an aggregate basis over multiple time periods are scant, limited to a few of the largest and most sophisticated transit operators in the United States, housed on multiple websites and in a variety of data types. A new data source, the General Transit Feed Specification (GTFS), has solved many of these issues simultaneously by providing a centralized database of current and historical transit stop locations, service times, and performance using the same file type. GTFS enables public transit accessibility measurements that were previously impossible or impractical. In a recent research project, which required obtaining the location of every bus stop in California, GTFS reduced data-gathering time from 3 months to 1 week and increased data completeness from 49 to 88 percent of counties (Bostic and Rodnyansky, 2016). The remainder of this article gives background and tips on using GTFS, showcases relevant research using GTFS, and provides ideas for future use in housing and neighborhood research. 

206 Data Shop

Do It Yourself: Obtaining Updated Transit Stop and  

Route Shapefiles in Urban and Nonurban Areas

GTFS Background

GTFS is an open-data tool connecting transit operators and users of transit data. GTFS is a unified standard file format for sharing transit route, stop, schedule, and performance data, interoperable among transit operators worldwide, regardless of size, language, or transit type.1 The GTFS file format was developed through a partnership between TriMet, the transit operator in the Portland, Oregon metropolitan area, and engineers at Google, Inc. (McHugh, 2013). This initial project helped launch and integrate transit tracking and transit time measurement into Google Maps (McHugh, 2013). The public and institutional success of the Portland example encouraged other U.S. and international transit agencies to adopt the GTFS file format and provide their data to the public. Since the launch of the Portland project, data from at least 1,000 agencies worldwide have become available through GTFS. Moreover, an extension of GTFS called “GTFS-realtime” expands the available data to include real-time trip updates, service alerts, and vehicle positions, enabling more nuanced analyses.2 

GTFS Do-It-Yourself

GTFS is a simple and accessible tool, by design. Researchers, policymakers, and others interested in transit data can download and perform analyses on GTFS data with tools as simple as spreadsheet software (for example, Microsoft Excel), text reader (for example, Windows Notepad), and GIS software (for example, Esri ArcGIS, PitneyBowes MapInfo, or open source QuantumGIS).

Each participating transit agency uploads as many as 13 files describing the various aspects of their transit operations. These files include stop locations and times, service frequencies, routes and route shapes, trips, fare attributes and rules, transfers, service calendars and off days, and agency and feed descriptions.3 Those agencies participating in GTFS-realtime include additional files describing in-time vehicle positions, trip updates, and service alerts. Note that not all agencies choose to upload every file, and not all agencies update their files with every service change.4 The files for each participating agency are downloadable in text, comma separated value, shapefile, or a combination of the three formats, depending on the agency and file type.

After downloading the data, users have several options in operationalizing it, depending on their purpose. At the simplest level, users can examine their data in a spreadsheet or a text reader, if they know the specific route or stop they seek. Most users, however, will want the view of all the stops and all the routes. To enable proper location of stops and routes on a map, GTFS data provide latitude and longitude coordinates. For agencies that provide GIS shapefiles, coordinates and projections will appear automatically once opened in GIS. For agencies that provide text files, users should import the text file into GIS, use the system’s coordinate reader, and set a projection.5 These 1 https://developers.google.com/transit/gtfs/.

2 https://developers.google.com/transit/gtfs-realtime/guides/feed-types.

3 https://developers.google.com/transit/gtfs/examples/gtfs-feed.

4 Some agencies update their GTFS data frequently. For example, the Sacramento Rapid Transit District has GTFS data posted from six time periods from 2013 through 2017 (SACRT, n.d.). 

5 Vance (2016) provides a handy tutorial for QGIS using the Cook County, Illinois Pace bus service. This tutorial readily generalizes to ArcGIS and other GIS software.  

Cityscape 207

Rodnyansky

files can now be integrated with other geographic data. Alternatively, GTFS files can also be converted to a KML file for use with Google Earth.6 To take advantage of the timetables and schedules provided via GTFS, users can plug GTFS into a network dataset for use with Esri ArcGIS Network Analyst, using a custom-written toolbox “Add GTFS to a Network Dataset” (Morang, n.d.). 

The U.S. Department of Transportation (DOT)’s National Transit Map project, through the Bureau of Transportation Statistics, provides a GTFS-derived national map of all transit stops and all agencies whose data are represented (DOT, n.d.). A GIS shapefile of 0.5- and 0.25-mile buffers around fixed-route transit stops is also provided and may be especially useful for those studying transit-oriented development and accessibility at the national level.

For users looking for data from a specific transit agency, no single website provides all GTFS-participating agencies, due to the data’s open-source nature. However, several sources provide overlapping lists of participating agencies and clickable links to download data directly or to the transit agency’s webpage on which the data are hosted (exhibit 1). Housing and neighborhood 

policy researchers can use these sources to complement their analyses with realistic and detailed portrayals of transit accessibility.

To demonstrate an example, I set out to find and display all the bus stop locations in Fresno and Madera Counties in the San Joaquin Valley of California. Fresno County, with a population of 950,000, contains the Fresno metropolitan area, and Madera County, with a population of 

150,000, contains the city of Madera. Fresno County has two main transit operators—Fresno 

County FAX and Clovis Transit—regularly operating 20 routes, and Madera County has 

one—Madera County Transit, with three routes. Using the GTFS Data Exchange listing, I sourced and downloaded the data from each county’s transit feed;7 both were already in shapefile format upon download. I imported both shapefiles into open-source QuantumGIS, in addition to layers 

showing county and city boundaries and a layer showing major roads. Exhibit 2 visualizes both counties’ transit stops with dots demarcating stop locations. Researchers can readily replicate and extend such an analysis and visualization with the available GTFS data.

Exhibit 1

Sources To Find GTFS Data for Specific Transit Agencies

Source Name

Total Transit Agencies Listed

Website

TransitLand

2,090; about 1,000 have GTFS

https://transit.land/feed-registry/ 

GTFS Data Exchange

1,000

http://www.gtfs-data-exchange.com/

agencies 

Transitfeeds

550

http://transitfeeds.com/feeds

Transitwiki.org “Publicly 

401

https://www.transitwiki.org/TransitWiki/

accessible public 

index.php/Publicly-accessible_

transportation data”

public_transportation_data 

Google Code Archive: 

256

https://code.google.com/archive/p/

googletransitdatafeed—

googletransitdatafeed/wikis/

PublicFeeds.wiki

PublicFeeds.wiki

Trillium

150 or more

https://trilliumtransit.com/gtfs/our-work/ 

GTFS = General Transit Feed Specification.

6 See Antrim (2015).  

7 Fresno County data feed: http://data.trilliumtransit.com/gtfs/fresnocounty-ca-us/; Madera County data feed: http://data.

trilliumtransit.com/gtfs/madera-ca-us/.

208 Data Shop

Do It Yourself: Obtaining Updated Transit Stop and  

Route Shapefiles in Urban and Nonurban Areas

Exhibit 2

Visualization of Bus Stop Locations in Fresno and Madera Counties Using GTFS Data

GTFS = General Transit Feed Specification.

Current Uses of GTFS

Current users of GTFS range widely from transit agencies, app and website developers, transit planners, researchers, and others. DOT’s Federal Transit Administration reports that the “GTFS 

format is used by many transit agencies to communicate their schedules to online mapping 

programs and smartphone/tablet applications that travelers use to plan their transit trips” (DOT, 2016). GTFS.org, a website supported by the nonprofit Rocky Mountain Institute, reports six 

major applications of GTFS: (1) trip planning and maps, (2) timetable creation, (3) accessibility for the disabled, (4) planning and analysis, (5) real-time transit information, and (6) public information displays (GTFS.org, n.d.).

The previously mentioned categories intuitively make sense but do not give an understanding of the use of GTFS in research. To assess the depth and breadth of GTFS penetration into research, I conducted a literature scan of GTFS-related scholarly publications, agency reports, and unpublished working papers and theses. I used the GoogleScholar and Google search engines with a 

timeline from 20058 to the present and a targeted list of search terms: GTFS, General Transit Feed Specification, Google Transit Feed Specification, GTFS and geography, GTFS and planning, GTFS 

8 The earliest development of GTFS by TriMet and Google was in 2005.

Cityscape 209

Rodnyansky

and urban, GTFS and rural, and GTFS and transit. The literature scan yielded 121 relevant results, most from 2013 through 2017, published in a variety of journals and fields. Studies were categorized based on their main research angle. More than one-half of the research was specific to the transportation field: transportation planning and analyses, improved transportation modeling, and developing trip planners (exhibit 3). Another 21 studies focused on software, standards, or data development stemming from GTFS, and 6 studies used GTFS for visualization. Finally, less than 20 percent of studies focused on accessibility to jobs, neighborhoods, or amenities—our topic of interest. These accessibility-related studies are most relevant for housing, planning, and neighborhood researchers and set a precedent for incorporating GTFS in such research.

Exhibit 3

Counts of GTFS-Related Research by Category

37  

22  

21  

12  

8  

7  

6  

4  

4  

Transport  

Modeling  

Data,  so>ware,  or   Accessibility  to   Accessibility  to  

Trip  planner  

VisualizaGon  

Developing  

Other  

planning  and  

standards  

jobs  and  

ameniGes  

development  

countries  

analysis  

development  

neighborhoods  

GTFS = General Transit Feed Specification.

Accessibility to Amenities

Select scholars have used GTFS data to more accurately measure public transit accessibility to amenities including healthcare facilities, grocery stores, schools, and retail. Given the reliance of many low-income households on public transit, these studies help inform policymakers on the disparities in amenity access between driving and transit modes. A 2015 study in Melbourne, Australia, was among the first to demonstrate the potential of GTFS to measure amenity access. Rocha et al. (2015) assessed the transit accessibility of emergency dental care by differentiating patients by socioeconomic status and proximity to high-frequency bus stops. Using GTFS, they found that households living in areas with no high-frequency bus stops were no less likely to seek emergency dental care than those living in areas with high-frequency bus stops. A study of shopping amenity access in Montgomery and Prince George’s Counties in Maryland found, using GTFS data, store locations to be “strongly influenced by access to transportation facilities, especially bus and light rail transit stops” (Ma, Knapp, and Knapp, 2014: 2). Mechaber (2015) found that access to selective enrollment and magnet high schools in Chicago was inequitable, when taking into account GTFS-derived public transit travel times, because it takes longer to get to these schools by transit than by car. School locations were found to be inequitable with respect to minority status and income, because lower-income and minority pupils both lived farther away from the schools and were more reliant on transit to get to school.

210 Data Shop

Do It Yourself: Obtaining Updated Transit Stop and  

Route Shapefiles in Urban and Nonurban Areas

Another set of studies compared spatiotemporal accessibility to grocery stores via public transit versus driving, using GTFS data from the Cincinnati, Ohio metropolitan area. They found that 

food deserts for transit-dependent households change shape depending on the time of day and day of the week, due to the transit schedule, while car-dependent households have a more fixed food desert definition (Farber, Morang, and Widener, 2014). Further, Widener et al. (2015) found that many Cincinnati-area residents have improved accessibility to supermarkets if they access them on their return trip from work rather than departing from home. Widener (2017) extended this 

research to show that transit-dependent households have poorer spatial access to healthy food, as measured by the cumulative access to multiple grocery stores, and that transportation to obtain the healthy food is more costly using transit compared with driving modes. A followup study in the greater Toronto, Canada area finds that grocery store access in late nights and early mornings is lower for transit users than for car users (Widener et al., 2017). This finding is relevant because many lower-wage service employees have unconventional schedules and lower access to cars, 

necessitating transit accessibility to grocery stores at very late or very early hours. These studies barely scratch the surface of what is possible with GTFS in measuring amenity accessibility.

Accessibility to Jobs and Neighborhoods

A limited number of studies have used GTFS to study neighborhood and employment ac-

cessibility—major topics of interest for housing, planning, and neighborhood researchers. A 

cross-sectional analysis by Owen and Levinson (2014) used GTFS to rank 46 of the top 50 most 

populous U.S. metropolitan areas by average transit accessibility to jobs. They provided a realistic weighted average transit-travel time between residences and employment areas, from 7:00 to 9:00 

a.m., including getting to and from public transit and any necessary transfer. New York, New York; San Francisco and Los Angeles, California; Washington, D.C.; and Chicago, Illinois, top the list in job accessibility by transit (Owen and Levinson, 2014). A San Francisco Bay Area simulation analysis found that the region has high job accessibility by walking and by transit, but disparities existed between census blocks in poverty and not in poverty (Blanchard and Waddell, 2017). Ma and Knaap (2014) demonstrated the use of GTFS and Open Street Map, another open data source, 

on neighborhood-level job accessibility for neighborhoods surrounding the proposed Purple transit line in the Maryland suburbs of Washington, D.C. Their model showed outcomes for two planned 

stations. Langley Park, a neighborhood with a high proportion of low-skilled workers, would see an 80-percent increase in low-skilled jobs accessible by transit when the transit line opens. For Bethesda, a regional employment center, 70,000 more employees would be within 1 hour on 

public transit once the line opens.

Several studies addressed transit equity at the neighborhood level by assessing differences in transit supply versus transit demand. Jiao and Dillivan (2013) found “transit deserts”—neighborhoods 

that lack adequate transit service but maintain high proportions of transit-dependent popula-

tions—near the historic downtowns and isolated rural areas of the Charlotte, North Carolina; 

Chicago; Cincinnati; and Portland metropolitan areas. Kahrobaei (2015) found that low-income 

neighborhoods with a high proportion of commuting workers have low bus frequency during the 

morning rush, necessitating a high degree of car ownership. Others proposed and evaluated a Gini coefficient-like measure of transit supply distribution equity, tested using GTFS data (Bertolaccini, 2013; Bertolaccini and Lownes, 2013).

Cityscape 211

Rodnyansky

Research relating housing to transit accessibility using GTFS has been limited. Zhong et al. (2017) harnessed GTFS to derive a model for optimally siting affordable housing to maximize residents’ 

access to public transit and reduce geographic clustering of affordable housing, subject to land acquisition and construction budget constraints. Tested for Tempe, Arizona, the model “could 

provide insightful spatial decision support to affordable-housing providers or tax-credit adminis-trators, facilitating the design of flexible strategies that address multiple social goals” (Zhong et al., 2017: 1). A study of California low-income housing tax credit affordable housing sites showed no statistical difference in transit accessibility between allocated (9-percent) and tax-exempt bond-derived (4-percent) tax-credit projects or between funded and nonfunded 9-percent tax-credit 

projects (Bostic and Rodnyansky, 2016). More research relating housing and transit accessibility is needed for both affordable and market-rate housing, and GTFS makes such research more attainable. 

Conclusion

Transit accessibility is an important topic for housing researchers and policymakers at the national, regional, and local levels. GTFS data improve the geographic coverage, depth, and accuracy of measuring transit accessibility. Researchers and practitioners can use GTFS in a do-it-yourself manner, which was previously unavailable, to include better measures of transit accessibility into their analyses. These better measures are especially needed in housing-related research, which is underrepresented in its use of GTFS.

Acknowledgments

The author thanks session participants at the American Association of Geographers 2015 conference for helpful comments and Anna Sporrong for excellent research assistance. 

Author

Seva Rodnyansky is a Ph.D. candidate in urban planning and development at the University of 

Southern California, Sol Price School of Public Policy.

References

Alonso, William. 1964.  Location and Land Use. Cambridge, MA: Harvard University Press.

Antrim, Aaron. 2015. “Importing GTFS Data Into ArcGIS.” Trillium. https://trilliumtransit.zendesk.

com/hc/en-us/articles/201382609-Importing-GTFS-data-into-ArcGIS.

Bertolaccini, Kelly L. 2013. “Assessing the Equity of Transit Supply Distribution in Metropolitan Areas Using Lorenz Curves and Gini Coefficients.” Master’s thesis, civil engineering, University of 

Connecticut. http://digitalcommons.uconn.edu/gs_theses/483.

Bertolaccini, Kelly L., and Nicholas E. Lownes. 2013. “Effects of Scale and Boundary Selection in Assessing Equity of Transit Supply Distribution,”  Transportation Research Record: Journal of the Transportation Research Board 2350: 58–64.

212 Data Shop

Do It Yourself: Obtaining Updated Transit Stop and  

Route Shapefiles in Urban and Nonurban Areas

Blanchard, Samuel D., and Paul Waddell. 2017. “Assessment of Regional Transit Accessibility in the San Francisco Bay Area of California With UrbanAccess,”  Transportation Research Record: Journal of the Transportation Research Board 2654: 45–54.

Bostic, Raphael W., and Seva Rodnyansky. 2016. Leveraging Federal Tools To Advance Local Goals: The Low Income Housing Tax Credit as a Case Study. Unpublished paper.

Brueckner, Jan K. 1987. “Structure of Urban Equilibria: A Unified Treatment of the Muth-Mills Model.” In  Handbook of Regional and Urban Economics, Vol. 2, edited by Edwin S. Mills. Amsterdam: Elsevier: 821–845.

Farber, Steven, Melinda Z. Morang, and Michael J. Widener. 2014. “Temporal Variability in Transit-Based Accessibility to Supermarkets,”  Applied Geography 53: 149–159.

Giuliano, Genevieve, Sandip Chakrabarti, and Mohja Rhoads. 2016. “Using Regional Archived 

Multimodal Transportation System Data for Policy Analysis: A Case Study of the LA Metro Expo 

Line,”  Journal of Planning Education and Research 36 (2): 195–209.

Giuliano, Genevieve, and Kenneth Small. 1991. “Subcenters in the Los Angeles Region,”  Regional Science and Urban Economics 21 (2): 163–182.

GTFS.org. n.d. “Getting Started.” http://gtfs.org/getting-started/#gtfs-applications.

Haas, Peter, Stephanie Morse, Sofia Becker, Linda Young, and Paul Esling. 2013. “The Influence of Spatial and Household Characteristics on Household Transportation Costs,”  Research in Transportation Business & Management 7: 14–26.

Jiao, Junfeng, and Maxwell Dillivan. 2013. “Transit Deserts: The Gap Between Demand and Sup-

ply,”  Journal of Public Transportation 16 (3): 2.

Kahrobaei, Sina. 2015. “Spatial and Temporal Measures of Mismatch Between Transit Supply 

and Employment for Low Income and Auto Dependent Populations.” Master’s thesis, civil 

engineering, University of Connecticut. http://digitalcommons.uconn.edu/cgi/viewcontent.

cgi?article=1918&context=gs_theses.

Ma, Ting, Eli Knaap, and Gerrit-Jan Knaap. 2014.  Retail Location and Transit: An Econometric Examination of Retail Location in Prince George’s and Montgomery County, Maryland. College Park, MD: University of Maryland, National Center for Smart Growth. http://smartgrowth.umd.edu/assets/

documents/research/retail_location_and_transit_100514_ek_gk.pdf.

Ma, Ting, and Gerrit-Jan Knaap. 2014. “Analyzing Employment Accessibility in a Multimodal Network Using GTFS: A Demonstration of the Purple Line, Maryland.” College Park, MD: University 

of Maryland, National Center for Smart Growth. http://smartgrowth.umd.edu/assets/documents/

research/acsp_tingma_20141029.pdf.

McHugh, Bibiana. 2013. “Pioneering Open Data Standards: The GTFS Story.”  Beyond Transparency.  

http://beyondtransparency.org/chapters/part-2/pioneering-open-data-standards-the-gtfs-story/.

Cityscape 213

Rodnyansky

Mechaber, Elias. 2015. “When Choice Is Not a Choice: A Geographic Accessibility Approach to 

Spatial Inequalities Concerning Magnet and Selective Enrollment Public High Schools in the City of Chicago.” Undergraduate thesis, geography, University of Chicago. http://eliasmechaber.com/

wp-content/uploads/2015/04/Mechaber-Elias_Thesis-Geography_2015.pdf.

Morang, Melinda. n.d. “Add GTFS to a Network Dataset—Overview.” http://transit.melindamo-

rang.com/overview_AddGTFStoND.html.

Owen, Andrew, and David M. Levinson. 2014.  Access Across America: Transit 2014. CTS 14-11. 

Minneapolis: University of Minnesota, Center for Transportation Studies Research. http://hdl.

handle.net/11299/168102.

Redfearn, Christian L. 2007. “The Topography of Metropolitan Employment: Identifying Centers of Employment in a Polycentric Urban Area,”  Journal of Urban Economics 61: 519–541.

Rocha, Carla M., Estie Kruger, Shane McGuire, and Marc Tennant. 2015. “Role of Public Transport in Accessibility to Emergency Dental Care in Melbourne, Australia,”  Australian Journal of Primary Health 21 (2): 227–232.

Sacramento Rapid Transit District (SACRT). n.d. “GTFS.” http://iportal.sacrt.com/GTFS/SRTD/.

U.S. Department of Transportation (DOT). 2016. “STOPS—General Transit Feed Specification 

(GTFS) Data.” Federal Transit Administration. https://www.transit.dot.gov/funding/grant-programs/

capital-investments/stops-%E2%80%93-general-transit-feed-specification-gtfs-data. 

———. n.d. “National Transit Map (Data, Maps and Apps).” Bureau of Transportation Statistics. 

https://www.rita.dot.gov/bts/ntm/map. 

Vance, Steven. 2016. “Converting a Transit Agency’s GTFS to Shapefile and GeoJSON with QGIS.” 

Steven Can Plan. http://www.stevencanplan.com/2016/02/converting-a-transit-agencys-gtfs-to-

shapefile-and-geojson-with-qgis/.

Widener, Michael J. 2017. “Comparing Measures of Accessibility to Urban Supermarkets for Transit and Auto Users,”  The Professional Geographer 69 (3): 362–371.

Widener, Michael J., Steven Farber, Tijs Neutens, and Mark Horner. 2015. “Spatiotemporal Ac-

cessibility to Supermarkets Using Public Transit: An Interaction Potential Approach in Cincinnati, Ohio,”  Journal of Transport Geography 42: 72–83.

Widener, Michael J., Leia Minaker, Steven Farber, Jeff Allen, Brigitte Vitali, Paul C. Coleman, and Brian Cook. 2017. “How Do Changes in the Daily Food and Transportation Environments 

Affect Grocery Store Accessibility?”  Applied Geography 83: 46–62. https://doi.org/10.1016/j.

apgeog.2017.03.018.

Zhong, Qing, Alex Karner, Michael Kuby, and Aaron Golub. 2017. “A Multiobjective Optimiza-

tion Model for Locating Affordable Housing Investments While Maximizing Accessibility to Jobs by Public Transportation,”  Environment and Planning B: Urban Analytics and City Science. DOI: 2399808317719708.

214 Data Shop

SpAM

SpAM (Spatial Analysis and Methods) presents short articles on the use of spatial 

statistical techniques for housing or urban development research. Through this depart-

ment of  Cityscape , the Office of Policy Development and Research introduces readers to the use of emerging spatial data analysis methods or techniques for measuring geographic 

relationships in research data. Researchers increasingly use these new techniques to 

enhance their understanding of urban patterns but often do not have access to short 

demonstration articles for applied guidance. If you have an idea for an article of no more than 3,000 words presenting an applied spatial data analysis method or technique, please 

send a one-paragraph abstract to rwilson@umbc.edu  for review.

Calculating Varying Scales of 

Clustering Among Locations

Ron Wilson

University of Maryland, Baltimore County 

Alexander Din 

Maryland Department of Housing and Community Development

The views expressed in this article are those of the authors and do not represent the official positions or policies of the State of Maryland.

Abstract

The Nearest Neighbor Index (NNI) is a spatial statistic that detects geographical patterns of clustered or dispersed event locations. Unless the locations are randomly distributed, the distances of either clustered or dispersed nearest neighbors form a skewed distribution that biases the average nearest neighbor distance used in calculating the NNI. If the 

clustering or dispersion of locations is moderate to extreme, the NNI can be inaccurate 

if the skew is substantial. Using Housing Choice Voucher program residential locations, 

we demonstrate in this article the method to derive an NNI based on a median and 

two quartiles that more accurately represents the midpoint of a set of nearest neighbor 

distances. We also demonstrate how to use these alternative point estimates to gauge 

multiple scales of clustering from different positions across the nearest neighbor distance distribution. Finally, we discuss how to use the average and standard deviation distances from the calculation of each NNI to more comprehensively gauge the scale of the 

geographic patterns. We also include a Python program that creates a randomized set of 

locations to calculate statistical significance for the median and quartile NNIs. 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 215

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Wilson and Din

Voucher Residence Locations and Clustering

The Housing Choice Voucher (HCV) program enables low-income families to relocate to neighbor-

hoods of their choice (HUD, 2012). A key objective of the HCV program is the deconcentration 

of families to select better neighborhoods in which to live and improve their lives (Winnick, 1995). A common concern about this relocation freedom is that HCV program participants will 

reconcentrate in the destination neighborhoods. Research shows that, after receiving assistance, voucher holders often relocate to neighborhoods similar to those in which they previously lived (Freeman and Botien, 2002; Huartung and Henig, 1997; McClure, 2010; McClure, Schwartz, and 

Taghavi, 2014; Metzger, 2014; Park, 2013; Pendall, 2000; Owens, 2017; Reece et al., 2010; Varady, Walker, and Wang, 2001; Varady et al., 2010; Wang, Larsen, and Ray, 2017; Wang, Varady, and 

Wang, 2008; Wilson, 2013; Zielenbach, 2015). Relocating to similar neighborhoods subverts the objective of the program, and voucher holders are little better off than they previously were. Therefore, housing authorities need to measure the degree of clustering or dispersion of HCV program participant residences to determine if the objective of deconcentration is being met. 

A common measure of location concentration or dispersion is a nearest neighbor analysis using the calculation of the Nearest Neighbor Index (NNI). The NNI is a common global measure of clustering or dispersion, but its accuracy is vulnerable because it is based on an average. Event locations are typically concentrated, which skews the nearest distance distribution positive, because most of the locations are within close proximity to each other. The NNI will be based on a skewed distribution of distances. This problem is especially acute with voucher holders, who often live very close together because the geography of affordable housing stock puts them in the same multifamily 

building or neighborhood. Another limitation is that these neighborhoods vary in size, meaning that the scale of residential clustering will vary across a geography, from close-quarter environments of multifamily housing, to dense townhomes, to single-family homes with land. This change in scale is not something the standard NNI can take into account, because it is a point estimate for one position on the nearest distance distribution. 

As such, a more reliable and multiscale measure must be used to determine the degree of HCV 

residence concentration. An inaccurate measure can report that voucher holders may be more 

or less concentrated than they really are, which would have adverse resource ramifications. For example, if the results show that voucher holders are more clustered than they are, then it may appear as though the program is not working and some other solution should be sought. Conversely, if the results show that voucher holders are more dispersed than they are, then it may appear the program is working and that it requires fewer resources. 

We demonstrate a method to conduct a more robust nearest neighbor analysis by calculating 

median and quartile NNIs to overcome the limitations of the common NNI. The medians and 

quartiles are less susceptible than an average to outliers, and they provide more visibility into concentration patters at multiple spatial scales. 

216 SpAM

Calculating Varying Scales of Clustering Among Locations

Nearest Neighbor Index

The NNI is an ordinal statistic that reports the existence and degree of clustering or dispersion of locations (geometric points). The NNI is a member of a family of cluster measuring statistics, which includes the more common Moran’s  I and Getis-Ord statistics. The NNI, however, is considered a distance analysis statistic because it strictly measures proximity between locations. 

In contrast, the Moran’s  I and Getis-Ord  G statistics can measure proximity between locations not only with distances, but also with buffers around any location or adjacency when data are in areal form (polygon geographies).

Two key assumptions of any analysis involving nearest neighbors are that the sample locations are (1) all included within a finite geography and (2) unimpeded in occurring anywhere in that finite geography (Ebdon, 1985). Empirically, neither assumption ever holds for human or physical events. 

Related event locations often exist outside of a geography but are unavailable for measurement, leading to missing data that impact the calculation of the NNI statistic. More importantly, it is unrealistic that locations can occur anywhere unimpeded across a geography because other spatial processes either facilitate or prevent locations from occurring anywhere. Nevertheless, these two assumptions are necessary for testing if locations exhibit a clustering or dispersing pattern in order to provide a counterfactual geographic distribution of locations for comparison.1 

To calculate the NNI, all nearest neighbor distances are summarized into an average. That summarization implies that a distribution of distances exists that can provide more point estimates about those minimum distances. For example, the standard deviation and percentiles can be used to determine patterns at several geographic scales beyond the average. Those estimates can be used as parameter specifications in identifying clusters, such as in kernel density estimation, Knox Test, SatScan, or the local Moran’s  I and Getis-Ord  G. A key aspect of cluster analysis is determining the distance at which locations are no longer related to each other, that is, the distance at which spatial dependence between the locations ceases. Lacking any theoretical reason or empirical evidence to select that expected distance leaves one to guess what that distance may be. With average, standard deviation, and percentile nearest neighbor distances, the data can be a guide to setting an expected clustering distance.

Calculating the NNI starts with a measurement of the distance between each location to the nearest location. The minimum distance between each location and it nearest neighbor is first summed 

and divided by the total number of locations in the geography to derive an average minimum 

distance. The average minimum distance is—

                                       , 

(1)

1 Caution should be exercised in regards to comparing results from any nearest neighbor analysis to which any comparison of techniques should be done in the same study whether (1) between two different location types or (2) the same location type distributions across time. Comparing analyses between any two geographies confronts the problem that is the root of spatial statistics; that is, having to use randomization as the comparison distribution for significance of a clustering or dispersion pattern. The main problem is that each geography is unique in size and shape and will impact the distribution of event locations that directly affect the statistical results for each geography. Another consideration is that two entirely different distributions can have the same result.

Cityscape 217

Wilson and Din

where  d is the minimum ( Min) distance between location  i and the closest location  j,  N is the total 

–

number of event locations, and  d(NN) is the average minimum distance from measuring all nearest neighbors to each location. The average minimum distance is used in comparison—as the numerator—with a random (expected) distance to determine if the locations exhibit an overall pattern of clustering, dispersion, or random distribution.

The random distance represents an expected minimum nearest neighbor distance from which the 

locations are uninfluenced in that geography by social, economic, physical, or contextual activity—

that is, the random locations from a distribution under complete spatial randomness (Cressie, 2015).2 

The NNI, then, is the observed average minimum distance divided by the expected (random) average minimum distance to produce a relative ratio that is interpreted as an index along a clustering-to-dispersion continuum.

                                                                                                                        (2) The NNI is interpreted in relation to a value of 1. Values around 1 indicate the observed distribution is random. Values less than 1 indicate clustering, with values closer to the floor of 0 indicating extreme clustering. An index of 0.0 means all the locations are in exactly the same place. Values greater than 1 indicate dispersion,3,4 with values closer to the ceiling of 2.149 indicating extreme dispersion.5 An index of 2.149 means that all the locations are exactly equidistant from each other in a systematic pattern. Exhibit 1 shows the relationship between NNI values and their corresponding patterns along the continuum from 0 to 2.149.

Three limitations hinder the NNI statistic from being more robust. The first is that the NNI is a global statistic; it cannot report where any local pattern of clustering or dispersion is in the geography, only that one of the patterns exists within.

The second is that the index is an average, subject to outliers pulling the observed mean away from its true location in the distribution. Outliers are typically present in location data because most social, economic, physical, or other processes produce clustered locations, with dispersed or random patterns seldom occurring. With a nearest neighbor analysis, a high frequency of short distances is produced with a small number of longer distances that skew the distribution positive. 

The third limitation, which is related to the first, is that the average distance is only a single-point estimate for a set of nearest neighbor distances. Single-point estimates give only a partial insight 2 See appendix A for details on the mechanics of calculating the NNI expected distance.

3 Another way to think about the NNI is that the range of values reflects a progression from absolutely clustered (0, with all points in exactly the same location) to evenly dispersed (2.15, with all points maximally spaced from each other). This approach is useful when comparing indices to determine if one pattern is more or less clustered or dispersed than another pattern.

4 The NNI can also be interpreted as a percentage more or less than the random distribution, because it is a ratio. For example, an NNI of 0.55 shows the observed nearest neighbor distances are 45 percent closer (1 – NNI = %) than the distances in the random distribution. An NNI of 1.67 shows the observed nearest neighbor distances are 67 percent farther (NNI – 1 = %) than the random distribution distances.

5 The value 2.149 is the empirical ceiling of the NNI. Theoretically, a value could be higher, but none has been observed in previous research.

218 SpAM

Calculating Varying Scales of Clustering Among Locations

Exhibit 1

Range and Patterns of the Nearest Neighbor Index 

R = ratio (between observed and expected).

into the spatial relationships between locations when varying scales of spatial relationships are likely in the geographic distribution. That is, different scales of clustering will likely exist across the geography due to variation in the structure of the environment. In this instance, neighborhoods where voucher holders live vary in scale, ranging from multifamily buildings, to dense inner-city blocks, to more spread out suburban, small town, or rural properties. 

Nearest Neighbor Index Medians and Quartiles as 

Indicators of Multiple Spatial Pattern Scales

Most software packages that include the standard NNI technique report only the average nearest neighbor distance, which only allows for the assessment of geographic patterns at only one scale, the average distance between locations. Unless the standard deviation distance is also reported, any pattern variation at distances greater or less than the average is undetectable. However, even if the standard deviation distance is reported, highly clustered locations will have a skewed nearest neighbor distance distribution, and the one-standard deviation distance below the average will likely be less than 0 and useless for identifying any scale changes below the average. Calculating median and quartile distances, however, can reveal differing pattern scales at three locations along the distance distribution. The first scale is that of the densest locations, represented by the first 25 

percent of nearest neighbor distances. The second scale is the moderately proximal locations, represented by one-half (50 percent) of the distances at the median. Finally, the third quartile NNI would be the more dispersed distances, represented by 75 percent of all locations, or the top 25 

percent furthest distances. 

Cityscape 219

Wilson and Din

If the scales were all similar, then the NNI would be the nearly the same for each of the point estimates, and the intervals between each index would also be similar. This result will suggest a distance distribution that is normal in shape with a high kurtosis. If the scales are different, then the NNIs will be far apart and the distribution of distances will be spread out, with the intervals between each index varying. This variation in intervals would indicate a distance distribution that is not normal in shape.

Lastly, the median and quartile NNIs can also measure the way in which the differing scales of clusters impact the average. Comparing the observed average with the median and quartiles 

reveals how far off the average is from the actual mean. Not only can the average distance be compared with the median to gauge how much they differ, but the average can also be compared 

with the quartiles to determine how far off the average is from different positions in the distance distribution. If the median and quartile NNIs are all below the average NNI, it indicates that the distribution is heavily skewed positive. If the first quartile and median NNIs are lower than the average, but the third quartile NNI is greater, it would indicate that distribution of distances is not too skewed, and the average distance may be acceptable in calculating the NNI, particularly if the average is closer to median NNI.

Data

The data used in this example are the counts of 2016 HCV program participants by census tract in the Baltimore-Columbia-Towson, MD Core Based Statistical Area (hereafter, Baltimore CBSA). The data were acquired via the U.S. Department Housing and Urban Development (HUD) enterprise 

geographic information systems (GIS) storefront. The total number of participant locations is 23,081. Because HUD does not provide program participant locations, we simulated the locations in a two-stage process to create residential locations based on where people actually live, with several being at the same location to emulate residents in the same apartment complex (exhibit 2).

First, the counts of program participants were divided according to the proportion of residents within each block group contained in each census tract. Second, a set of randomly distributed locations was created within each block group to simulate the locations based on known residential patterns. Some of these randomized locations were situated to be in the same coordinates to 

emulate voucher holders living in apartments or other multiunit residences. This two-stage process allows for a reasonable approximation of where HCV program participants live and reduces the 

risk of placing them in areas where populations do not reside (for example, forested portions, lakes, parks areas, and industrial sites).

220 SpAM

Calculating Varying Scales of Clustering Among Locations

Exhibit 2

Simulated Housing Choice Voucher Program Participant Locations Across the 

Baltimore CBSA

Baltimore CBSA = Baltimore-Columbia-Towson, MD Core Based Statistical Area.

Calculating the Nearest Neighbor Index Based on the 

Average Nearest Neighbor Distances 

We first conducted a basic nearest neighbor analysis in CrimeStat IV to create the statistics for the NNI on the voucher holder locations in the Baltimore CBSA.6 Exhibit 3 shows the results.

The results show the HCV program participant locations are moderately clustered, with an NNI of 0.58. This NNI value suggests that voucher holders are not too concentrated. An average distance of about 159 meters and standard deviation distance of about 835 meters suggest that voucher 

holders are quite spread out at a scale of several neighborhoods—about one-half mile. A standard deviation distance of about five times greater than the mean raises the concern that the NNI may be adversely affected from a skewed distribution of nearest neighbor distances. With geo-processing, we created a variable of nearest neighbor distances and examined the distribution to assess the accuracy of the reported NNI.

6 We used CrimeStat IV because other GIS programs produce only a minimal listing of statistics, whereas CrimeStat IV 

provides much more valuable information that allows for more insight into the distance distribution. The nearest neighbor distance is typically all that is reported in most other GIS or spatial statistics programs, leaving the inability to complexly assess the scale of the clustering. With CrimeStat IV reporting the nearest neighbor standard deviation distance, the scale of clustering can be assessed, because it shows how far the locations are spread around the average distance. The standard deviation distances can be used in clustering routines to visualize the concentration of locations, and the distance can be used to gauge the number of blocks or neighborhoods the voucher holders actually cover. 

Cityscape 221

Wilson and Din

Exhibit 3

Nearest Neighbor Statistics and Diagnostics

Descriptors of Nearest Neighbor Distances

Statistic/Diagnostic

Sample size

23,081

Measurement type

Direct

Mean nearest neighbor distance

158.60 meters

Standard deviation of nearest neighbor distance

834.76 meters

Minimum distance

0.00 meters

Maximum distance

438,731.80 meters

Area of geography (based on user input)

6,819,093,973.49 square meters

Mean random distance

271.77 meters

Mean dispersed distance

584.06 meters

Nearest neighbor index

0.5836

Standard error of random distance

0.94 meters

Test statistic ( Z)

– 121.0332

p-value (one tail)

0.0001

p-value (two tail)

0.0001

Using the Near tool in ArcGIS, we calculated the distances of each location and its nearest neighbor and added them as a variable to the HCV location layer. The distance distribution showed an 

extraordinarily positive skew, with a skewness statistic of 14.65—a value far above the 0.50 

threshold for skewness. Our analysis showed the NNI based on the average was very inaccurate 

and that deriving a median and quartile NNIs would prove worthwhile to improve our assessment of voucher holder concentration.

Calculating Median and Quartile NNIs

To create median and quartile NNIs, we first randomized 23,081 locations with 999 permutation trials within the Baltimore CBSA to create an expected distribution.7 Randomizing the locations repeatedly is known as  bootstrapping and builds an expected distribution against which to compare the observed statistics. The expected distributions represent the distances between locations if no social, physical, economic, or contextual process was influencing their placement. Bootstrapping produces a distribution from which a mean and standard error can be sampled for any point estimate, in this instance quartiles and the median. The mean of any point estimate from the 999 trials becomes the expected value against which to compare the observed statistics, with the standard error used to determine statistical significance of the observed quartile and the median NNIs.

Exhibit 4 shows an example geographic distribution from one permutation trail, in which HCV 

residences would be under complete spatial randomness with each voucher holder having equal 

7 Appendix B contains the Python code that randomizes the data set of 999 trials, including the output of the descriptive statistics for each trial.

222 SpAM

Calculating Varying Scales of Clustering Among Locations

Exhibit 4

Example of Random Permuted Voucher Locations Across the Baltimore CBSA

Baltimore CBSA = Baltimore-Columbia-Towson, MD Core Based Statistical Area.

probability of residing anywhere in the Baltimore CBSA.8 We then repeated the geo-processing of nearest neighbor distances with the Near tool to create the random distribution of the 999 permutation trials and visualize the difference with the observed voucher holder nearest neighbor distances. 

The random distribution has an approximately normal shape, with a skewness value of 0.71.9 

Exhibit 5 shows both the observed and randomized frequency nearest neighbor distance distributions for comparison. 

8 To make a more reasonable random distribution, the randomization process could be restricted to only geographies where voucher holders could live. The use of census tracts or block groups that show residential populations would comprise the area within which the randomization process would be distributed. The use of these geographies would give a more accurate expected average distance and standard error with which to compare the observed average and more precise NNI. 

Having the ability to randomize in a GIS allows for creating more realistic randomization processes. One of the fallacies of the randomization process under complete spatial randomness is that a location has equal probability of being anywhere in a geography, because nothing should prevent it from being anywhere. That may be likely for a physical process, but not for human settlement patterns. Only so many places exist in which voucher holders have an equal likelihood of residing, which would be places where affordable housing options are available, even if they are not likely to be housing options voucher holders can afford. However, that is where complete spatial randomness matters. If nothing prevents the voucher holders from residing in any housing unit, then randomizing across those units is more reasonable, because the voucher holders are not going to live where no housing exists at all. Therefore, the comparison is between the observed and the random places where a person could actually live. To be even more reasonable, only a certain percentage of housing units in a tract keeps the randomization process from locating a random location in just the areas with few housing units.

9 Even a random distribution will produce some locations that are far apart, skewing the otherwise normal distribution positive.

Cityscape 223

Wilson and Din

Exhibit 5

Observed Versus Random Distances Between Nearest Neighbor Locations

To ensure we calculated our 999 permutation trials correctly, we compared the random means and standard errors with results from the CrimeStat and ArcGIS formulas, noting that the differences were slight.10 The comparison showed that our randomized trial results could be used to calculate the statistics for the median and quartile NNIs. 

We then calculated the median and the quartile distances from the observed nearest neighbor distance distribution (exhibit 6), which were 0.00 meters for the first quartile (25th percentile), 16.20 

meters for the median (50th percentile), and 118.82 meters for the third quartile (75th percentile). 

The average nearest neighbor distance showed to be greater than the third quartile of distances and nearly equivalent to the 82nd percentile of distances. Thus, the average distance used in the standard NNI calculation is representing a larger proximity scale—more spread out—between 

10 We compared the results with the NNI and z-score from CrimeStat and ArcGIS to compare the accuracy of our randomization process. The results between our randomization process and that of the software are similar enough, but not exact, because the two programs use a formula to estimate a standard error of a random distribution. CrimeStat and ArcGIS produced an average expected distance of 271.77 meters, and our randomization process produced 275.44 meters, a difference of 3.67 meters. CrimeStat produced a standard error of 0.97 meters, and ours was 1.01 meters, a difference of 0.04 meters. ArcGIS does not produce the standard error to allow for a comparison. We did a difference of means test comparing the CrimeStat formula results and the randomization results, which show the two are statistically different. This comparison shows that the formula is only an estimate that produces a result that is close enough but not as precise. Even though the formulas in CrimeStat and ArcGIS are reasonable approximations, they still are not as truly representative of a random distribution as permutation. Our results appear to back this conjecture, because the formulas produce a result that is close enough for practical purposes. Whether the average and standard errors from the formula or random process are used, the resulting NNIs will not be different enough to affect interpretation and will be identical if rounding to two decimal points to the right. Using the expected average nearest neighbor distance from CrimeStat and ArcGIS, the NNI is 0.584. 

The expected average distance from the randomization process is 0.576, a difference of 0.008. Nothing changes in the interpretation of the NNI, in that they both indicate a moderate level of clustering. Rounded to two decimal places to the right, as the NNI is often reported, both become 0.58, that is, identical.

224 SpAM

Calculating Varying Scales of Clustering Among Locations

Exhibit 6

Quartile, Median, and Mean Distance Distributions of Randomized Trials

NN = nearest neighbor. Std. Err. = standard error. 

locations than ground truth. Although the NNI based on the average shows that voucher holders are clustered, for program evaluation purposes it shows them to be not nearly as concentrated as they really are. 

With the statistics from the randomized trials, we calculated the average median and quartiles to create corresponding expected statistics for three NNIs, representing three different scales of location patterns (exhibit 7). The NNI for the first quartile is 0.00, median is 0.06, and the third quartile is 0.33. Each of these NNIs is less than the NNI based on the average, further showing that the NNI from the average distance is unreliable when a large number of geographic locations are in very close proximity. 

To determine if our median and quartile NNIs were statistically significant, we used the standard errors of the median and quartiles from the 999 trials to calculate corresponding z-scores. The first quartile NNI of 0.00 has score of -111.63, the median NNI of 0.06 has a score of -159.34, and the third quartile NNI of 0.33 has a score of -146.91. All three NNIs are highly statistically significant. 

The intervals between the quartiles and the median are imbalanced, showing a change in clustering scales across the observed distance distribution. The difference between the first quartile and the median is 0.06, but the difference between the median and the third quartile is 0.27, showing that Cityscape 225

Wilson and Din

Exhibit 7

Observed and Random HCVP Location Nearest Neighbor Index Statistics

HCVP Location Distances (in Meters)

Observed

Random

NNI

z-score

Average

158.60

275.44

0.576

– 115.22

Standard deviation

740.03

1.01

1st quartile (25th percentile)

0.00

165.32

0.000

– 111.63

Standard error

—

1.48

Median (50th percentile)

16.20

257.30

0.063

– 159.34

Standard error

—

1.51

3rd quartile (75th percentile)

118.82

365.24

0.325

– 146.91

Standard error

—

1.68

HCVP = Housing Choice Voucher participant. NNI = Nearest Neighbor Index. 

the 50 percent of voucher holders above the median are about 4.5 times more dispersed than the 50 percent of voucher holders below the median. This finding indicates the voucher holders are, indeed, clustered at different scales. The use of median and quartile NNIs, therefore, can provide more information about geographic patterns in the data.

With the nearest neighbor distances created from geo-processing, the scale of the clusters can be assessed with the percentiles around the quartile and median to evaluate the varying scales of clustering among the locations. Exhibit 8 shows the NNI values at +10 percentiles around the 

quartiles and the median are used to reveal the scales of clustering by depicting the spread around each point estimate, including the interquartile range.11

The interquartile range shows the clustering of locations in the upper outer quartile to be about 6.3 times more dispersed than those in the inner quartile. This outcome indicates that locations with distances below the median are clustered in very close proximity. This finding reveals that 50 

percent of voucher holders likely live on the same block, and the other 50 percent likely live in neighboring city blocks. 

The percentile ranges provide more detail about the scale of clustering at each of the quartile and median distances. For the first quartile, the range of 0.00 meters between the 15th and 35th percentiles of nearest neighbor distances reports that the scale of clustering for 35 percent of voucher Exhibit 8

Cluster Scales

Pertentile  and  Quartile  Distance  Ranges  (in  Meters)

Ranges

Min 15th  Pctl 1st  Qrtl 35th  Pctl 40th  Pctl Median 60th  Pctl 65th  Pctl 1st  Qrtl 85th  Pctl

Max

0.00

0.00

0.00

0.00

4.05

16.20

39.26

57.55 118.82 187.34 23,444.30

Percentile

0.00

35.22

129.80

Lower

Upper

Inter-‐quartile

16.20

102.62

Max = maximum. Min = minimum. Pctl = percentile. Qrtl = quartile.

11 Any percentile range can be used to examine the ranges around the quartile and median NNI.

226 SpAM

Calculating Varying Scales of Clustering Among Locations

holders is that of the same building or complex; the minimum is also 0.00 meters and indicates that two or more locations are in the exact same place. At the median, the range of nearest neighbor distances between the 40th and 60th percentile is 35.22 meters, indicating that 20 percent of voucher holders live on blocks with dense housing. Finally, between the 65th and 85th percentiles of nearest neighbor distances, 20 percent of voucher holders live within 129.80 meters of each other and are likely on contiguous blocks in a neighborhood. However, after the 85th percentile of nearest neighbor distances (187.3 meters), the voucher holders spread out substantially and are isolated from others given they are up to 23,444.30 meters away from the nearest voucher holder. 

Summary and Extensions of the Interquartile NNIs

With highly clustered locations, we showed that the average nearest neighbor distance proves to be an inaccurate base to calculate the NNI. When using the statistic to help in assessing a program’s performance, such as the HCV program, an alternative measurement must be used. The median 

and quartile NNIs not only provide more accurate results of the geographic pattern of clustering, but the statistics also provide more information about changes in clustering at differing geographic scales. In this example, the NNI based on the average distance inaccurately showed that voucher holders are only moderately concentrated, when they are actually far more clustered. Using the median and quartile NNIs, however, revealed that at least 25 percent of voucher holders were 

highly concentrated at the same location, with 26 to 75 percent likely living together in small neighborhoods. Given the nature of housing availability for HCV program participants, future 

analyses of the program will likely need to be analyzed with the median and quartile NNIs. 

We offer a final thought about the geographies used to randomize when calculating any NNI. 

Typically, randomization is either implemented in the permutation process or is estimated with a formula that uses the area of the geography in which the locations occur. The use of the entire geography is based on the assumption that the locations can be equally likely to occur anywhere within that boundary. This assumption is unrealistic, because locations do not have equal probability of being anywhere, which is due to physical and human influences on a geography that restrict the occurrence of locations. We suggest running a second analysis limiting the geography to only areas in which the locations can actually occur. With voucher holders, these geographies would only be the areas in which rental housing is available. The boundary of the limited geography can be used in the permutation process with GIS as an alternative, by identifying all the areas that the analyzed locations can actually have the opportunity to occur.

Appendix A: Formula for Calculating an Expected Nearest 

Neighbor Distance

A random (expected) distance is generated by one of two randomization methods. The first 

method—which is rarely implemented in software—is to randomly distribute the same number of 

observed locations within an area that is either the size of their minimum bounding rectangle or Cityscape 227

Wilson and Din

within the study geography.12,13 This randomization process is known as permutation, by which the observed data are used to generate a counterfactual (expected) distribution for what would occur in that unique geography. The second method uses a formula to approximate the Monte Carlo 

process, which is—

                              , 

(3)

where  e–( NN) is the expected average distance,  A is the total from the study geography in which the locations occur, and  N is the total number of locations. The ratio produces a density, of which the square root is taken to produce a linearized value.14 The constant 0.5 is multiplied to the linearized density ratio to rescale it and prevent the expected average distance from being larger than the study geography.

Appendix B: Python Code for Creating the Randomized 

Locations

# -*- coding: utf-8 -*-

"""

#This file creates random points in a given geography then takes statistics of distances of 

the  ## nearest points.

## Author: Alex Din

"""

import arcpy

import csv

import numpy  as np

import os

from   random import randint

import time

##

start_time = time.time()

## this needs to be 1000 in order to get 999 iterations because of the range loop starts 

at 1, not 0

iterations = 1000

## the number of random points to be created each iteration

pointsNum  = 23081

## the prjArea is the project area area geography which is the Baltimore CBSA in this 

example

prjArea    = r"C:\Path\to\the\feature\class\for\the\project\geography"

## the csvName is the name of your output file, it MUST have '.csv' appended to the string

csvName    = "Baltimore_CBSA.csv"

## workspace is the geodatabase where functions will be performed

workspace  = r"C:\Path\to\the\working\geodatabase.gdb"

## dirspace is where your csv will be written, the directory must already exist prior to 

running the script

dirspace   = r"C:\Path\to\the\working\directory"

## csvPath is the combination of the csvName and dirspace for outputing the final CSV

csvpath    = os.path.join(dirspace,csvName)

12 Randomizing observed data is known as permutation. 

13 A minimum bounding rectangle is the outermost boundary of the furthest locations in each Cartesian plane orthogonal direction.

14 The square root is taken to transform the two-dimensional density ratio into a one-dimensional distance so that it is geometrically comparable with the observed distance.

228 SpAM

Calculating Varying Scales of Clustering Among Locations

## the name of the random points

ptsName    = "samplepoints"

## the random number is used to export a random set of points for visualization purposes

randomNumb = randint(1, iterations)

## the work environments are set

arcpy.env.overwriteOutput = True

arcpy.env.workspace = workspace

os.chdir(dirspace)

## the print statement informs the user which

print "Iteration %s will be exported as a random copy for map purposes" %(randomNumb)

##

for number in range(1,iterations):

    print "Processing number %s" %(number)

    small_time = time.time()

    # create a set of random points within the project area

    try:

        arcpy.CreateRandomPoints_management(workspace, ptsName, prjArea, "", pointsNum, 

"", "POINT", "")

    except Exception as e:

        print e

    # compute nearest neighbor distance

    try:

        nearValueList = []

        arcpy.Near_analysis(ptsName, ptsName, "", "NO_LOCATION", "") with arcpy.da.SearchCursor(ptsName,["NEAR_DIST"]) as cursor:

            for row in cursor:

                nearValueList.append(row[0])

        nearValueList.sort()

        # print the values in the IPython console to inspect while processing

        p25  = round(np.percentile(nearValueList, 25), 2)

        p50  = round(np.percentile(nearValueList, 50), 2)

        p75  = round(np.percentile(nearValueList, 75), 2)

        mean = round(np.mean(nearValueList), 2)

        std  = round(np.std(nearValueList), 2)

        var  = round(np.var(nearValueList,ddof=1), 2)

        maxx = round(np.max(nearValueList), 2)

        minn = round(np.min(nearValueList), 2)

        del cursor, row

    except Exception as e:

        print e

    # get the time it took to run just this one iteration

    small_time_end = time.time()

    small_elapse   = round((small_time_end - small_time),2)

    print "This iteration took %s seconds" %(small_elapse)

    # log the information to a CSV file

    # if the CSV does not yet exist, the CSV will be created with headers and append the 

first iteration of data

    # else, if the CSV does exist, the information will be appended to a new row

    try:

        headRows = ["Number", "Seconds","25P", "Median", "75P", "Mean","STD", "Variance", 

"Maximum","Minimum"]

        dataRows = [number,small_elapse,p25,p50,p75,mean,std,var,maxx,minn]

        if not os.path.exists(csvpath):

            with open(csvName, 'wb') as f:

                wtr = csv.writer(f, delimiter= ',')

                wtr.writerow(headRows)

                wtr.writerow(dataRows)

        else:

            with open(csvName, 'ab') as f:

                wtr = csv.writer(f, delimiter= ',')

                wtr.writerow(dataRows)

Cityscape 229

Wilson and Din

    except Exception as e:

        print e

    del f, wtr

    # if the iteration matches the random number, export the sample data set for visual-

ization purposes

    try:

        if number == randomNumb:

            out_name = "%s_random_%s" %(ptsName,number)

            arcpy.FeatureClassToFeatureClass_conversion(ptsName,workspace,out_name)

    except Exception as e:

        print e

    print("--------------------------------------------------------------------")

##

end_time = time.time()

print("Total time elapsed was %g seconds" % round((end_time - start_time),2)) 

Acknowledgments

The authors thank Danielle Wilson from the University of Maryland, Coro Chasco of the Universi-dad Autónoma de Madrid, Julia Koschinski of the University of Chicago, and Jay Lee of Kent State University for reviewing this method and for providing valuable comments toward improving the manuscript.

Authors

Ron Wilson is an adjunct faculty member of the Geographic Information Systems Program at the 

University of Maryland, Baltimore County.

Alex Din is a housing research and GIS analyst with the Maryland Department of Housing and 

Community Development.

References

Cressie, Noel. 2015.  Statistics for Spatial Data, revised ed. New York: Wiley.

Ebdon, David. 1985.  Statistics in Geography Second Edition: A Practical Approach. Malden, MA: Blackwell Publishing. 

Freeman, Lance, and Hilary Botein. 2002. “Subsidized Housing and Neighborhood Impacts: A 

Theoretical Discussion and Review of the Evidence,”  Journal of Planning Literature 16 (3): 359–378.

Hartung, John M., and Jeffrey R. Henig. 1997. “Housing Vouchers and Certificates as a Vehicle for Deconcentrating the Poor: Evidence From the Washington, D.C. Metropolitan Area,”  Urban Affairs Review 32 (3): 403–419.

McClure, Kirk. 2010. “The Prospects for Guiding Housing Choice Voucher Households to High 

Opportunity Neighborhoods,”  Cityscape 12 (3): 101–122.

230 SpAM

Calculating Varying Scales of Clustering Among Locations

McClure, Kirk, Alex F. Schwartz, and Lydia B. Taghavi. 2014. “Housing Choice Voucher Location Patterns a Decade Later,”  Housing Policy Debate 25 (2): 215–233.

Metzger, Molly W. 2014. “The Reconcentration of Poverty: Patterns of Housing Voucher Use, 2000 

to 2008,”  Housing Policy Debate 24 (3): 544–567. 

Owens, Anne. 2017. “How Do People-Based Housing Policies Affect People (and Place)?”  Housing Policy Debate 27 (2): 266–281.

Park, Miseon. 2013. “Housing Vouchers as a Means of Poverty Deconcentration and Race Desegre-

gation: Patterns and Factors of Voucher Recipients’ Spatial Concentration in Cleveland,”  Journal of Housing and the Built Environment 28 (3): 451–468. 

Pendall, Rolf. 2000. “Why Voucher and Certificate Users Live in Distressed Neighborhoods,”  Housing Policy Debate 11 (4): 881–910.

Reece, Jason, Samir Gambhir, Craig Ratchford, Matthew Martin, Jillian Olinger, John A. Powell, and Andrew Grant-Thomas. 2010.  The Geography of Opportunity: Mapping To Promote Equitable Community Development and Fair Housing in King County, WA. Columbus: The Ohio State University, Kirwan Institute for the Study of Race and Ethnicity. http://kirwaninstitute.osu.edu/docs/KingCounty.pdf. 

U.S. Department of Housing and Urban Development (HUD). 2012. “Public and Indian Housing 

Tenant-Based Rental Assistance: 2012 Summary Statement and Initiatives.” https://www.hud.gov/

sites/documents/TENANT_BR_ASSIS_2012.PDF.

Varady, David P., Carole C. Walker, and Xinhao Wang. 2001. “Voucher Recipient Achievement of 

Improved Housing Conditions in the US: Do Moving Distance and Relocation Services Matter?” 

Urban Studies 38 (8): 1273–1304.

Varady, David P., Xinhao Wang, Yimei Wang, and Patrick Duhaney. 2010. “The Geographic 

Concentration of Housing Vouchers, Blacks, and Poverty Over Time: A Study of Cincinnati, Ohio, USA,”  Urban Research & Practice 3 (1): 39–62. 

Wang, Ruoniu, Kristin Larsen, and Anne Ray. 2017. “Rethinking Locational Outcomes for Housing Choice Vouchers: A Case Study in Duval County, Florida,”  Housing Policy Debate 25 (4): 715–738.

Wang, Xinhao, David Varady, and Yimei Wang. 2008. “Measuring the Deconcentration of Housing 

Choice Voucher Program Recipients in Eight U.S. Metropolitan Areas Using Hot Spot Analysis,” 

Cityscape 10 (1): 65–90.

Wilson, Ron. 2013. “Using Near-Repeat Analysis To Measure the Concentration of Housing Choice Voucher Program Participants,”  Cityscape 15 (3): 307–318. 

Winnick, Louis. 1995. “The Triumph of Housing Allowance Programs: How a Fundamental Policy 

Conflict Was Resolved,”  Cityscape 1 (3): 95–121.

Zielenbach, Sean. 2015. “Moving Beyond the Rhetoric: Section 8 Housing Choice Voucher Program and the Lower-Income Urban Neighborhoods,”  Journal of Affordable Housing & Community Development Law 16 (1): 9–39.

Cityscape 231

232 Departments

Evaluation Tradecraft

Evaluation Tradecraft presents short articles about the art of evaluation in housing and 

urban research. Through this department of  Cityscape , the Office of Policy Development and Research presents developments in the art of evaluation that might not be described 

in detail in published evaluations. Researchers often describe what they did and what 

their results were, but they might not give readers a step-by-step guide for implementing their methods. This department pulls back the curtain and shows readers exactly how 

program evaluation is done. If you have an idea for an article of about 3,000 words on 

a particular evaluation method or an interesting development in the art of evaluation, 

please send a one-paragraph abstract to marina.l.myhre@hud.gov .

Household Survey on Tribal Lands: 

Frame Building Through Rural 

Address-Based Sampling and 

Traditional Enumeration

Carol Hafford

Steven Pedlow  

NORC at the University of Chicago 

Nancy Pindus 

Urban Institute

Abstract

The congressionally mandated Assessment of American Indian, Alaska Native, and 

Native Hawaiian Housing Needs was a housing needs assessment designed to produce 

national-level estimates of housing needs in U.S. tribal areas (HUD, 2017a). Special 

care was taken so that the process would not only be technically effective (to ensure 

reliable results) but also be fully acceptable to the tribes involved. The foundation for the in-person household survey was the development of the sample frame of eligible 

American Indian and Alaska Native (AIAN) households from which to derive national 

estimates of housing needs. Three methods were used to construct the list of AIAN 

households and addresses for the sample frame and to select the households to interview: 

(1) United States Postal Service address lists, (2) tribal maps and lists, and (3) in-

person enumeration. Use of these methods yielded sufficient coverage to provide reliable 

estimates of housing needs. 

Cityscape: A Journal of Policy Development and Research • Volume 20 Number 1 • 2018

Cityscape 233

U.S. Department of Housing and Urban Development • Office of Policy Development and Research

Hafford, Pedlow, and Pindus

Introduction

The  Assessment of American Indian, Alaska Native, and Native Hawaiian Housing Needs was designed to study housing needs in U.S. tribal areas. The previous similar assessment was conducted in 1996, prior to the passage of the Native American Housing Assistance and Self-Determination Act of 19961 

that fundamentally changed the way federal funding for housing is delivered to Native people. 

This study was a 6-year effort, from 2011 to 2017, that included consultations with tribal leaders, analysis of U.S. Census Bureau data and U.S. Department of Housing and Urban Development 

(HUD) administrative data, three surveys, and site visits. The most important new data collection effort in this project was a major in-person household survey in a sample of American Indian and Alaska Native (AIAN) tribal areas. This effort was one of the largest and most complex surveys ever undertaken in Indian country. A nationally representative survey of tribally designated housing entities was also conducted. 

The study team took special care to make the process technically effective (that is, to ensure reliable results) and also fully acceptable to the tribes involved. All tribal areas, as defined by the Census Bureau, with an AIAN-alone2 population of at least 150 were eligible for selection. The minimum of 150 was to make sure that a sufficient number of interviews (approximately 30 

eligible AIAN households) could be collected from each selected tribe to develop the national estimate, with a proportionally greater number collected from the largest tribes, including the Navajo and Cherokee Nations. The tribal area probabilities were derived from the AIAN-alone population in the 2010 census. From a sample of 595 eligible tribes, the research team selected two embedded representative samples: (1) a representative sample of 120 tribal areas that included the tribally designated housing entities sample; and (2) a representative subsample of 60 tribal areas that included the 40 tribal areas for the household survey and also 20 tribal areas as a reserve, if needed, to replace any of the original 40 sampled tribal areas. The team selected with certainty 7 tribal areas with populations greater than 15,861 AIAN-alone persons for the household survey: (1) Navajo Nation reservation and off-reservation trust land, (2) Cherokee Oklahoma Tribal Statistical Area (OTSA), (3) Lumbee State Designated Tribal Statistical Area, (4) Muscogee (Creek) OTSA, (5) Choctaw OTSA, (6) Chickasaw OTSA, and (7) the Oglala Sioux Pine Ridge Reservation.

After participating in the tribal consultations that HUD held in 2012, the research team worked closely with tribal leaders in each of the 40 tribal areas selected for the household survey to obtain permission to conduct the study. This process ensured tribal stewardship and oversight of any research conducted on sovereign lands while safeguarding community wellbeing and protecting 

the community from harmful research (Sahota, 2007). Each tribe participating in the study had different protocols and requirements. In nine cases, it was necessary to obtain approval from the tribe’s Institutional Review Board and from the tribal government. Ultimately, 37 originally selected 1 Pub. L. 104–330, 110 Stat. 4016. October 26, 1996.

2  AIAN alone is defined as people reporting that they belong to a single race (American Indian or Alaska native), not in combination with any other race, on the census form.

234 Evaluation Tradecraft

Household Survey on Tribal Lands: Frame Building Through  

Rural Address-Based Sampling and Traditional Enumeration

tribes and 1 replacement tribe in the sample agreed to participate.3 They included reservation-based tribes; large and small pueblo, woodland, and coastal tribes; tribal jurisdiction service areas in Oklahoma; and native villages in Alaska.

Using tribal-specific sample frames, AIAN households were selected for interviews.4 In-person interviews were conducted with 1,340 households. This article focuses on the development of the household sampling frame in those selected tribal areas. We describe the procedures used and the experiences of this survey in order to guide other researchers working in tribal or in rural areas. 

Information for each sampled tribe, including their total populations based on the 2010 census, their AIAN-alone populations, the selection probability, the frame method used, estimated coverage, and the unweighted and weighted response rates are in the technical appendixes to the final report (HUD, 2017b).5 

The Methodological Challenge

The foundation for the in-person household survey was the development of the sample frame of 

eligible AIAN households from which to derive national estimates of housing needs. However, no such national frame across tribal areas exists. The research team’s experience with data collection in Indian country suggested at the outset that constructing an address-based list of households for each tribal area would be necessary to form the universe from which to draw the sample. 

The advantages of using address-based sampling for probability-based surveys include increased coverage of households and access to cost-effective and timely sampling frames (AAPOR, 2016). 

However, coverage is not evenly distributed for some rural geographies or subpopulations, such as tribes, which can result in undercoverage errors, through either omissions or erroneous exclusions (O’Muircheartaigh, English, and Eckman, 2007). Use of general delivery postal addresses remains common in rural areas, including Indian country; one of the drawbacks of in-person household 

surveys is that post office box addresses and other rural route addresses are not locatable. Although many tribes have mapped locatable housing units for emergency response services and have 

assigned households with city-style addresses (that is, a house number and street name), others are still in the process of doing so. Consequently, once the sample of tribal areas was drawn, the research team had to identify alternative sources and methods for developing high-quality, tribal-specific sample frames to select and locate households for a hard-to-reach population within the time and cost parameters of the study.

Creating the Household Sampling Frame 

Building the address-based, household-level sampling frame required multiple methods. Many 

households on tribal lands rely on post office box addresses and other rural route addresses, which 3 A sample of 40 tribal areas originally was selected, but HUD deemed that 2 tribal areas were ineligible because they were not Indian Housing Block Grant program grantees.

4 Tribal member households were those households in which the owners or renters, their spouses, or custodial children age 17 or younger self-identified as Native American or Alaska Native, alone or multiracial.

5 See Exhibit E.1. Summary of 40 Selected Tribal Areas for Household Sample (HUD, 2017b). 

Cityscape 235

Hafford, Pedlow, and Pindus

do not provide a physical location for data collection. Taking this factor into consideration, the team used three methods to construct the list of AIAN households and addresses for the sample frame and to select the households to interview: (1) United States Postal Service (USPS) address lists, (2) tribal lists, and (3) in-person enumeration. Use of each method involved outreach with each tribe to discuss appropriate procedures and protocols.6 The research team conducted a pilot test with 15 tribes to assess the feasibility of each approach, taking into consideration the density of the population, the ratio of the AIAN population to the total population (to forecast the level of screening needed), potential access to a tribally held list, the extent of the tribal terrain to cover, and the time and costs involved. 

Of the 38 tribal areas, USPS address lists were usable for 9 tribal areas (24 percent). These address lists were used only if the estimated coverage was at least 80 percent. Coverage was determined by dividing the number of city-style addresses within the tribal area by the number of occupied housing units according to the 2010 census. 

For another 16 tribal areas (42 percent)—which included land-based reservations, joint-use areas claimed by multiple tribes, OTSAs, and Alaska Native village statistical areas—the study team obtained a single source of housing units or eligible persons, provided by the tribe, to develop the tribal-specific sample frame. These sources varied and included maps of housing units and roads, spreadsheets and printouts of housing units, 911 or fire lists, and tribal membership lists. Gaining access to these lists involved extensive outreach with tribal leaders and tribal departments. Using a structured protocol, the research team consulted with relevant tribal entities to understand the content and quality of the lists or maps (that is, data fields, frequency of updating, percentage of population included, omissions, and so on) and assessed their utility and limitations as sampling frames. Once the quality of the lists or maps was determined, the team needed tribal consensus to share the lists. The research team then negotiated access through multiple entities, including tribal leaders, councils, housing authorities, and others. Use of these proprietary tribal resources was bound by strict confidentiality requirements. 

For 12 tribal areas (32 percent), using USPS or tribal-specific lists was not possible. Therefore, housing units in selected blocks within tribal areas were systematically enumerated (listed) in person by field personnel to form the sampling frame. The team conducted listing on large and small land-based reservations, on pueblos, across OTSAs, and in Alaska Native villages. The 38th tribal area, a land-based reservation divided across two states, provided a county-based housing unit list for the portion in one state, but the portion in the other state needed to be listed. 

Experienced field interviewers were trained in the enumeration methods. Maps, to help them find the selected areas, were prepared using census geography and MapMarker® software. Driving (sometimes great distances) throughout identified communities across the reservations or tribal areas, interviewers identified and plotted every dwelling in a defined area on a list. Using the list of all housing units identified for a tribal area, statisticians selected a sample of households for the study. 

For two reservations, the research team used a list-and-go methodology to expedite the sample frame development-and-selection process, as permission to conduct the data collection was obtained in the final months of the field period. Instead of enumerating (listing) all units in the field, survey 6 HUD (2017b), exhibit E.1 shows the frame used for each of the 38 tribal areas. 

236 Evaluation Tradecraft

Household Survey on Tribal Lands: Frame Building Through  

Rural Address-Based Sampling and Traditional Enumeration

methodologists developed listing sheets based on geocoded maps that identified preselected housing units for each block at a specified sampling rate (this rate depended on the tribal area). Field interviewers received training before starting the list-and-go process and were monitored and coached throughout the listing activity. They were responsible for enumerating the area, screening eligible households, gaining cooperation, and conducting the in-person interviews with AIAN respondents.

The largest tribal area in the study, the Navajo Nation reservation and off-reservation trust land, contains more than 17 percent of the entire AIAN-alone population in tribal areas. With no maps or lists of rural-based addresses available for sampling purposes, listing was necessary. For the Navajo Nation, the research team first selected 15 chapters, or local jurisdictions, before selecting two segments within each chapter. Chapters were selected across five regions, with probabilities proportional to the AIAN population in those areas. The exception was that, within the western region, the Cameron chapter was selected with certainty by request of the Navajo Nation to include households affected by the Bennett Freeze area.7 No chapters were large enough to be selected with certainty. Within chapters, segments of blocks were selected using census block housing 

unit counts so that the sample would be representative of all Navajo chapters. Partial or entire block groups (all census blocks with the same first digit within a census tract) were selected that contained approximately 100 to 150 housing units, according to 2010 census data. The research team listed 11 of the 15 chapters in this manner and conducted household interviews. Due to time limitations, interviews with 4 of the 15 chapters were not started before project closedown.8 

Sampling and Selection Probabilities 

When an address-based USPS list or a tribal list of households or persons was available as the sampling frame, the selection probability was very simple—the number of selected housing units divided by the total number of eligible housing units on the list. Determining the number of housing units to select was based on a nonvacancy rate of 85 percent, a screening completion rate of 90 

percent, an interview completion rate of 70 percent, and the tribal area-specific person eligibility rate. Only AIAN residents were eligible, and it was assumed that the tribal area-specific eligibility rate was simply the 2010 census AIAN-only population divided by the total tribal area population.9 

When no list of addresses was available from the USPS address lists or the tribal list (or if these lists did not provide coverage of at least 80 percent), the research team listed specific areas, or segments, and drew samples from the listed units. Different numbers of segments were selected for different tribal areas, often depending on the tribal area-specific eligibility rate. When this eligibility rate was lower, more housing units needed to be selected, and spreading them over more segments 7 In 1966, the U.S. Bureau of Indian Affairs placed a development ban on 1.5 million acres of Navajo land, encompassing all or parts of nine western Navajo chapters, in order to promote negotiations over a land dispute between the Navajo and Hopi Nations. Under the ban, Navajo families were not allowed to repair their homes; housing construction and infrastructure projects, including installation of water and power lines, were halted. Thousands of Navajo families lived in substandard housing with no running water and electricity. The Bennett Freeze was lifted in 2009. Community development efforts are under way to build new housing units and develop needed infrastructure.

8 To compensate for this chapter subsampling, the team adjusted weights within the same region. Further information about the 15 chapters selected and their selection probabilities can be found in HUD (2017b), exhibit E.2. 

9 As noted in HUD (2017b), exhibit E.1. 

Cityscape 237

Hafford, Pedlow, and Pindus

was preferable. Segments were listed in 11 tribal areas, with the most (22) being listed within Navajo Nation, the largest tribal area (2 in each of 11 chapters). Researchers aimed for segments of approximately 100 housing units to minimize effort and costs while still providing enough housing units for selection. Selection probabilities for listed tribal areas were determined using decennial census counts of housing units to establish the selection probability of the segment, and this was multiplied by the selection rate within the segment.

In the list-and-go procedure, the researchers determined in advance which households would 

be selected block by block based on the expected number of housing units in each block. If the number of housing units differed from expectations, the team prepared the materials so that any additional housing units had a selection probability equal to all others in the segment.

Household Survey Implementation 

The integrated approach to onsite data collection was contingent on completing multiple interrelated activities, including tribal approval to conduct the study; developing the sample frame; drawing the tribal-specific sample; and the recruitment, hiring, and training of tribal field interviewers. 

Working with 38 tribal nations, the timeline for completing these activities differed, so the research team implemented a staggered data collection schedule. Ideally, the research team forecasted 12 

weeks of data collection for the household survey, starting from the time permission was granted and field staff were hired and trained. The research team conducted the tribally designated housing entities telephone survey and site visits to 22 tribal areas during this field period. 

With the encouragement of the tribal nations, and to ensure that the household survey was 

conducted in a culturally competent manner, the research team recruited, hired, and trained tribal members to conduct the interviews. Field interviewers’ training focused on contacting sampled households, key respondent rules, gaining cooperation, obtaining informed consent, conducting the interview and the enumerator observation, securely mailing completed paper-and-pencil 

instrument questionnaires, and quality control procedures.

Each household selected for the survey received an advance package about 10 days before the start date of the field data collection period. The field interviewer mailed or hand delivered (to those with post office boxes) advance letters to all sampled households. After allowing sufficient time for receipt of the materials, the field interviewer telephoned or visited the household to schedule an appoint-ment to conduct the in-person interview and assess the exterior conditions of each housing unit. 

Field interviewers recorded each attempt to contact a household. Interviewers varied contact attempts to the selected households during times household members were most likely to be home.

In many tribal areas, the low density of the AIAN population relative to the non-Native population required extensive screening by field staff to identity eligible AIAN households. For example, extensive screening was necessary for seven of the eight Oklahoma tribes in the sample, as they lack reservations and the American Indian population is dispersed throughout tribal jurisdiction service areas.

After addressing initial questions or concerns, the field interviewers conveyed the need to conduct interviews in respondents’ homes to ensure privacy and to conduct the enumerator observation of exterior housing conditions. Depending on tribal protocols, AIAN heads of household or alternate 238 Evaluation Tradecraft

Household Survey on Tribal Lands: Frame Building Through  

Rural Address-Based Sampling and Traditional Enumeration

respondents provided informed verbal or written consent to participate in interviews. As with the information letters and brochures, the content of the consent form was tailored to different tribal research conditions or Institutional Review Board requirements. 

Interviews focused on how residents viewed their own housing conditions. A key element of 

administering the household survey was to obtain a complete roster of persons living in the 

household at the time of the interview. This roster was used to assess the degree of overcrowding and the prevalence of doubled-up households being used as a way to afford housing or to avoid homelessness. Topics addressed included housing unit characteristics and conditions (based on the American Housing Survey’s worst case housing needs), satisfaction with housing, culturally responsive housing, needed services and amenities in the community, preferences for homeownership, 

living on tribal lands, attitudes on tribally assisted housing, and household income and housing costs. After completing the in-home interview, the field interviewer conducted the observation of exterior housing conditions, noting the type of structure, access from the road, and the conditions of the roof, walls, windows, and foundation. 

A completed interview consisted of responses to all modules and the enumerator observation of housing conditions. At the close of both parts of the interview, respondents received incentives valued at $20. The team informed each tribe about the post-data collection quality-control procedures to ensure that tribes understood the importance of verification calls to respondents. 

Fieldwork on the household survey began in July 2013 and was completed successfully in Febru-

ary 2016. Beyond the anticipated challenges of gaining cooperation with a hard-to-reach population, the scale and remoteness of the geography and extreme weather (that is, blizzards, flash floods, and drought-related fires) challenged field interviewers. The overall weighted response rate was 60 percent.10 Response rates varied across tribes, with 19 tribal areas having weighted response rates greater than 70 percent. 

Conclusion

No one source of address-based household lists is available in Indian country. Ensuring sufficient coverage of the population and developing the sampling frame were key methodological challenges for the Assessment of American Indian, Alaska Native, and Native Hawaiian Housing Needs study (HUD, 2017a). The foundation for this nationally representative survey was the development of the sample frame of eligible AIAN households from which to derive national estimates of housing needs. Building the sampling frame necessitated reliance on multiple methods, including use of USPS address lists; maps and lists obtained from tribes; and in-person enumeration on tribal lands, including use of a list-and-go approach as a last resort. Approval from each tribe was necessary to implement these methods on sovereign tribal lands. Ensuring a high degree of coverage and developing a robust sample frame meant representing, mapping, and enumerating tribal communities 

so that their housing needs could be assessed and that the study team could have confidence in the national estimates derived. 

10 A weighted response rate is reported for nationally representative surveys, because that is an average of the response rates according to the location of the population.

Cityscape 239

Hafford, Pedlow, and Pindus

Acknowledgments

This article is dedicated to the memory of G. Thomas Kingsley of the Urban Institute, who served as the Principal Investigator (PI) for U.S. Department of Housing and Urban Development’s 1996 

Assessment of American Indian Housing Needs and Programs and was the PI in the early years for this study. He was a champion of safe, affordable, and decent housing for all.

The authors thank the tribal leaders and housing directors who agreed to participate in this study and facilitated the approval process and data collection efforts. They also thank the tribal governments and research Institutional Review Boards that approved this study, providing oversight and assurances that encouraged participation and forthright responses. The authors are especially grate-ful to the household survey respondents residing in the 38 sampled tribal areas who were generous with their time and willing to share their stories. 

Authors 

Carol Hafford is a principal research scientist at NORC at the University of Chicago. 

Steven Pedlow is a senior statistician at NORC at the University of Chicago. 

Nancy Pindus is a senior fellow at the Urban Institute. 

References 

American Association for Public Opinion Research (AAPOR). 2016. “Address-Based Sampling.” 

http://www.aapor.org/Education-Resources/Reports/Address-based-Sampling.aspx#1.2. 

O’Muircheartaigh, Colm, Edward English, and Stephanie Eckman. 2007. “Predicting the Relative 

Quality of Alternative Sampling Frames.” In  2007 Proceedings of the American Statistical Association, Survey Research Methods Section [CD ROM]. Alexandria, VA: American Statistical Association. 

Sahota, Puneet. 2007.  Research Regulation in American Indian/Alaska Native Communities: Policy and Practice Considerations. Washington, DC: National Congress of American Indians Policy Research Center.

U.S. Department of Housing and Urban Development (HUD). 2017a. “Assessment of American 

Indian, Alaska Native, and Native Hawaiian Housing Needs.” huduser.gov/portal/native_american_ 

assessment/home.html.

———. 2017b.  Technical Appendixes: Housing Needs of American Indians and Alaska Natives in Tribal Areas: A Report From the Assessment of American Indian, Alaska Native, and Native Hawaiian Housing Needs. Washington, DC: U.S. Department of Housing and Urban Development.

240 Evaluation Tradecraft

What Have We Learned From Paired 

Testing in Housing Markets?

Sun Jung Oh

John Yinger

Correction

The volume 17, number 3 issue of  Cityscape contained errors in exhibit A-1. The corrected version of the relevant rows in the table follows.

Exhibit A-1

Results of Housing Discrimination Audit Studies (1 of 2)

Other  

Factors  

Location, 

Data/ 

Considered 

Period, 

Authors

Scale

Main Findings

Methodologya

With Race/ and Market 

Ethnicity and  Examined

Gender

Ahmed and 

E-mail 

500 units 

Sweden

Arabic males have 21 to 26 

Hammarstedt  correspondence  (500 x 3 

2007

percentage points lower 

(2008)

tests

= 1,500 

probability of being invited 

e-mails)

to further contacts or to 

Probit model

Rental  

showings than do Swedish 

tests

males. Swedish males are 

almost 13 percentage points 

less likely to be invited 

to further contacts or to 

showings than Swedish 

females.

Hanson and 

E-mail 

4,728 

Socioeconomic 10 U.S. 

African-American renters 

Hawley (2011) correspondence  tests 

status using 

cities  

have 4.5 percentage points 

tests

(4,728 x  the prose 

2009

lower probability of receiving 

2 = 9,456  quality of 

an e-mail from landlords 

e-mails)

e-mails

than do White renters. This 

Probit model

Rental  

difference ranges from over 8 

tests

percentage points in Boston 

and Los Angeles to less than 

1 percentage point in Atlanta 

and Dallas. African-American 

renters of higher social class 

experience small and not 

statistically distinguishable 

racial discrimination.

Cityscape 241

Exhibit A-1

Results of Housing Discrimination Audit Studies (2 of 2)

Other  

Factors  

Location, 

Data/ 

Considered 

Period, 

Authors

Scale

Main Findings

Methodologya

With Race/ and Market 

Ethnicity and  Examined

Gender

Hanson, 

E-mail 

3,153 

10 U.S. 

African-American renters are 

Hawley, and 

correspondence  tests 

cities

treated less favorably than 

Taylor (2011)

tests

(3,153 x 

2009

White renters by landlords. 

2 = 6,306 

Landlords reply faster, reply 

e-mails)

Rental  

with an e-mail that is longer to 

tests

inquiries made, make formal 

greetings, and use polite 

language when replying to 

e-mail inquiries from a White 

homeseeker.

Roychoudhury  In-person tests

569 tests

Detroit, 

For each additional apartment 

and Goodman 

Michigan 

available to an agent, the 

(1992)

1980–1990 probability of discrimination 

Ordered probit 

Rental  

against an African-American 

model

tests

auditor increases by 0.5 for 

the number of units withheld 

and by 0.58 for the number of 

units inspected.

a Methodology indicates any statistical analysis other than computing gross and net measures or the differences-in-means tests.

242 Correction

Contents

Symposium 

Selected Outcomes of Housing Assistance ..........................................1

Guest Editors: Meena Bavan and David Hardiman

Guest Editors’ Introduction 

Findings From PD&R’s Multidisciplinary Research Team ..........................................................3

Length of Stay in Assisted Housing  by Kirk McClure ............................................................11

Housing Cost Burden in the Housing Choice Voucher Program: The Impact of HUD  

Program Rules  by Casey Dawkins and Jae Sik Jeon ............................................................39

Opting In, Opting Out: A Decade Later  by Anne Ray, Jeongseob Kim, Diep Nguyen,  

Jongwon Choi, Kelly McElwain, and Keely Jones Stater .....................................................63

The Quality of Assisted Housing in the United States  by Sandra Newman and Scott Holupka ...89

An International Perspective: Reflection on the Symposium  by Kwan Ok Lee ........................113

Refereed Papers ...........................................................................................115

Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach 

by Halil Toros and Daniel Flaming ...................................................................................117

Scale in Housing Policy: A Case Study of the Potential of Small Area Fair Market Rents 

by Matthew Palm............................................................................................................147

Can a Car-Centric City Become Transit Oriented? Evidence From Los Angeles  

by Jenny Schuetz, Genevieve Giuliano, and Eun Jin Shin .................................................167

Departments ..................................................................................................191

Data Shop

First-Time Homebuyers: Toward a New Measure  by Arthur Acolin, Paul Calem,  

Julapa Jagtiani, and Susan Wachter ...............................................................................193

Do It Yourself: Obtaining Updated Transit Stop and Route Shapefiles in Urban and  

Nonurban Areas  by Seva Rodnyansky .............................................................................205

SpAM

Calculating Varying Scales of Clustering Among Locations  by Ron Wilson and  

Alexander Din ................................................................................................................215

Evaluation Tradecraft

Household Survey on Tribal Lands: Frame Building Through Rural Address-Based Sampling  

and Traditional Enumeration  by Carol Hafford, Steven Pedlow, and Nancy Pindus ............233

Correction

What Have We Learned From Paired Testing in Housing Markets?  by Sun Jung Oh and  

John Yinger ....................................................................................................................241

Cityscape

U.S. Department of Housing

and Urban Development

VOlume 20 number 1 

Office of Policy Development and Research

• 2018

Document Outline

Cover

Title Page

Contents

Symposium 

Guest Editors’ Introduction 

Length of Stay in Assisted Housing

Housing Cost Burden in the Housing Choice Voucher Program: The Impact of HUD Program Rules

Opting In, Opting Out: A Decade Later

The Quality of Assisted Housing in the United States

An International Perspective:  Reflection on the Symposium

Refereed Papers 

Prioritizing Homeless Assistance Using Predictive Algorithms:  An Evidence-Based Approach

Scale in Housing Policy:  A Case Study of the Potential of Small Area Fair Market Rents

Can a Car-Centric City  Become Transit Oriented? Evidence From Los Angeles

Departments 

Data Shop - First-Time Homebuyers: Toward a New Measure

Data Shop- Do It Yourself:Obtaining Updated Transit Stop and Route Shapefiles in Urban and Nonurban

SpAM- Calculating Varying Scales of Clustering Among Locations

Evaluation Tradecraft- Household Survey on Tribal Lands: Frame Building Through Rural...

Correction- What Have We Learned From Paired Testing in Housing Markets?