Assessment Tools for

Allocating Homelessness Assistance:

State of the Evidence

PD&R Expert Convenings

Summary Report

February 2015

Visit PD&R’s website

www.huduser.org

to find this report and others sponsored by HUD’s Office of Policy Development and Research (PD&R). Other services of HUD USER, PD&R’s research information service, include listservs, special interest reports, bimonthly publications (best practices, significant studies from other sources), access to public use databases, and a hotline (800-245-2691) for help accessing the information you need.

Summary Report

Assessment Tools for

Allocating Homelessness Assistance:

State of the Evidence

Prepared for:

U.S. Department of Housing

and Urban Development

Washington, D.C.

February 2015

The contents of this report are the views of the authors/participants and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. government.

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence On November 5, 2014, HUD’s Office of Policy Development and Research, in partnership with the National Alliance to End Homelessness (the Alliance), convened a panel of experts to discuss assessment tools that communities are using to allocate homeless assistance and to consider the evidence base for the questions used in the tools.

Assessment Tools Overview: Context and Evidence

The McKinney-Vento Homeless Assistance Act, as amended by the Homeless Emergency Assistance and Rapid Transition to Housing Act of 2009, authorizes HUD’s Continuum of Care (CoC) program, which provides funding to nonprofit, government, and other entities that serve persons experiencing homelessness within a defined geographic area. The CoC program interim rule requires CoCs to use a coordinated assessment system, including a standardized assessment tool.1 The coordinated assessment is intended to prioritize resources for those with the greatest need, match people with the services that are most likely to help them exit homelessness, reduce the time it takes for clients to access services, and ensure that limited resources are allocated efficiently. Additional benefits of coordinated assessment may include improved standardization of referrals (because coordinated assessment relies more on standardized inputs than on caseworker judgment) as well as an orientation toward the hardest to serve populations rather than households anticipated to be more responsive to assistance.

Through coordinated assessment, CoCs create a standard assessment process and community agreement on referrals so that individuals will be directed to the most appropriate available services regardless of how they enter the system. This standard assessment process can both match clients with the most appropriate services and prioritize need. Although CoCs are eager to adopt tools and scoring systems from other communities, these tools may not always be appropriate for implementation in a different community. Choosing a tool without considering whether it is suitable for, or applicable to, the CoC’s goals may lead the CoC to overassess, allow insufficient time for self-resolution of a homelessness episode, assess for needs without the ability to provide appropriate services, or use the tools beyond their intended purpose. In addition, different tools may define need differently; for example, in terms of risk of harm or mortality, risk of continued homelessness, or cost of service utilization.

A strong tool would be reliable (produce consistent results) and have validity (measure what it claims to measure) so that stakeholders could have faith in the instrument, and it would have predictive value. For the purposes of this discussion, the Vulnerability Index —

Service Prioritization Decision Assistance Tool (VI-SPDAT), the Alliance Comprehensive Assessment tool, and the Hennepin County Eligibility Criteria and Rating Tool and Vulnerability Index were provided to the experts as examples of tools currently being used by communities. The tools are evidence informed, but because they are still relatively new, the evidence base is limited.

Participant Discussion: Examining the Evidence for Assessment

Tools and Questions

Participants offered general observations about the assessment tools, the evidence underlying the tools, and how they are being, or should be, applied in the field.

n The tools should be based on a clear understanding of their purpose, whether that is matching clients to services, prioritizing the allocation of available assistance, or both.

n Assessments should be tailored to the services that providers can offer. For example, asking about diabetes in an assessment is unnecessary if no services will be offered to directly treat the condition.

PD&R Expert Convenings

1

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence n Several participants stated that the tools should be used in ways that take client preferences and choices into account. One expert noted that an evaluation of preventive models in New York City indicated that the models predicted needs better than caseworkers. Another expert cautioned that the assessment or assessment instrument should not override the decisions made by the provider and the client about the best course of action for that client. Some tools allow providers to override the assessment results, which may also help reduce providers’ resistance to the tools.

n The experts general y (but not unanimously) agreed that different population groups should be assessed and treated differently.

Assessments also have to account for developmental stages in youth; for example, tools that are effective for 17- and 18-year-olds may not be appropriate for 12- to 16-year olds.

n Participants noted that local context, including the capacity of providers, available resources, and political climate, is critical y important to the utility of a given assessment tool. Other participants noted, however, that the very use of an assessment tool, regardless of how well it achieves its intended purpose, may catalyze systems change and galvanize communities toward a desired response.

Participants were then asked to respond to specific questions from HUD and Alliance staff about existing assessment tools.

Summary of Questions and Responses

n What questions should be included on an assessment tool designed to screen for prevention or diversion?

q A study of families applying for homelessness prevention services in New York City found that families with past interactions with the homeless service system, a history of frequent moves, a head of household who is young or with a history of disrup-tive experiences as a child, and the presence of a current conflict (either within the family or with a landlord or leaseholder) were more likely to enter a shelter. For single adults, being younger, receiving public assistance, having higher rental arrearag-es, experiencing a verbal eviction threat, reintegrating into the community from some sort of institution, having a shelter application from within the past three months, and having a city record of a previous shelter stay were predictive of subsequent shelter stays. These factors are more predictive for entry to homelessness for individuals than families. For families, the risk factors are additive, meaning the more risk factors an individual family has, the higher their risk for shelter entry.

q A large-scale, two-year panel study in Australia finds that having a current housing issue is more predictive of entry into homelessness than homeless history is.

n If the goal of the tool is to match people with available resources, what core set of questions should the tool include?

q Tools should include questions that reflect the services available in the community. Tools should also be focused on the assistance that is being offered; for example, for housing interventions, a tool needs to include questions that relate to housing.

q For the purposes of matching people with available resources, the assessment should include questions related to eligibility for an intervention so that people are referred only to interventions for which they are eligible. In right-to-shelter communities, eligibility is essential y an assessment tool for entry into shelter; households enter a shelter or are diverted based simply on their eligibility.

PD&R Expert Convenings

2

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence n Does evidence point to good questions for prioritizing permanent supportive housing among chronical y homeless individuals and families?

q Existing assessment tools may identify a population of need — for example, the Vulnerability Index identifies those most likely to die — but it is not known whether permanent supportive housing services make the most difference among those who are likely to die.

q Evidence exists that permanent supportive housing ends chronic homelessness, but research is less clear about prioritization beyond identifying those meeting the definition of chronic homelessness.

n Are questions about family structure useful in an assessment?

q It depends on what question you are trying to answer — for preventing homelessness, questions about family structure are useful. Since we do not have definitive evidence yet on which interventions work better for which families, knowing a family’s structure would not give providers the information needed to offer the family the “right” intervention. However, communities should ask questions relevant to a referral. For example, a provider would need to know a family’s size to determine how large a housing unit it would need, but this question does not enable a provider to predicting the family’s success in a given housing unit or intervention.

n Is there anything about family structure that is predictive about what intervention is needed?

q An evaluation of HUD’s Rapid Re-Housing for Families Demonstration Program did not find correlations between rates of return to shelter and family structure, but research in New York City found that larger families with more children exit homeless assistance more slowly than do smaller families with fewer children.

q Research shows that a mother’s mental health or distress at the point of initial assessment is not predictive of what intervention might be most appropriate to resolve homelessness, and that mothers’ mental distress dissipates over time, regardless of the type or intensity of the intervention they receive.

n What about families with a mother under age 24?

q Younger mothers and mothers with young children are associated with higher rates of entry to homelessness, but the intervention that is best for ending homelessness for this group is not known.

n After determining whether a client is below the federal poverty level, does asking about income make sense?

q Income does not seem to predict anything, and most people fall within a relatively small (and low) range.

n How much time should be allowed for self-resolution of homelessness?

q The panelists did not reach consensus on the amount of time to give families or individuals to self-resolve homelessness. One expert suggested that a month is a threshold of sorts, after which stepping up interventions for individuals might make sense.

For families, however, allowing a week for self-resolution might be more appropriate.

PD&R Expert Convenings

3

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence n Are there strong predictors of remaining homeless?

q Very few factors predict who will remain homeless.

q Being male predicts a lower likelihood of exiting homelessness and a greater likelihood of repeat homelessness.

q How long one has been homeless shows mixed results.

q Serious mental illness is not a predictor.

q Family contact and support does help clients exit homelessness and return to stable housing (broadly defined).

q In a U.S. Department of Veterans Affairs universal screener, respondents who indicated that they did not have family members with whom they could stay were more likely to become homeless.

q Proximal events — those which make up an individual’s current situation — are the strongest predictors.

n Does the act of assessing individuals or sharing the results of the assessment affect individuals’ behavior or the length of their shelter stay?

q There is a risk that people will try to “game the system.” Assessments with closed-ended questions, in particular, pose the risk that individuals will be able to predict what type of answer will yield a referral to a particular intervention.

q On the other hand, if clients know up front the services for which they are eligible— or what they will or will not receive —

they will not wait for services they are not going to get.

n Are any aspects of residential history predictors of becoming or remaining homeless?

q Most evictions do not result in homelessness. Various ways of being thrown out of housing can precipitate episodes of homelessness.

q Being thrown out of a doubled-up situation is at least as important as a formal eviction.

Future Research Directions

The panelists were asked to discuss what additional research is needed as communities adopt and implement standardized assessment tools as part of their coordinated assessment systems. The consensus was that the evidence base for existing assessment tools was not strong and that ongoing evaluation and research are necessary. To validate individual tools, researchers would need to identify the outcome or outcomes the tools are intended to predict and test their predictive and false alarm rates at various levels of risk. One expert said that rather than testing the validity of the assessment tools, the relevant task should be to find out how well systems work by comparing systems in randomized controlled trials. These trials, another acknowledged, would be challenging to conduct because of the difficulty involved in teasing out all of the variables. One expert expressed concern that evaluating whether coordinated assessment itself is work-ing would be difficult. Because of the regulatory requirement that all CoCs use coordinated assessment, no comparison group exists.

Several experts pointed out that as CoCs use the tools, they are generating vast amounts of data — in some cases integrated with their Homeless Management Information System — that researchers can use. Challenges to such research include the limited capacity of local PD&R Expert Convenings

4

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence CoCs to use assessment data to improve their processes and the limited ability of national-level researchers and HUD to capture all of the coordinated assessment data for research purposes.

Some experts recommended mixed-methods research to better understand the systems involved. Among the specific topics identified for further inquiry were providers’ and clients’ experiences with the tools, such as ease of use, time spent using the tools, stress on providers and clients, completeness of data in relation to length of time needed to complete an assessment, progressive engagement and assessment, the concept of “light touch” interventions with additional engagement when necessary, and the effectiveness of rapid rehousing. (Many indicated that rapid rehousing had a strong evidence base, but others voiced dissent.) One expert suggested that data from HUD’s Homelessness Prevention and Rapid Re-Housing Program might provide insight on progressive engagement or assistance because the program has a built-in recertification at three months.

Final y, a number of participants suggested looking at other fields that have dealt with similar questions and issues, specifical y mention-ing the employment and criminal justice systems. One expert noted that practitioners in the employment field have had little success in predicting which clients benefit most from specific interventions.

Key Themes

The invited experts general y agreed that existing assessment tools do not have a strong evidence base and are limited in their ability to select the best interventions for families and individuals or to predict which families would be the most successful in different interventions. There appears to be some research practitioners can use to target prevention assistance. In addition, considerable research supports targeting permanent supportive housing to those who experience chronic homelessness. Beyond that, however, little evidence exists to support targeting interventions to specific families or individuals.

The experts contributed their thoughts on how to evaluate and improve the assessment tools but also discussed alternatives to assessment-based allocation of assistance. Several participants pointed to lotteries as the fairest way to allocate assistance. Proponents argued that lotteries are simple, easy to understand, and can be implemented without much training, although they conceded that lotteries do not result in the most efficient allocation of resources.

A sizable group supported a second alternative, progressive engagement, through which everyone would receive the “lightest touch” to help as many clients as possible. More intensive and expensive interventions would be provided only if the lightest touch proved insufficient to resolve the household’s homelessness. The experts acknowledged that at this point, the evidence base for progressive engagement is not strong either.

Because all CoCs are currently required to administer coordinated assessment and standardized assessment tools, the experts urged HUD to communicate clearly to CoCs exactly what outcomes HUD seeks, what HUD knows about the utility of different assessment tools and the evidence informing them, guidance on how to prioritize service allocation, and what opportunities exist for communities to experiment.

The group acknowledged that the broader context for these discussions is one in which resources are extremely limited. An additional opportunity of utilizing assessment tools is to determine more precisely which resources are needed in greater quantity for whom, in order to make the case for increasing resources in proportion to documented need.

1 U.S. Department of Housing and Urban Development. 2012. “Homeless Emergency Assistance and Rapid Transition to Housing: Continuum of Care Program; Interim Final Rule,” Federal Register 77:147 (July 31), 45423, 45425.

PD&R Expert Convenings

5

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence Participants

n Matt Aronson

n Rachelle Levitt

U.S. Department of Housing and Urban Development

U.S. Department of Housing and Urban Development

n Samantha Batko

n Marge Martin

National Alliance to End Homelessness

U.S. Department of Housing and Urban Development

n Amanda Benson

n Norweeta Milburn

U.S. Department of Health and Human Services

University of California, Los Angeles

n Jennifer Biess

n Abby Miller

Urban Institute

U.S. Department of Housing and Urban Development

n Richard Cho

n Ann Elizabeth Montgomery

United States Interagency Council on Homelessness

National Center on Homelessness Among Veterans

n Dennis Culhane

n Cynthia Nagendra

University of Pennsylvania

National Alliance to End Homelessness

n Mary Cunningham

n Jim O’Connel

Urban Institute

Boston Health Care for the Homeless Program

n Iain De Jong

n Brendan O’Flaherty

OrgCode Consulting

Columbia University

n Claire Desjardins

n Ann Oliva

U.S. Department of Housing and Urban Development

U.S. Department of Housing and Urban Development

n Liza Doran

n Katherine O’Regan

National Alliance to End Homelessness

U.S. Department of Housing and Urban Development

n Anne Fletcher

n Deborah Padgett

U.S. Department of Housing and Urban Development

New York University

n Andrew Greer

n Teresa Pittman

Westat

U.S. Department of Veterans Affairs

n Jennifer Ho

n James Riccio

U.S. Department of Housing and Urban Development

MDRC

n Sarah Hunter

n Debra Rog

U.S. Department of Housing and Urban Development

Westat

n Guy Johnson

n Nan Roman

RMIT University

National Alliance to End Homelessness

n Stefan Kertesz

n Emily Rosenoff

University of Alabama at Birmingham

U.S. Department of Health and Human Services

n Jill Khadduri

n Lynn Ross

Abt Associates

U.S. Department of Housing and Urban Development

PD&R Expert Convenings

6

Assessment Tools for Allocating Homelessness Assistance: State of the Evidence n Elizabeth Rudd

n Carol Star

U.S. Department of Housing and Urban Development

U.S. Department of Housing and Urban Development

n Judith Samuels

n Norm Suchar

New York University, Langone Medical Center

U.S. Department of Housing and Urban Development

n Carrie Schuettpelz

n Marcy Thompson

U.S. Department of Housing and Urban Development

U.S. Department of Housing and Urban Development

n Todd Shenk

n Paul Toro

U.S. Department of Housing and Urban Development

Wayne State University

n Marybeth Shinn

n Jesse Vazzano

Vanderbilt University

U.S. Department of Veterans Affairs

n Jeff Simms

n Marge Wherley

U.S. Department of Health and Human Services

Abt Associates

n William Snow

n Matt White

U.S. Department of Housing and Urban Development

Abt Associates

n Brooke Spel man

Abt Associates

PD&R Expert Convenings

7