Evaluation of the HUD-DOJ

Pay for Success Permanent

Supportive Housing

Demonstration

Year 2 Report

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

DISCLAIMER

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

Evaluation of the HUD-DOJ Pay for

Success Permanent Supportive Housing

Demonstration

Year 2 Report

Akiva Liberman

Mary Cunningham

Sarah Gil espie

Samantha Batko

Matt Eldridge

Nicole DuBois

Alexandra Ricks

Kelly Walsh

Urban Institute

October 2019

Foreword

In 2016, the U.S. Department of Housing and Urban Development (HUD) and the U.S. Department of Justice (DOJ) launched a joint program initiative: the Pay for Success Permanent Supportive Housing Demonstration (Demonstration). Using authority under the Second Chance Act of 2007, together with funding provided by Congressional appropriations, this HUD-DOJ partnership is aimed at both reducing chronic homelessness and addressing recidivism and barriers to re-entry from the criminal justice system—issues which often affect the same at-risk and vulnerable population. This study, the

“Evaluation of the HUD-DOJ Pay for Success Permanent Supportive Housing Demonstration, Year 2

Report,” documents the progress made during 2018 by the local partnerships that are implementing the Demonstration in their communities.

There is an ongoing crisis of homelessness in America. In 2019, HUD estimated about 568,000

people experience homelessness on any given night, including over 96,000 individuals who are chronically homelessness. Homelessness often has exceedingly high long-run public costs, including for emergency room visits and homeless shelters. Too often, the costs are also shifted to the criminal justice system, with jails and prisons acting as health care providers of last resort, a role they may be ill-equipped or not designed to perform. There is a significant body of evidence showing that permanent Supportive Housing, combining long-term affordable housing with supportive services, is a more cost-effective strategy, with lower costs and better outcomes over the long-term. The Demonstration implements this approach through the Housing First model—providing a stable housing platform to then facilitate access to a range of needed services. The Demonstration further combines this approach with an innovative structure for financing and implementation—Pay for Success.

Pay for Success is a new model for accessing private capital to meet public needs. The model shifts the start-up costs for innovative approaches, from the traditional government-only spending structure to a variety of other potential vehicles, from social impact bonds to direct upfront philanthropic investment. The Demonstration—and the findings in this study—have potential y significant implications beyond the issues of homelessness and prevention, but also for the Pay for Success model itself.

i i

F OR E W O R D

This report documents the experiences and obstacles faced by each of the seven local grantees initially selected for the Demonstration. After one of the initial grantees opted out of the Demonstration, the six remaining sites were: Anchorage, Alaska; Austin/Travis County, Texas; Lane County, Oregon; Los Angeles County, California; Montgomery and Prince George’s Counties, Maryland; and the state of Rhode Island. Each of these participants had its own unique approach to forming their own local collaborations and partnerships, in designing variations in program interventions, and in structuring their own innovative financing structure—al of which is documented and described in-depth in this report. Key findings from Year 2 include—

(1) Five of the six remaining Demonstration sites moved on to the next PFS phase.

(2) Three sites advancing from feasibility analysis to transaction structuring.

(3) Two sites advancing to implementation (one of the two sites started making success payments).

(4) One site remaining in transaction structuring.

The joint HUD-DOJ Pay for Success Permanent Supportive Housing Demonstration remains in progress. This study, as a “formative evaluation,” not only describes the process followed by the local participants to benefit future efforts, but also provides critical feedback and qualitative findings to inform the ongoing design, implementation, and success of the Demonstration itself. As stated, the study also has important findings and implications for the overal Pay for Success model, with practical lessons from implementation and actual experience in the field. This study builds on two previous reports, including a baseline report and a research brief on data sharing uses and chal enges.

While it is too early to determine the success of the Demonstration, it is encouraging that in Year 2

most sites progressed to the next phase. Future reports wil explore the progress at Demonstration sites, outcome-based success metrics, and individual project evaluations to determine if the Pay for Success framework is successful framework for providing Permanent Supportive Housing for this at-risk population.

F OR E W O R D

i i i

Seth D. Appleton

Assistant Secretary for Policy Development and Research

U.S. Department of Housing and Urban Development

i v

F OR E W O R D

Contents

Executive Summary

ix

Background

ix

Cross-Site Findings on Feasibility Analysis and Transaction Structuring xii

Feasibility Reports

xii

Building PFS Partnerships

xii

Defining the Target Population

xiii

Establishing the Financial Feasibility of PFS

xiii

Establishing the Feasibility of Implementing PSH

xv

Establishing Success Metrics and Evaluation

xvi

Implementing PSH

xvi

Los Angeles County

xvi

Lane County

xix

The Benefits and Costs of Using Pay for Success to Launch PSH

xix

I. Introduction

1

Overview

1

Background

3

Permanent Supportive Housing

3

Expanding PSH Using PFS

4

PFS Partners

7

The HUD-DOJ Pay for Success Supportive Housing Demonstration 8

Demonstration Theory of Change

9

Demonstration Requirements and Phases

10

Demonstration Sites

11

Year 2 Evaluation Questions and Methods

13

Data Collection Activities

14

II. Progress at Sites in the Transaction Structuring Phase 17

Anchorage, Alaska

17

Austin/Travis County, Texas

20

Lane County, Oregon

22

Montgomery and Prince George’s Counties, Maryland

25

Rhode Island

27

III. Cross-Site Findings on Feasibility Analysis and Transaction Structuring 29

Feasibility Reports

30

A. Building PFS Partnerships

33

C O NT E N T S

v

Collaboration and Partnership

36

Data Sharing and Data Use

37

Time Burden

39

B. Defining the Target Population

40

Completing Data Analyses

44

C. Establishing the Financial Feasibility of PFS

45

Feasibility Reports’ Estimates of Costs and Possible Savings 45

Net Cost Savings and Other Motivations for PFS

49

Potential Savings from Successful PSH Implementation

50

Identifying and Securing End Payors

51

The Costs of Implementing PSH

52

Finding Investors and Managing Risk

53

Political and Legal Complexities

54

D. Establishing the Feasibility of Implementing PSH

55

Participant Identification and Enrol ment Pathways

56

Pilot Periods

57

Service Provider and Housing Capacity

58

Housing First

59

Permanent Supportive Housing and Criminal Justice

59

E. Establishing Success Metrics and Evaluation

61

Success Payments

62

Evaluation

63

IV. Implementing PSH

68

Los Angeles County Implementation

68

Participant Enrollment and Housing

69

Service Provision

70

Implementation Challenges and Criminal Justice Involvement 71

Success Payments

71

Lane County Implementation

73

Housing

73

Services

74

V. Conclusion

75

Appendix - Stakeholder Benefits Survey Descriptives

78

A. Respondent Information

78

B. To What Extent Would You Agree with the Fol owing Statements about Your PFS project?

80

v i

C O NT E N T S

C. PFS Organization. To What Extent Would You Agree with the Fol owing Statements about Your Organization’s Role in the Project?

83

D. PFS Partners. Would You Agree with the Fol owing Statements about the PFS Partnership?

85

E. Community Support. Would You Agree with the Fol owing Statements about the Community Support for the Project?

89

F. Col aboration. Would You Agree with the Fol owing Statements about Permanent Supportive Housing Providers in Your Community?

90

G. Col aboration. Over the Past Year, How Much Have the Following Systems Collaborated with Each Other in Working with the Target Population?

92

H. Col aboration. Over the Past Year, Has Col aboration Increased or Decreased?

94

I. Collaboration. Over the Past Year, Did Any of the Following Pose a Serious Problem for PFS Partners Col aborating?

96

J. Col aboration. Over the Past Year, How Often Has Your Organization Engaged in the Fol owing Activities with Other Organizations Serving as Partners in the PFS Project?

98

K. Data sharing. Would You Agree with the Following Statements about Data Sharing over the Past Year?

101

L. Performance Data. Would You Agree with the Fol owing Statements about the Use of Evidence to Manage Supportive Housing Providers over the Past Year?

103

M. Data Sharing. Over the Past Year, How Often Has Your Organization Shared or Received Information about the Clients in the Target Population?

105

N. Data Sharing. Over the Past Year, Has Your Organization Shared Criminal Justice Information with Other Organizations Serving the Target Population?

105

O. Data Sharing. Over the Past Year, Has Your Organization Shared Client Information with Other Organizations Serving the Target Population?

107

P. Service Chal enges. Over the Past Year, Has Your PFS Project Experienced Any of the Fol owing as Serious Problems for Providing Services to the Target Population?

109

Q. Service Challenges. Over the Past Year, Have Any of the Following Posed a Serious Problem for Providing Services to the Target Population for Your PFS Project?

112

R. Community Infrastructure. Would You Agree with the Following Statements about Your Community?

116

References

119

C O NT E N T S

v i i

Executive Summary

In 2016, the U.S. Department of Housing and Urban Development (HUD) and the U.S. Department of Justice (DOJ) launched the Pay for Success (PFS) Permanent Supportive Housing (PSH) Demonstration (the demonstration), funding seven sites across the country. The goal of the demonstration is to strengthen communities’ abilities to reduce recidivism and homelessness among the reentry population by increasing PSH through PFS, a financing mechanism that aims to unlock private and philanthropic investment in social programs, build the evidence base on effective programs, and ensure public funds pay for outcomes. Of the seven sites, two launched a PSH program through PFS, four are working on structuring the PFS transaction, and one withdrew from the demonstration—

deciding to fund PSH without PFS. This report describes the challenges sites encountered and milestones they reached working toward launching PSH with PFS.

Background

PSH combines a housing subsidy with intensive supportive services; it helps a subset of the reentry population that experiences long-term homelessness and cycles in and out of crisis services, including jails, homeless shelters, hospital emergency departments, and psychiatric and detoxification centers.

This cycling is detrimental to an individual’s health and well-being and comes at a high cost to the public. Despite strong evidence supporting its expansion, PSH is not available at the scale many communities need.

The demonstration aims to scale PSH by launching partnerships among government agencies, social service providers, and private and philanthropic investors through PFS. Under PFS, investors pay the up-front costs of PSH and are repaid by the government if the program succeeds, as measured by agreed-upon outcomes such as reductions in jail stays or the use of shelters. PFS involves multiple actors within a jurisdiction, including governments, funders, financial intermediaries, knowledge intermediaries, service providers, and independent evaluators who assess a program’s success. HUD-DOJ funded three phases of PFS projects: feasibility analysis, transaction structuring, and implementation. The implementation phase includes evaluation and success payments.

HUD issued a Notice of Funding Availability (NOFA) in October 2015. Together with DOJ, HUD

evaluated applications from across the country, ultimately awarding $8.7 million to seven demonstration sites: Anchorage and Matanuska-Susitna Borough, Alaska; Austin/Travis County, Texas; E XE C U TI VE SU M M AR Y

i x

Lane County, Oregon; Los Angeles County, California; Montgomery and Prince George’s Counties, Maryland; Pima County, Arizona; and Rhode Island (exhibit 1). In 2017, HUD contracted with the Urban Institute to conduct a formative national evaluation of the demonstration. The goals of Urban’s evaluation are to examine how PFS is implemented in each site, capture lessons learned across sites, and examine the benefits and chal enges of using PFS to fund PSH for a reentry population.

In October 2016, the sites entered into grant agreements, at which point they were at different points in their projects. Los Angeles County began in the transaction structuring phase, completing a PFS contract and beginning implementation in 2017. Austin/Travis County also began in the transaction structuring phase, whereas the other five sites began in the feasibility analysis phase. After completing its feasibility analysis, Pima County withdrew from the demonstration, opting to implement PSH without the PFS financing mechanism. Urban describes the seven projects’

accomplishments and chal enges during the first period of the demonstration (through December 2017) in a baseline report focusing on the feasibility analysis phase of PFS.

This report summarizes grantee progress in Year 2 of the demonstration (January through December 2018), focusing on the transaction structuring phase as well as the requirements, challenges, and accomplishments grantees faced as they moved closer to implementing their projects. By the end of 2018, al remaining sites that had started the demonstration in the feasibility analysis phase had moved into transaction structuring. The Alaska site began as a joint effort between Matanuska-Susitna Borough and Anchorage; although Matanuska-Sustina Borough dropped out of the project this year, Anchorage is still engaged in PFS planning. Lane County joined Los Angeles County in the implementation phase, beginning enrollment in the first phase of their project in September 2018. Exhibit 1 shows summarized progress for the sites through the end of Year 2 of the demonstration.

x

E XE C U TI VE SU M M AR Y

EXHIBIT 1

Pay for Success Permanent Supportive Housing Demonstration Sites

Status as of

Start of the

Demonstration

(October

December

December

Site

Intermediary (Grantee)

2016)

2017

2018

Anchorage, AK

United Way of Anchorage

Feasibility

Feasibility

Transaction

analysis

analysis

structuring

Austin/Travis County,

The Ending Community

Transaction

Transaction

Transaction

TX

Homelessness Coalition

structuring

structuring

structuring

Lane County, OR

Third Sector Capital Partners Feasibility

Feasibility

PFS contract

analysis

analysis

implementation

Los Angeles County,

Corporation for Supportive

Transaction

PFS contract

PFS contract

CA

Housing

structuring

implementation implementation

Montgomery and

American Institutes for

Feasibility

Feasibility

Transaction

Prince George’s

Research

analysis

analysis

structuring

Counties, MD

Pima County, AZ

The Sorenson Impact Center Feasibility

Feasibility

No longer part

at the University of Utah

analysis

analysis

of the

demonstrationa

Rhode Island

The Rhode Island Coalition

Feasibility

Feasibility

Transaction

for the Homeless

analysis

analysis

structuring

PFS = Pay for Success.

a Pima County is implementing permanent supportive housing but without the PFS financing structure.

In 2018, Urban continued to collect data from the sites through monthly communication (usually with the site’s intermediary organization), interviews with key informants (conducted during an annual 2-day visit to each site), and a review of important documents such as feasibility and success payment reports. In November 2018, Urban launched an Annual Partnership Survey. It was fielded to six to eight active partners on each site’s project and asked about the col aboration, data sharing, support for the target population, and barriers to service provision in the community.

E XE C U TI VE SU M M AR Y

x i

Cross-Site Findings on Feasibility Analysis and Transaction Structuring

Feasibility Reports

By the end of 2017, three sites had completed and shared feasibility reports (Austin/Travis County, Pima County, and Rhode Island). By the end of 2018, three more sites had completed and shared feasibility reports (Anchorage, Lane County, and Montgomery and Prince George’s Counties). 1 Most feasibility reports identified a target population and began assessing both the financial feasibility of a PFS project and the feasibility of implementing PSH. Most reports included information on the existing status quo costs, the costs of implementing PSH, and expected cost savings. During transaction structuring, sites often revised prior assumptions and updated previous analyses as new end payors became engaged and as project plans became more concrete.

A successful launch of a PFS project requires the completion of five crucial tasks, which are described below.

Building Pay for Success Partnerships

Building partnerships required educating stakeholders about PFS, a practice that continued through the transaction structuring phase as new stakeholders, and a broader set of partners became involved. When evaluators were involved in a project early, they were able to serve as thought partners and provide support. The following key PFS partnership themes emerged from our Year 2

data collection:



The PFS process increased collaboration and partnership and was widely mentioned by stakeholders during interviews as a benefit of the PFS process, and responses to the Annual Partnership Survey confirmed this. Most respondents believed that operational col aboration had increased during the preceding year.



One facet of increased collaboration is improved data access and sharing among actors.

PSH projects involve multiple sectors, and PFS projects require relevant data to assess the target population and its size and needs, as well as to understand the costs and possible cost 1 Los Angeles’s feasibility work predated the demonstration, so they are not expected to submit a feasibility report as a product of this project.

x i i

E XE C U TI VE SU M M AR Y

savings. The demonstration motivated several sites to improve their data-sharing infrastructure .



Collaborative work requires considerable investments of time by a diverse set of stakeholders, both in cross-sector partnerships to implement PSH and for the difficult work of structuring a PFS transaction. Most sites reported that a core group of partners had devoted considerable time to the project, whereas a larger group had devoted much less time.

Defining the Target Population

Defining the target population involves obtaining and matching data on homelessness, criminal justice, and (sometimes) healthcare utilization. The HUD NOFA requires that, at a minimum, the population identified must demonstrate a pattern of homelessness, 2 be involved with the criminal justice system, 3 and have high needs. Feasibility analyses helped assure stakeholders that the target population was large enough to merit an intervention and to support a rigorous impact evaluation.

Although all sites defined a target population during the feasibility analysis phase, several sites found that more work was necessary during the transaction structuring phase after they had begun better defining the program and identifying services and providers. Also, eligibility criteria occasionally evolved as new partners became involved, and some sites conducted additional analyses after securing access to additional data.

Establishing the Financial Feasibility of Pay for Success Most feasibility reports estimated the status quo costs of serving the target population, the implementation costs of PSH, and expected PSH cost savings for different sectors. Cost estimates varied widely, however, which suggests that sites used different assumptions and methods, perhaps tailored to the requests of

local stakeholders.

Although most sites initial y expected PSH to generate net savings to their jurisdictions (a primary motivation for some stakeholders), most sites’ analyses suggested that PSH would not generate net 2 People with a pattern of homelessness, defined as meeting the definition of chronic homelessness according to 24 CFR 578.3 or being homeless for 12 months over the past 3 years or for one or more nights during each year in the past 3 years.

3 Defined as multiple jail or prison stays within 3 years, including at least one in the past year.

E XE C U TI VE SU M M AR Y

x i i i

savings. Though some agencies could see savings, those savings were unlikely to exceed the costs of implementing PSH. As a result, partners in most sites focused less on net cost savings and more on spending money more effectively and achieving better outcomes for the target population.

Although PSH could reduce the service burden of the target population, sites often did not expect to generate considerable budgetary returns that could be reallocated to fund PSH. Fixed and capital costs cannot be recouped, and the possible recouped marginal costs per person can be quite low. For example, in corrections, the marginal costs for a day in jail may be only the cost of food and laundry, and some of the service resources that effective PSH might free up are expected to be used for other clients; though beneficial, those resources do not generate budgetary savings that can be used to fund PSH. Moreover, in some sites, services provided to the target population are partially funded by other parties, so any savings would not accrue to the local partners who could be end payors.

Generally, potential PFS end payors are government agencies that expect cost savings from the preventive effects of the interventions. In some cases, however, sites also sought to engage the healthcare system end payors because of the expected savings to that sector. Some sites worked with multiple end payors that might make success payments for different metrics, even though that makes financial transactions more complex.

PSH is a high-cost intervention. Sites sought additional funding sources, including philanthropic support and federal programs, such as 1115 waivers (which allow Medicaid funds to cover some PSH

service costs) and federal housing vouchers (which subsidize units). Sites sought other funding sources when the costs of implementing PSH were not expected to be completely covered by the resulting budgetary savings and to support the long-term sustainability of the project.

Most demonstration sites approached potential investors after securing end payors, establishing the project’s viability, and setting bounds on the project’s design. Relatedly, few government stakeholders discussed the risk of funding an ineffective program or cited a desire to transfer that risk to investors as a motivation for engaging in PFS; more stakeholders expressed concern about paying interest to investors, on the assumption that the program would be successful.

In 2018, political cycles interrupted the PFS transaction structuring process in three sites, with elections causing increased uncertainty about key decisionmakers and their commitments. Because the feasibility analyses and transaction structuring phases can be lengthy, this was not unusual.

x i v

E XE C U TI VE SU M M AR Y

Other legal and political factors that complicated transaction structuring included legislative restrictions on committing future funds, local government concerns about taking on debt and credit ratings, and budget cycles—particularly at the state level.

Establishing the Feasibility of Implementing Permanent Supportive Housing Establishing a feasible project requires assessing the feasibility of PSH, including examining enrollment pathways, housing availability, services and provider capacity, and potential training and technical

assistance needs.

Individual sites have adopted different enrol ment pathways. In Los Angeles County, the first site to enter the implementation phase, potential participants are incarcerated when first contacted through jail in-reach4. In Lane County, the second site to enter the implementation phase, the project serves people reentering the community from prison, and participants are identified after their release.

Several sites identified possible service providers in 2018. Across sites, there was wide variation in provider experience with an approach called Housing First that is key to PSH. It supplies permanent housing without preconditions or barriers; it then engages clients in voluntary supportive services.

Providers new to Housing First may need time to adapt, and working with the target population may require recruiting new staff and strengthening internal capacities, such as the staff Housing First knowledge base and the skills necessary to implement. Some contracted service providers in Los Angeles County did not have Housing First experience before this project. Stakeholders in Anchorage indicated that only one organization in the area had operated PSH using a Housing First approach, and that familiarity with Housing First practice varied across the site. Stakeholders in Montgomery and Prince George’s Counties pointed out that gaps in knowledge of Housing First practice could be a concern, also indicating that the two jurisdictions did not have the same Housing First knowledge or experience. Implementing Housing First with people who are reentering the community and may be under supervision requires building a shared understanding with parole and probation departments about supervision requirements and revocation policies that can be implemented along with Housing First.

4 Jail in-reach, in contrast to outreach, refers to service provision activities that begin while someone is in custody. Depending on the model, these services may continue after release to help ensure continuity if care E XE C U TI VE SU M M AR Y

x v

Two sites launched pilot projects in 2018 to answer important questions related to service delivery and project design before the planned full project implementation. By the end of 2018, through these pilot projects, Austin/Travis County had placed 24 people in market-rate apartment PSH units, and Lane County had placed 92 people in new PSH units. The pilots were intended to test enrollment pathways and to define relationships and divisions of labor among project partners.

Although pilot participants’ outcomes might be monitored, success payments are not included in these pilot plans.

Establishing Success Metrics and Evaluation

Finally, by the end of the transaction structuring phase, sites must establish metrics for success and initiate an independent evaluation. Site partners must agree on specific metrics and benchmarks that must be met to trigger success payments. By the end of 2018, four sites were still finalizing their payment outcomes.

As sites developed their success metrics and payment outcomes, the following common themes emerged:



End payors were an important part of the discussion about successful outcomes.



Metrics should be easy to understand, but nuanced.



Research provides guidance on some measures and benchmarks but is unclear on others.

Evaluations are core components of PFS, and evaluation partners play an essential role in the project. During project implementation, evaluators support PFS partners in monitoring program implementation, recommending course changes as necessary, and understanding the results of the project. Evaluators ensure that the project adds to the knowledge base of what works, both for PFS

financing mechanisms and for evidence-based programs, to achieve outcomes of interest for the target population. Finally, evaluators provide the documentation and validation to trigger success payments. Because of this, it can be useful to have an evaluator at the table during the transaction structuring phase, when decisions about success metrics are being made.

Al but one site had selected an evaluator by the end of 2018. The following themes emerged across sites as they worked with these partners to solidify an evaluation plan:



Success payments will be made on outcomes, not impact.

x vi

E XE C U TI VE SU M M AR Y



There was considerable uneasiness with random assignment designs because of concerns about sample size and the ethics of withholding services. Most sites planned to use quasi-experimental designs with nonrandomized comparison groups to determine program outcomes.



Many sites wanted to start making success payments before the final evaluation outcomes, and many sites planned to base success payments on program participants’ early observed outcomes. Housing stability was the most common payment metric, sometimes coupled with avoidance of reincarceration.



Stakeholders aspired to fund process studies to understand project implementation better.



Data fragmentation is an ongoing challenge, and sites were designing automated dashboards (i.e. tools that provide at-a-glance information from complex data) to track progress.

Implementing PSH

Los Angeles County and Lane County implemented PSH through PFS by the end of 2018.

Los Angeles County

Los Angeles County launched its implementation phase in October 2017. By the end of 2018, the project had placed 182 people in housing. The project uses a jail in-reach model, enrolling people who had been sentenced to probation, had experienced homelessness, and had other needs (for example, substance use and mental health disorders) within the county criminal justice system. Upon leaving jail, participants enter interim housing5 in the community (these placements last 6 to 9

months and have more rules and requirements than the eventual PSH placement). After participants are placed on a “housing ready” list, the project’s housing search agency pairs them with potential housing. At the start of the implementation phase, clients transitioned from interim housing to a permanent unit within 3 to 6 months; however, that time frame had stretched to 9 to 12 months by the end of 2018. Though this enrollment pathway is aligned with the probation department’s required 5 People are placed in interim housing while awaiting placement in the PSH program. Interim housing contracts are administered by the Office of Diversion and Reentry and come in a variety of different housing and service structures. For example, some interim housing placements are congregate living style, where clients have some personal space, and others are shelter style, where clients share spaces. Some interim housing providers mandate service participation, and others do not.

E XE C U TI VE SU M M AR Y

x vi i

supervision policies, the use of interim housing and the conditions of supervision are deviations from the Housing First approach. The program offers a variety of interim housing, from a large shelter to clinically enhanced shelters with more specialized case management and medical services. Across all the diverse types of interim housing, however, people are no longer in custody and are not in locked settings.

Four organizations provide intensive case management services. As project implementation ramped up, service providers experienced high levels of staff burnout and turnover, and they struggled to keep sufficient staffing levels for ongoing program enrollment. The project responded to these challenges by increasing the budget to allow service providers to hire a licensed clinician to support clients’ serious mental health needs and by decreasing the staff-to-client ratio from 1:20 to 1:15.

In addition, service providers reported several implementation challenges that stemmed from the target population’s criminal justice involvement. Those challenges included the fol owing:



Working with the reentry population required working with local criminal justice partners, including the probation department, the sheriff’s department, and the court system. This required a high level of coordination and exchange of information.



Caseworkers were required to provide documentation to the courts showing that participants’

activities complied with court orders. It was unclear whether participants knew of that requirement or whether it affected the relationship between participants and service providers.



Securing buy-in for the program from clients was not a chal enge when they were in jail, but they sometimes lost touch once released.



Planning with the reentry population was difficult because of unpredictable jail release dates, which made anticipating which clients would need bridge housing and coordinating permanent housing units a challenge. The Office of Diversion and Reentry (ODR) assisted by becoming involved in court cases and helping track and manage conditional releases.



Landlords often created barriers for—or screened out—people with criminal records, particularly people who had been convicted of sex offenses.

In order to measure outcomes for success payments and for the broader evaluation of project effectiveness, the project designed paral el processes. Success payments were made for the first three quarters of implementation based on 6-month housing retention outcomes.

x vi i i

E XE C U TI VE SU M M AR Y

Lane County

Lane County began implementing the project in September 2018. This report documents the early implementation of the program. 6 Participants are identified through a collaborative process between the service provider (Sponsors, Inc.), the Lane County Office of Parole and Probation, and the local public housing agency (Homes for Good). As of the end of 2018, Lane County referred 43 people for the program and determined that 20 of those referrals were eligible and consented for housing. Five people in that group were successfully placed in housing through the project. Homes for Good provides project housing through three pathways: an affordable housing project, public housing, and housing choice vouchers. Low turnover at the affordable housing project, as well as landlords’

perceived reluctance to rent to program participants, led to slower-than-expected enrollment.

Sponsors, Inc., a reentry service provider that has traditional y focused on transitional housing, provides services. This project is the first experience of Sponsors, Inc. with Housing First and adapting to serving the target population, and the model was an essential part of technical assistance.

The Benefits and Costs of Using Pay for Success to

Launch Permanent Supportive Housing

The Urban Institute’s formative evaluation asks the following question: What are the benefits and costs (or advantages and disadvantages) of launching PSH for the target population of frequent utilizers through a PFS process? This evaluation is not a traditional benefit-cost analysis. An annual partnership survey captures the stakeholder perceptions of benefits. Costs are captured through quarterly requests for the time spent, by person, on the PFS initiative. Additional qualitative information on benefits and costs are collected through in-person, semi-structured interviews.

Stakeholders broadly reported that the demonstration was catalyzing partnership and col aboration and was helping force improved cross-sector data sharing. The cross-sector collaboration was necessary for successfully implementing PSH for the target population, and it involved different agencies and levels of government. Some stakeholders believed that PFS served to engage partners and additional stakeholders, to increase awareness of the target population and its high-cost load, and to educate people about PSH.

6 The first full year of implementation wil be documented in future reports.

E XE C U TI VE SU M M AR Y

x i x

Some stakeholders also believed that the demonstration was helping change traditional thinking about funding and about blending funding streams, as well as helping shift thinking to program outcomes rather than program inputs .

Collaborating across service sectors, levels of government, and public and private stakeholders requires considerable time, however. Some of that time would be spent launching any PSH project with the target population, but launching a PFS project and getting to a transaction adds considerable time to the process. Our qualitative data suggest the projects have involved intense time commitments for a core group of partners and a relatively smal time commitment for a wider group of stakeholders. In the last quarter of 2018, Urban began collecting data on the PFS project’s time costs; results from that work will be presented in future reports.

Whether the benefits from PFS warrant those time costs wil partly depend on whether the sites are successful in launching PSH projects through the demonstration. We wil continue to monitor the strengths of partnerships through an Annual Partnership Survey and to collect additional data on time use. Future reports will integrate that data with information on the sites’ progress (as more of them move into program implementation) and on each site’s chal enges and successes.

x x

E XE C U TI VE SU M M AR Y

I. Introduction

Overview

In 2016, the U.S. Department of Housing and Urban Development (HUD) and the U.S. Department of Justice (DOJ) entered an interagency agreement to launch the Pay for Success (PFS) Permanent Supportive Housing (PSH) Demonstration (the demonstration). The purpose of the demonstration is to help communities reduce recidivism and homelessness among the reentry population by increasing PSH through PFS, a promising financing mechanism that aims to unlock private and philanthropic investment in social programs and to improve the outcomes of social programs.

PSH is an evidence-based intervention that combines a housing subsidy and intensive supportive services to help people who experience long-term homelessness and cycle in and out of jail, prison, and crisis services such as hospital emergency departments and psychiatric and detoxification centers.

This cycling is detrimental to their health and wel -being and comes at a high cost to the public.

Studies show that PSH results in long-term housing stability, improved physical and behavioral health outcomes, and reduced use of crisis services, including jails (Aidala et al., 2014; Culhane, 2018). PSH

embraces Housing First, a philosophy in which permanent housing is provided without preconditions or barriers to entry, such as required treatment, service participation, or sobriety (HUD, 2015).

Despite strong evidence showing PSH works, it is not available at the scale necessary to address the need in many communities. Thousands of people who cycle in and out of jail or leave prison without a stable housing option experience homelessness each year (USICH, 2016). The demonstration aims to increase PSH by tapping private investors to pay the up-front costs of PSH for a reentry population experiencing a pattern of homelessness. If the intervention is successful, an end payor, usual y the government, pays for “success,” as measured by reductions in homelessness and recidivism and improved health outcomes. This structure involves multiple actors within a jurisdiction, including governments, funders, financial intermediaries, knowledge intermediaries, service providers, and independent evaluators.

As part of the interagency agreement between HUD and DOJ, DOJ made the Second Chance Act (SCA) funds available for the demonstration. Congress passed the SCA (Public Law 110-199) with bipartisan support in 2008. The goals of the act are to increase reentry programming and improve outcomes for people released from state prisons and local jails. The DOJ’s Bureau of Justice Assistance (BJA) and Office of Juvenile Justice and Delinquency Prevention (OJJDP), located within the Office of H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

1

Justice Programs, award SCA grants to communities. Grants from the SCA support efforts related to employment assistance, substance use disorder treatment, housing, family programming, mentoring, victim support, and other reentry efforts. Grants must be col aborative across criminal justice and social service systems, and grant recipients must collect data to measure performance. BJA and OJJDP

have funded dozens of projects to improve reentry outcomes in communities across the United States. Evidence shows that a subset of the reentry population experiences homelessness and that PSH is a cross-sector solution that results in housing stability and reduced recidivism, Consequently, PSH is a good fit for achieving the goals of the SCA. Accordingly, DOJ gave HUD the responsibility of issuing a Notice of Funding Availability (NOFA) and providing oversight for the demonstration in partnership with DOJ.

HUD issued the NOFA in October 2015, inviting applicants from nonprofit organizations and institutions of higher education to apply for grant funds. Applicants could apply for funding for work in three phases of PFS projects: the feasibility analysis phase, the transaction structuring phase, and the implementation phase. During the feasibility analysis phase, sites form partnerships and conduct analyses to determine whether a PFS-funded PSH project is feasible. During the transaction structuring phase, sites work through the details needed to complete a PFS transaction, including details of PSH implementation and PFS financing. During the implementation phase, sites implement PSH, launch an independent evaluation, and determine success.

HUD-DOJ evaluated applications from across the country on a competitive basis, awarding $8.7

mil ion to seven grantees in June 2016. Grantees could receive a maximum of $1.3 million to support the three phases of PFS—feasibility analysis, transaction structuring, and project implementation—as well as evaluation and success payments.

HUD-DOJ contracted with the Urban Institute (Urban) to conduct a formative national evaluation of the demonstration. The goals of Urban’s evaluation are to examine how PFS is implemented in each site, capture lessons learned across sites, and examine the benefits of using PFS to fund PSH for a reentry population. This report on the demonstration’s second year follows a previous report on the baseline year (Liberman et al., 2019).

2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Background

Permanent Supportive Housing

A considerable need exists for PSH; although homelessness in the United States has declined significantly in the past decade, trends have reversed in the past few years, driven in large part to increases in homelessness in large cities. In 2018, nearly 553,000 people across the country experienced homelessness on a given night, a slight increase from the previous year (Henry et al., 2018). People experiencing chronic, or long-term, homelessness increased by 2 percent over the previous year, to 88,640 people. Chronically homeless people are more likely to stay in unsheltered conditions, on the streets, or in places considered unfit for human habitation.

Research demonstrates the link between incarceration—and other kinds of involvement with the criminal justice system—and homelessness. In 2016, more than 6.6 million people were held in prisons and jails or under supervision through probation and parole in the United States. Of those, 740,700

people were held in local jails (Kaeble and Cowhig, 2018). Furthermore, people living in unsheltered conditions are exposed to the possibility of interactions with the criminal justice system because they may be jailed for public nuisance offenses such as panhandling, trespassing, or public intoxication.

Incarceration can then exacerbate housing instability. It also compounds the difficulty of addressing underlying health and mental health conditions; when people reenter the community, they may experience interruptions in medications or treatments or cease receiving all care. Instead, they may be forced to rely on crisis systems, including sobering centers and hospital emergency rooms, which are both costly and ineffective strategies for maintaining long-term health. Although this population is responsible for significant costs to local institutions (including the government) because of jail stays and use of shelters and emergency health care, no one system is adequately positioned or prepared to address its needs comprehensively. People fal through the cracks. These findings clearly illustrate the need for PSH.

PSH is an evidence-based intervention that fol ows a Housing First model, meaning that people are placed in housing without preconditions (such as sobriety). PSH also supports them by providing (but not mandating) services to address health and other needs. Unlike other types of housing assistance, such as rapid rehousing or transitional housing, PSH provides long-term subsidies to make housing affordable, ensuring residents pay no more than 30 percent of their income toward rent. The subsidy may be project-based, for housing in a specific building, or tenant-based, so that people can use the subsidy for a variety of housing options. Fortunately, PSH capacity has increased dramatical y in recent years.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3

PSH can reduce incarceration among this population. In a quasi-experimental study of supportive housing for frequent utilizers, Aidala and col eagues (2014) found a 40 percent reduction in jail days over 24 months for people in supportive housing compared with a quasi-experimental comparison group. In 2018, there were more than 361,000 PSH slots nationwide, and permanent housing beds made up 40.3 percent of all available housing (including emergency shelters, transitional housing, and other types of beds) for homeless people (Henry et al., 2018). Although PSH is a critical intervention, it can be costly. The PFS model is one way that jurisdictions can finance PSH and other evidence-based interventions.

Expanding Permanent Supportive Housing Using Pay for Success Developing PSH for people who are experiencing a pattern of homelessness and frequently interact with criminal justice and health services can be challenging; it requires collaboration from many stakeholders—including housing, corrections, and health agencies—as well as supportive service providers. It can also involve coordination across multiple agencies at different levels of government, including city, county, and state.

Financing for PSH can be difficult to marshal because the up-front costs are usual y borne by one system, even though the program’s preventive benefits and cost savings accrue to systems that do not share implementation costs. This is often referred to as the “wrong pockets” problem. For the reentry population, the cost of PSH is typically carried by homelessness programs and housing systems, such as state and local housing agencies and Continuums of Care, which are local planning bodies responsible for coordinating federal funding for homelessness programs. Benefits beyond housing stability—such as reductions in recidivism and improvements in health, which are the big-ticket cost offsets—are accrued by corrections and health systems. Another barrier to financing PSH is that expenses are significant and up-front, whereas benefits and savings accrue later.

A central idea of PFS is to help governments shift from paying for services directly to paying only for successful outcomes from those services. Risk aversion among governments often limits the implementation of preventive programs like PSH. Under traditional public financing, governments assume the risk of paying to implement interventions that may ultimately be ineffective. When budgets are lean, this risk may prevent any funding from being allocated (Galloway, 2015), and tightened budgets limit opportunities to experiment with innovative prevention strategies, reducing or eliminating the prospect of downstream savings to governments (GAO, 2015). PFS aims to shift the risks of financing a program that could be ineffective from governments to private investors.

4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Additionally, PFS can provide an opportunity to increase the scale of evidence-based programs like PSH. In PFS projects, governments pay for programs only when the programs have demonstrated

“successful” outcomes for their participants, based on agreed-upon success criteria. Private investors fund the programs in the interim, and the government repays them with a return on the investment if the program is successful. Box 1 shows HUD’s description of PFS in the NOFA for this demonstration.

BOX 1

HUD Notice of Funding Availability Definition of Pay for Success

“PFS [Pay for Success] strategies are typically public-private arrangements that enable a government to test or expand innovative programs while paying only for those that achieve agreed-upon target outcomes. PFS strategies are typically associated with preventative social interventions.

They are put in action through PFS contracts between a government (or other payors for social outcomes) and a nonprofit social service provider. PFS contracting models involve end payors and service providers who agree that all or some portion of the payment will not be paid until an agreed-upon set of outcomes or level of impact has been verified. Such payments for outcomes are known as success payments. Instead of being applied to the direct costs of housing or services, success payments are made based on the degree to which specific milestones agreed upon in advance are achieved, using payment rates also determined in advance. Achievement of outcomes is typically verified by an independent evaluator agreed upon by all parties to a transaction.

Given that verification of outcomes that trigger success payments may take several years, service providers often will not have the resources to self-finance costs of implementing a preventative intervention during a contract period. For this reason, PFS contracting may require PFS financing, sometimes referred to as ‘social impact bonds’ or ‘outcomes financing,’ through which third-party investors provide financial resources necessary to carry out the intervention. Such third-party investment is typically at-risk, and return of capital (and any potential return on investment) via the payor is dependent, in whole or in part, on the achievement of outcomes identified in the PFS

contract.”

Source: U.S. Department of Housing and Urban Development (HUD) 2015, Pay for Success Permanent Supportive Housing

Demonstration NOFA

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

5

By transferring much of the up-front cost and risk of preventive programs to private investors, and by requiring a rigorous evaluation to direct how and when the government will pay for outcomes, PFS can give governments more flexibility to implement promising prevention strategies and programs (GAO, 2015). Similarly, PFS can help circumvent the “wrong pockets” problem by allowing government end payors to pay for beneficial outcomes in their domain (for example, health) from investment in another domain (for example, housing) (Roman, 2015).

The PFS field is in the early stages of development, and the use of PFS as a financing mechanism for PSH, in particular, is in its infancy. In addition to the two launched through this demonstration, PFS

projects are funding PSH interventions in Massachusetts, the County of Santa Clara, California, and the City and County of Denver, Colorado. The Massachusetts and Santa Clara projects are serving chronically homeless people and are using housing retention as the outcome for success payments.

The Denver project is serving people who are homeless and have frequent criminal justice involvement. It is using two measures for success payments: housing retention for interim payments and the impact on jail stays (as measured by a randomized study) for the final payment. According to the Nonprofit Finance Fund, seven other PFS projects that address homelessness, not including the sites in the demonstration, are being developed (NFF, 2019b).

During the feasibility analysis phase in Denver, Urban helped develop the logic model, define the target population, and provide evidence for the assumptions included in the financial model. It also worked with the PFS team to develop the research design. Now that Denver is in the project implementation phase, Urban is serving as the independent evaluator. The latest evaluation report (Cunningham et al., 2018) presents promising evidence. Nearly 3 years into the project, participants have high rates of housing stability and have spent less time in jail than predicted. Overall, 285 people have been leased-up through the program (usually within 6 months of being referred to service providers), and 85 percent have remained in housing. These are promising housing stability results and strong interim outcomes. After 1 year in housing, 44 percent of housed participants have stayed out of jail. Housed participants spent significantly less time in jail (34 days, on average) than was predicted for this target population in the absence of housing (77 days). The evaluation will release findings on the 3-year impact on jail stays in 2021.

Although PFS might be the most visible embodiment of the evidence-based decisionmaking movement, governments are looking at other results-based approaches and funding models. This includes a more widespread adoption of rigorous performance measurement. It also includes management approaches and models that tie performance and outcomes to payments through vehicles like outcome-based contracting. These models aim to incentivize improved results,

6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

emphasize the importance of data and evidence in making funding decisions, and demonstrate good stewardship of public resources.

The 2018 Social Impact Partnerships to Pay for Results Act provides $100 million in new federal funding for PFS projects, with resources available through the U.S. Department of the Treasury for outcome payments and project costs like evaluations and feasibility assessments. By requiring projects to demonstrate the potential to generate federal value—through increased revenue, decreased costs, or a combination of the two—the act returns to an original tenet of PFS: the ability to fund upstream projects that are cost-effective and yield net downstream benefits (benefits outweighing the project costs). Although the field has general y moved away from demonstrating net cashable savings, the desire to show that PFS is more cost-effective is widespread.

Pay for Success Partners

Exhibit 2 provides an overview of the key actors typically involved in PFS projects. Under a typical PFS

arrangement, governments function as project initiators, identifying which outcomes to pay for if established metrics are met. Private and philanthropic funders provide the up-front source of capital, assuming both the financial risk and the potential benefit of financial return if the project meets specified outcome metrics. The term “intermediaries” refers to two conventional roles, which can be played by the same or separate partners. The financial intermediaries provide deal-making expertise for the project, such as facilitating negotiations and assembling requisite funds. The knowledge intermediaries provide support for the content of the intervention, including identifying high-performing programs, managing performance, and conducting ongoing research on PSH models and the target population. Service providers implement the chosen program model for the target population to achieve agreed-upon outcomes. Independent evaluators assess whether the intervention has met the agreed-upon outcome metrics.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

7

EXHIBIT 2

Pay for Success Actors

PFS = pay for success.

Source: Urban Institute (2016)

Existing PFS projects have diverse stakeholders: end payors, which are often city, county, or state governments; private and philanthropic funders, including foundations, individual investors (brokered through a firm), and large multinational banks; service providers, such as local or national nonprofit organizations, school districts, or public housing authorities; and independent evaluators, such as nonprofit research firms and public universities.

The HUD-DOJ Pay for Success Permanent Supportive

Housing Demonstration

HUD and DOJ launched the PFS-PSH Demonstration in 2016. The demonstration funded sites to explore launching a PFS project to scale up PSH in their communities for people involved with the criminal justice system who are experiencing homelessness. By encouraging collaboration among housing, criminal justice, and healthcare systems, the demonstration aims to address this population’s needs better and save money for the jurisdictions.

8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Demonstration Theory of Change

Exhibit 3 shows the theory of change behind using PFS to fund PSH, along with expected results. A theory of change is an articulation of how a program or intervention is expected to bring about specific outcomes or results. The target population for the demonstration interacts with several systems, including the criminal justice system and those that address housing, physical health, and behavioral health needs. These interactions can be tremendously costly to individual sectors and on aggregate, but because the chal enge is cross-sectoral coordination and collaboration among systems can be difficult. At the center of its theory of change is the idea that PFS brings cross-sector partners together to col aborate for better outcomes for a vulnerable population and to work strategical y in tandem. Bringing these sectors together to investigate the feasibility of PFS can increase the use of evidence to inform program development, facilitate the sharing of data to understand the target population, promote the identification and removal of system barriers to providing housing for homeless reentry populations, and obtain investor funding to implement PSH.

PSH integrates rental assistance with case management and the services of multiple systems to deliver coordinated care. Using a Housing First approach engages people and helps get them into housing as quickly as possible, supplies permanent housing without preconditions or barriers, and then engages them in services. The provision of these services has several expected outcomes, including decreases in recidivism and criminal justice system involvement, and increases in housing stability, and appropriate healthcare service use, which will lead to improved health outcomes. All these outcomes should reduce costly stays in shelters, jails, and prisons, as well as the use of crisis health and mental health care services.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

9

EXHIBIT 3

Theory of Change: Using Pay for Success to Fund Permanent Supportive Housing Cost Avoidance,

Individual-Level

Offsets, or Savings

PFS

PFS Outcomes

Intervention

Outcomes

Accrued

Bring together

Data sharing

Housing First

Increase housing

Avoid costly shelter

cross-sector

 to define the

stability

stays, jail/prison stays,

partners,

target population Housing subsidy

 reduce homelessness and use of crisis

including

healthcare services.

 provide rental

 provide a safe,

private

Use evidence base

assistance for a

healthy, and stable

investors, to

 to define

housing unit that is

housing unit

investigate the

outcome-based

safe, sustainable,

 decrease shelter use

feasibility of

payments

functional, and

PFS, structure

Decrease criminal

Identify and

conducive to tenant

the transaction,

justice involvement

remove system-

stability

and develop a

and reentry to jail and

level barriers

PFS contract.

Case management

prison

 increase access

 decrease arrests

to effective

services

 decrease jail and

services

 develop a case plan

 facilitate access to

prison days

 to improve

 decrease jail and

outcomes

benefits

 provide referrals

prison reentry

Increase funding

 coordinate care and

Increase the use of

and PSH

system involvement

appropriate health

 for the target

care services and

population

Coordinate with

community

improve health

corrections for people  decrease detox visits

on probation or parole  decrease avoidable

ER and hospital visits

 connect to mental

and physical health

care and substance

abuse treatment

 decrease the severity

of illness

 improve mental

health

 improve physical

health

ER = emergency room. PFS = pay for success. PSH = permanent supportive housing.

Demonstration Requirements and Phases

Each demonstration site could receive up to $1.3 mil ion for PFS phases: feasibility analysis, transaction structuring, and project implementation, including evaluation and success payments.

1 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

HUD-DOJ set criteria for participation in the demonstration to provide standard guidelines while al owing sites to tailor their projects to address the specific needs of their communities. Each project must do

the following:



Focus on high-needs people who are experiencing a pattern of homelessness and have been or are currently involved with the criminal justice system, using definitions shown in exhibit 4.



Fund at least PSH 100 units for people in the target population.



Use an evidence-based PSH intervention with a Housing First approach that will “address the needs of the target population—permanent affordable housing paired with voluntary supportive services that help individuals remain in housing, live with maximum independence, connect to needed clinical and mainstream services, and facilitate the attainment of their goals and aspirations.” 7 Required components are shown in exhibit 4.



Engage in eligible PFS activities, defined in three phases: feasibility analysis, transaction structuring, and contract implementation. This last phase also includes success payments and evaluation.

EXHIBIT 4

HUD-DOJ Requirements for the Target Population and the Permanent Supportive Housing Intervention

Target Population

PSH Model Components

 people with criminal justice involvement, defined

 outreach

as multiple jail or prison stays within a 3-year

 quality, permanent, and affordable housing

period, including at least one in the past year

 accessible transportation and employment

 people with homelessness history, defined as

opportunities

meeting the definition of chronic homelessness

 Housing First approach

according to 24 CFR 578.3 or being homeless for

 housing stability services

12 months over the past 3 years or for one or

 informed property or landlord management

more nights during each year in the past 3 years

 care management and service coordination

 people with high needs, defined as having a

 coordination with the criminal justice system (for

history of high-cost use of services or significant

example, courts, or community corrections)

physical or behavioral health challenges that

require high-cost support

PSH = permanent supportive housing.

7 https://www.hud.gov/sites/documents/2015-PFSPSHDEMO-NOFA.PDF.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

1 1

Demonstration Sites

In June 2016, HUD awarded a total of $8.7 mil ion to seven sites: Anchorage, Alaska; Austin/Travis County, Texas; Lane County, Oregon; Los Angeles County, California; Montgomery and Prince George’s Counties, Maryland; Pima County, Arizona; and Rhode Island. Grant agreements were finalized in October 2016. Los Angeles County began in the transaction structuring phase, and in 2017

completed a PFS contract and began implementation. Austin/Travis County, Texas, also began the demonstration in the transaction structuring phase and was still in that phase at the end of 2018. The other five sites began their projects in the feasibility analysis phase. After completing its feasibility analysis, Pima County decided not to pursue transaction structuring, instead deciding to fund its project outright, possibly using performance-based contracting.

By the end of 2018, Anchorage, Montgomery and Prince George’s Counties, and Rhode Island had al entered the transaction structuring phase. Lane County had moved through the transaction structuring phase into the implementation phase (it began enrolling people in the first phase of its project in September 2018). Los Angeles County, meanwhile, continued its implementation throughout 2018. Exhibit 5 shows the sites and their progress. This report focuses heavily on the transaction structuring phase of the demonstration.

EXHIBIT 5

Pay for Success Permanent Supportive Housing Demonstration Sites

Status as of

Start of

Demonstration

(October

December

December

Site

Intermediary (Grantee)

2016)

2017

2018

Anchorage, AK

United Way of Anchorage

Feasibility

Feasibility

Transaction

analysis

analysis

structuring

Austin/Travis County,

The Ending Community

Transaction

Transaction

Transaction

TX

Homelessness Coalition

structuring

structuring

structuring

Lane County, OR

Third Sector Capital Partners Feasibility

Feasibility

PFS contract

analysis

analysis

implementation

Los Angeles County,

Corporation for Supportive

Transaction

PFS contract

PFS contract

CA

Housing

structuring

implementation implementation

Montgomery and

American Institutes for

Feasibility

Feasibility

Transaction

Prince George’s

Research

analysis

analysis

structuring

Counties, MD

1 2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Pima County, AZ

The Sorenson Impact Center Feasibility

Feasibility

No longer part

at the University of Utah

analysis

analysis

of the

demonstrationa

Rhode Island

The Rhode Island Coalition

Feasibility

Feasibility

Transaction

for the Homeless

analysis

analysis

structuring

PFS = pay for success.

a Pima County is implementing permanent supportive housing but without the PFS financing structure.

Year 2 Evaluation Questions and Methods

HUD-DOJ contracted with the Urban Institute to conduct a formative national evaluation of the demonstration. This evaluation is distinct from the other evaluations that will be conducted at each site to determine the impact of the projects. Urban’s national evaluation aims to understand how PFS

is implemented in each demonstration site, capture lessons learned across sites, and examine the benefits of using PFS to fund PSH for a reentry population.

The evaluation’s design provides a multidisciplinary, multimethod approach to “learn as we do.”

The evaluation’s objectives are to document progress and processes at the sites and to understand the benefits and costs of using PFS to support the implementation of PSH. As part of the evaluation contract, Urban also provides consultation to demonstration grantees and their independent third-party evaluation teams on data col ection and evaluation design.

During the first project year (through the end of 2017), five of the seven grantees were in the feasibility analysis phase, which was the main focus of Liberman and coauthors’ (2019) baseline implementation report. After Pima County exited the demonstration, five of the remaining six sites spent most of 2018 in the transaction structuring phase, in which sites move toward a PFS transaction.

Consequently, this Year 2 implementation report concerns 2018 and primarily documents the requirements, challenges, and accomplishments of the transaction structuring phase. Questions to be addressed include the following:



What have been the primary tasks and chal enges? Who have been the primary partners?



Is the PFS-PSH project changing “business as usual” for the partners involved?



What are the proposed local data col ection efforts and evaluation plans, and what challenges do they face?



How does the work change between the feasibility analysis and transaction structuring phases?

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

1 3

Lane County was the only site to move into the implementation phase near the end of 2018; section II details the Lane County PSH project model, financing arrangements, and success payments.

Los Angeles County’s work in 2018 was considerably different from that of the other sites. It did not request funding for feasibility analyses because of prior feasibility analyses and pilot work.

Instead, Los Angeles County’s grant began in the transaction structuring phase, and, by the end of 2017, the site had a signed PFS contract and had begun implementation. By the end of 2018, it had begun making success payments. Therefore, Los Angeles County’s 2018 work has been in the project implementation phase and is described in section IV.

Data Collection Activities

In Year 2, Urban continued qualitative data collection activities that it had begun in Year 1. Those activities included the following:



Monthly communication. The evaluation team conducted monthly calls with site liaisons (usually the intermediary), sometimes by joining the site’s team meetings, to learn about grantees’ progress and any barriers to progress.



Key informant interviews. The evaluation team conducted a 2-day visit to each site to interview key local actors. Interview respondents included staff members from the intermediary organization, government end payors (potential and secured), service providers (potential and secured), investors, and evaluators. The interviews focused on the primary tasks in each PFS phase, milestones achieved, chal enges the sites faced and how they were mitigated, and the benefits and costs of the PFS project. A coding scheme was developed for qualitative data, based on eligible activities prescribed by HUD-DOJ for each grant phase (described in exhibit 3 above), and guided by our research questions. This was applied to 2018 monthly call notes and site visit interview notes using NVivo software.



Document review. The team reviewed feasibility reports, local evaluation plans submitted by sites to HUD, and other project materials that described the program and PFS development at the sites.

Urban also began the following two new data col ection activities in Year 2:



Cross-site project enrollment tracker. In 2018, Urban developed a tracker to document monthly site enrol ment progress once sites begin enrol ing people into their projects. This typically starts once a complete PFS contract is signed and the site formally enters the

1 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

implementation phase, although it is also used to track pilot periods that might not be a part of the contract years. The tracker includes metrics such as housing referrals, housing placements, housing exits, progress toward target placement, and reasons for housing exit.

Urban site teams review monthly progress and receive the data for the tracker during their regularly occurring calls with project intermediaries. This information is fed into the tracker by Urban project staff members monthly, with documentation explaining the time frame covered, which can differ among sites as a result of varying reporting schedules.



Annual Partnership Survey. The survey was launched in November 2018, with a sample of six to eight active partner organizations on each site’s project, for a total of 129 potential respondents. This survey was intended to measure changes over time in the strength of the partnerships and in the benefits at the population- and system-levels. The survey asked questions about respondents’ backgrounds and roles in the project, collaboration with partners, data sharing, and barriers to service provision. The survey was administered using Qualtrics, a web-based software, and was designed to take 15 to 20 minutes to complete.

In 2018, 71 stakeholders participated, a 55 percent response rate. 8 Exhibit 6 shows who responded to the survey, including breakouts by site, organization role on the PFS project, title within the organization, and tenure on the PFS project. Responses to some survey questions from the 2018 survey wave are included in this report. Future reports will present changes over time. These descriptors of the respondents provide helpful context about the findings from the 2018 baseline survey. For example, some sites (like Anchorage) were represented more heavily than others (Austin/Travis County and Rhode Island). Nearly one-half of respondents were directly involved with the target population, either as a direct service provider, a housing partner, or as a referral source. Seventy percent of respondents had been involved in the project for at least a year or longer.

8 Several factors may have contributed to the low response rate. The survey launched shortly after the 2018

midterm elections, by which time approximately 8 percent of our sample of potential respondents had left government positions, and replacements knowledgeable about the PFS program could not be found. In addition, the survey was launched just before the end-of-year holidays (although the survey was held open for more than a month into the new year). The original timeline scheduled this launch for earlier in the year, but it was delayed because the Office of Management and Budget review period was longer than expected and because of the federal government shutdown. Two sets of email reminders were sent, intermediaries were asked to contact non-respondents to encourage participation, and a round of phone cal s was made to non-respondents late in the 2-month fielding process.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

1 5

EXHIBIT 6

Annual Partnership Survey: Respondents

Those who

Completed

Total

Site Sample

N

Survey (%)

Sample (%)

(%)

Site

Anchorage

17

24

13

57

Austin/Travis County

9

13

7

69

Lane County

13

18

10

62

Los Angeles County

13

18

10

68

Montgomery and Prince George’s Counties

10

14

8

30

Rhode Island

9

13

7

69

Total

71

100

55

55

Organization role on PFS project

Intermediary

11

16

9

Evaluator

2

3

2

Other technical assistance

8

11

6

Financial partner

11

16

9

Direct service for the target population

15

21

12

Housing partner for the target population

10

14

8

Criminal justice: referral source for the target

6

9

5

population

Other role

8

11

6

Total

71

100

55

Title or position within the organization

Director

22

32

17

Deputy

21

30

16

Officer

26

38

20

Total

69

100

53

Tenure on PFS project

Less than 6 months

9

13

7

6–12 months

12

17

9

12–18 months

11

16

9

18–24 months

12

17

9

24–30 months

9

13

7

More than 30 months

18

25

14

Total

71

100

55

PFS = pay for success.

Note: The total sample is 129 respondents.

Source: 2018 Annual Partnership Survey

1 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

II. Progress at Sites in the

Transaction Structuring Phase

This section reviews the progress at each of the five sites that were engaged in transaction structuring during 2018. It then applies a cross-site lens to the feasibility and transaction structuring work.

Anchorage, Alaska

In early 2018, the project—led by the United Way of Anchorage, the Anchorage Mayor’s Office, and Agnew Beck, a local consulting firm—completed the feasibility analysis phase, which focused primarily on identifying the target population and the costs associated with this population across multiple public systems.

Partners used data from the Alaska Department of Corrections and the Anchorage Homeless Management Information System (HMIS) to identify 330 potential participants for the PFS project.

Those participants were identified using targeting criteria that included multiple encounters with both the Alaska Department of Corrections and homeless service providers. Accessing the data during the feasibility analysis phase was challenging. For example, partners never resolved the consent issues involved with accessing homelessness data for the Matanuska-Susitna Borough, a community neighboring Anchorage that was a partner in the application for the PFS project. Because these data were not available and the community was not represented in the feasibility findings, the borough decided not to move forward as a project partner.

Moving out of the feasibility analysis phase, the biggest remaining question (and point of contention among partners) was how potential participants who met the eligibility criteria would be identified during project implementation. Options discussed included using the homeless services coordinated entry system, referrals generated through homeless service provider outreach, or a list of names generated by administrative data, such as the names of people who encountered the criminal justice system. Though partners agreed that targeting a reentry population was important, some partners were concerned that project feasibility could be affected by reforms intended to increase H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

1 7

diversion and reduce incarceration, 9 which could leave many in the intended target population ineligible for PSH if arrest history were a central eligibility criterion.

To examine associated costs for the target population and potential cost savings from supportive housing, the project used data from national and statewide studies of supportive housing, as well as cost data from supportive housing providers and Medicaid. Most partners were primarily interested in identifying potential cost savings that could fund the project’s success payments, perceiving such savings as the determining factor for project feasibility. Although partners agreed that the feasibility analysis did reveal potential cost savings from the project, not all partners were convinced that those savings would be sufficient to cover project costs. Furthermore, a few partners disagreed with some of the cost findings in the feasibility analysis. The Alaska Department of Health and Social Services disagreed with the way Medicaid savings were presented; Alaska is a fee-for-service state, so if the population used fewer Medicaid-eligible services, there would be no meaningful savings. Medicaid revenue in the state would decrease as a result. Additional y, the service provider who supplied data on the costs of supportive housing services found the cost information incomplete because it did not account for shortfalls in services or insufficient staffing to meet participant needs. For all these reasons, the argument for moving forward with the project shifted from one of cost savings to one of securing better results for the same outlay of resources.

Despite these concerns, in May 2018, partners found the project to be feasible overall. Moving into transaction structuring, United Way and the Anchorage Mayor’s Office remained core partners (despite transitions of lead staff members at United Way). New partners were brought in to lead working groups to focus on three elements that had not been addressed during the feasibility phase: implementation, finance, and evaluation.

The Corporation for Supportive Housing (CSH) was brought in as a consultant to focus on the design and implementation of the supportive housing program. As part of an analysis of local capacity to provide supportive housing, CSH interviewed local providers and stakeholders to inform a Request for Qualifications (RFQ) for housing and service providers launched by the site. The request brought in proposals from 11 organizations that partners planned to interview. Despite the response, at the time of the site visit, partners anticipated that none of the agencies would have the capacity to 9 In July 2016, Governor Bil Walker signed Senate Bil 91 into law to reduce the number of people incarcerated by 13 percent by 2024 and save $380 mil ion, with reinvestments of $98.8 mil ion into behavioral health care (including substance use disorder and mental health care treatment in both the community and correctional facilities), pretrial assessments and supervision, and reentry supports. This law also created the Pretrial Enforcement Division of the Alaska Department of Corrections. Diversion might mean that some people who previously would have been screened into the project would now be left out.

1 8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

serve all the 100 to 250 estimated participants, meaning col aboration among providers would be necessary. There were also questions about the region’s ability to supply a skilled workforce for service providers to hire sufficient staff, as well as doubts about whether organizations had the skills and expertise to provide Housing First services for a highly vulnerable population. Few of the organizations were supportive housing providers at that point, so partners planned to launch training and technical assistance for selected providers before implementation. Because project partners intended to provide intensive case management services along with ongoing supervision from the Alaska Department of Corrections. This brought on an additional concern about how corrections policies would blend with Housing First principles. Corrections policies often require sobriety and participation in services while Housing First principles explicitly do not place such requirements on residents.

For housing resources, partners anticipated using a mix of scattered-site and project-based housing, subsidized through turnover vouchers provided by the Alaska Housing Finance Corporation.

The corporation expressed some concern about the PFS model and whether the project would take limited housing resources away from other populations. Other concerns related to housing included the projected pace of enrol ment and whether the project could meet enrol ment targets by waiting for vouchers to turn over or become available in cases where the previous resident is no longer using the voucher.

To tackle the financial aspects of transaction structuring, United Way applied for and received a technical assistance grant from Social Finance, a financial intermediary. Social Finance led partners on issues concerning end payors, investors, success payments, and the project’s financial model. By September 2018, project partners were working to introduce the project to potential end payors and investors. Given the complications of Medicaid financing, the Alaska Department of Health and Social Services communicated early on that it could not be an end payor for the project. The Anchorage Mayor’s Office verbal y committed to be an end payor, but contract negotiations had not begun as of the end of 2018. The Alaska Department of Corrections leadership was also interested in being an end payor. The agency’s ability to serve in that role, however, was in question because of the upcoming gubernatorial election and the agency’s future priorities. Additional potential end payors included local hospitals, such as Providence Alaska Medical Center.

By fall of 2018, partners had approached local philanthropic groups about being investors, particularly philanthropies that had funded similar programs in the past, such as Rasmuson Foundation and the Alaska Mental Health Trust Authority. These potential investors were new to the PFS model and felt unsure whether they were interested in receiving a repayment as part of a PFS

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

1 9

contract. At that time, no discussions had begun around success payments or contract specifics. Both funders were considering requests from project partners, however, to fund a ramp-up or pilot period for project implementation.

An evaluation-planning working group was also launched, chaired by the site’s lead evaluator, Marny Rivera from Northwest Professional Consortium, Inc. This group met regularly in 2018 to draft the evaluation plan, the Institutional Review Board proposal, and data-sharing agreements. Although the evaluation methodology was not yet clear in fall 2018, project partners anticipated that different end payors and investors would be interested in different success metrics.

Statewide elections in November 2018 and uncertainty about the stability of key decisionmakers and stakeholders were impediments to transaction structuring. Ultimately, voters elected a new governor, leading to significant turnover at the state agencies that project partners had been working with, including the Alaska Department of Corrections and Department of Health and Social Services (DeMarban and Zak, 2018; Brooks, 2018). Project partners were working to maintain their ties to these agencies throughout the end of 2018 as new staff came on board.

A 7.1 magnitude earthquake that struck Anchorage on November 30, 2018, significantly damaged the area and caused a major disruption to project work. A federal disaster was declared, and partners took time to address the damage in their community ( Anchorage Daily News, 2018). For example, tenants of the largest Housing First site in Anchorage had to be moved out while the property was examined, and the largest soup kitchen lost power. Almost all data on which this report is based were col ected before the earthquake, however.

Moving into 2019, project partners plan to launch a pilot program for 60 participants, funded through direct grants from the Rasmuson Foundation and the Alaska Mental Health Trust Authority, which are not yet tied to performance metrics or repayments.

Austin/Travis County, Texas

In early 2016, Ending Community Homeless Coalition (henceforth ECHO, the grantee and project intermediary) and CSH (the PSH advisor) completed the project’s feasibility report, which identified the 500 costliest utilizers of criminal justice, homeless, and healthcare systems, finding a PFS project to be feasible. The site then proceeded to transaction structuring, and Social Finance joined the team as the financial intermediary in early 2017. In 2017, the project was primarily focused on structuring a pilot program and engaging potential end payors. In 2018, partners launched the pilot program and

2 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

focused on refining the program model in preparation for full implementation. That included issuing RFQs for service providers and evaluators, confirming commitments from end payors, and finalizing outcome metrics and success payments. Since the feasibility report was finalized, project partners have refined the previous data analysis by narrowing in on the 250 people experiencing a pattern of homelessness who are the highest utilizers of healthcare services and the criminal justice system.

The pilot program launched in May 2018 with funding from St. David’s Foundation—a charitable foundation focused on improving health in Central Texas—and aimed to place two dozen eligible clients in scattered-site housing. With Caritas—a local nonprofit organization—as the supportive services provider and ECHO managing outreach and housing placement, the pilot reached ful enrollment (24 clients) by the end of 2018. The pilot was intended to test assumptions made during the feasibility analysis and transaction structuring phases regarding recruitment, referrals, and service delivery for the target population. Partners reported several early lessons that will inform program-model changes before full implementation, including the need to clarify roles and responsibilities among service providers and the importance of refining how and when participants are engaged during the referral process.

In July 2018, while the pilot was ongoing, project partners issued an RFQ for service providers interested in being part of the ful implementation program. The RFQ solicited responses from organizations that could provide housing, services, or both, and project partners anticipated selecting more than one service provider. Responses were due in August 2018, and interviews were conducted in September. Partners were considering different case management models for the project, including intensive case management and assertive community treatment. Four organizations responded to the RFQ, and project leaders intended for the selected provider(s) to play an important role in finalizing the service model.

Project partners also invested significant time in securing end payors, in part because of the potential payors’ unfamiliarity with and uncertainty about the PFS model. The site identified five end payors across three outcome areas (housing, criminal justice, and health), each with its reason for involvement. The potential end payors were the City of Austin, Travis County, Episcopal Health Foundation, Central Health, and Community Care Col aborative. Although, by the end of 2018, only the City of Austin had set aside the committed project funds. Most partners considered the complicated end-payor structure to be the most chal enging hurdle in finalizing the PFS contract.

Although the partners focused on confirming end payor commitments, their work also included finalizing outcome metrics, looking at the payment cap for each outcome metric, and confirming H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

2 1

which end payor was committed to paying for which outcomes. Once end payors are confirmed, Social Finance wil work to make recommendations on specific outcome metrics and success payments.

Beyond these main areas of work, by the end of 2018, the site was in early-stage conversations with potential investors, though no decisions or commitments had been made. Partners expected to engage philanthropic and private investors, but they saw Austin’s relatively young philanthropic community as a potential chal enge.

Project partners also issued an RFQ for an evaluator in May 2018, but they did not find the right partner at that time. Partners made smal adjustments to the RFQ and rereleased it in July 2018 with broader distribution, and seven organizations responded. Partners conducted four interviews, and Abt Associates was selected based on its combination of evaluation experience, expertise, and flexibility.

An evaluation contract was signed in October 2018. Although partners did not have a specific evaluation design in mind, a randomized control ed trial (RCT) was considered too expensive and not a good fit for the project. Partners anticipated that a process study would be an important component of the evaluation.

Lane County, Oregon

Led by Third Sector Capital Partners as the intermediary, and with primary support from Sponsors, Inc.

(the service provider) and Homes for Good (the housing provider), the Lane County project completed the feasibility analysis in May 2018. Project partners operated a pilot program from April 2017

through September 2018 (during the feasibility analysis and transaction structuring phases); they successful y housed 92 people in PSH at a new 54-unit affordable housing complex called The Oaks at 14th (The Oaks). 10 Substantial unit turnover meant that al of the 92 people considered housed during the pilot were placed in The Oaks. The pilot not only served to demonstrate that project partners could meet agreed-upon success metrics, but it also helped to refine the Housing First program model, determine enrollment pathways for participants, to inform the expected pace of enrol ment and evaluation design, and to give potential funders confidence that the project could be successful y 10 “The Oaks,” Sponsors, Inc., https://sponsorsinc.org/tag/the-oaks/.

2 2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

implemented. Then, informed by a robust pilot program, the project moved quickly through transaction structuring and launched full program implementation in October 2018. 11

The completed feasibility analysis leveraged data from the Oregon Department of Corrections, the local Homeless Management Information System, and Trillium, a health insurer and Medicaid provider for Lane County. The feasibility analysis identified a sufficient population in need of PSH and found high capacity among housing and services providers. Because this project has begun implementation, the model being used in Lane County is described in more detail than those being used in sites stil in the transaction structuring phase.

Because of “off-cycle” timing in Oregon’s traditional biennial budget period, a traditional government end payor for the first year of the implementation phase was not secured. 12 The project began implementation with a loan from a local community foundation to be repaid by Sponsors, Inc.

(the service provider), regardless of the outcomes achieved. If the project performance benchmarks are met, Sponsors, Inc. wil be repaid through HUD’s demonstration grant. Therefore, Sponsors, Inc.

took on the financial risk linked to project performance. Project partners chose to frame the financial argument for the project around the social value of better outcomes for the target population and the Sponsors, Inc. track record of meeting performance benchmarks.

Potential end payors at the state level were engaged with the project since 2017 to inform the program design, define successful benchmarks for various outcomes, and learn about direct outcomes contracting with service providers. Because traditional PFS financing (funded by third-party investors to be repaid by the government) was determined to be too complex, and because there were concerns about time and expenses, all project stakeholders and state partners supported a direct funding arrangement for services to be contingent on outcome achievement. Project partners approached the Oregon Community Foundation (OCF) with the project plan and with evidence of outcomes from the pilot period. The foundation had just created an impact fund with a priority for housing projects and made a direct loan to Sponsors, Inc. to launch implementation. Sponsors, Inc. is ultimately responsible for repaying the loan, making this arrangement much like a program-related investment, a funding tool used by many foundations. Separate from the OCF loan, Sponsors, Inc. will draw down outcome payments from HUD if the project meets its agreed-upon performance benchmarks. Sponsors, Inc. wil use the success payments from HUD to repay the loan to OCF. If the 11 “The Way Home: Lane County Reentry Collaborative,” Sponsors, October 29, 2018,

https://www.thirdsectorcap.org/wp-content/uploads/2018/11/The-Way-Home-Press-Release.pdf.

12 The project launched in September 2018. The upcoming state’s budget cycle commences in July 2019 through June 2021.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

2 3

project does not meet performance benchmarks, Sponsors, Inc. shoulders all the financial risk and must still repay the OCF loan.

Stakeholders saw the OCF loan and the project’s current financial model as bridge funding for the first 10 months of the project until the state could initiate performance-based funding at the start of the new biennium in July 2019. As of late 2018, project partners were engaged in negotiations with the Oregon Criminal Justice Commission, state budget office representatives, and legislators regarding ongoing funding beginning in Year 2 of the project. Partners aimed to have the project included as a Policy Option Package in the governor’s proposed budget for the fiscal year 2019–2021.

Based on the pilot performance, project partners decided payment outcomes would be housing stability and the absence of incarceration for a new felony conviction within 1 year. Housing stability is defined by a person remaining in stable housing—program housing or otherwise—for 1 year. Success payment will be made at the client level and will be assessed every 6 months. Payment benchmarks were set so that Sponsors, Inc. could be ful y reimbursed for their repayment of the loan if 85 percent of participants remained stably housed 6 months after enrolling in the project, 78 percent remained so 12 months after enrol ment, and 3 percent or less of participants were incarcerated for a new felony conviction 12 months after project enrollment. The project evaluator, Mark Eddy of New York University, will determine whether these outcomes are met before Sponsors, Inc. draws down HUD

end payment funds to repay the OCF loan.

Because the evaluator had evaluated other Sponsors, Inc. programs and had participated in PFS

project planning meetings since 2017, evaluation planning was completed throughout the feasibility analysis and transaction structuring phases. Although some key partners would have preferred the rigorous evidence produced by a random assignment study, the project wil use a natural experiment.

The study wil use a comparison group created by the fluctuations in housing availability when people become eligible for enrol ment and through setting an enrol ment window of 6 months post-release.

If housing is not available within 6 months of people becoming eligible, they will become part of the comparison group. Although they may receive housing through other organizations or different opportunities from Sponsors, Inc., they will no longer be eligible for housing through the PFS project.

The evaluator has worked with the project since the feasibility analysis phase, and it is constructing the comparison group during project enrollment. The evaluation will follow each participant for three years through administrative data and interviews. Follow-up interviews with all study participants, including comparison group members who may not maintain ties to Sponsors, Inc., will require significant effort. At the end of 2018, partners were seeking funding in addition to what could be paid from the HUD-DOJ funding to pay for the evaluation fully.

2 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Montgomery and Prince George’s Counties, Maryland

Led by AIR (the project intermediary), project partners completed the feasibility analysis phase in early 2018. Project partners stil had questions about the size and needs of the target population, however, and whether supportive housing was the right intervention for the potential participants, the counties, or the PFS financing mechanism.

In Montgomery County and Prince George’s County, the feasibility analysis focused primarily on sharing knowledge about the PFS process with stakeholders and securing stakeholder buy-in.

However, by the end of 2018, many partners still found the complexity of the PFS model to be a barrier to moving forward. The feasibility analysis phase also included data analysis to identify whether both counties had a sufficient target population to warrant a PFS-funded intervention. One of the major accomplishments of this phase was the establishment of a data-sharing agreement among AIR and the various justice agencies, health departments, and homelessness departments in both counties. These agreements built the counties’ data-sharing capacity. For example, the Prince George’s County Department of Corrections was previously unable to identify how many people booked into the jail were also experiencing homelessness or substance use disorders. The feasibility report concluded that at least 212 people were eligible to enroll in the PFS project, based on the criterion that they had more than one jail stay in 2017. Because of the PFS feasibility analysis, along with the two counties’ efforts under the Data-Driven Justice Initiative, 13 this type of analysis became possible. The feasibility analysis did not convince al project partners that a substantial target population needed supportive housing, however. At least one stakeholder in Montgomery County expressed concern that other efforts to decrease chronic homelessness would keep the service population below the level anticipated in the feasibility analysis.

In addition to having questions about the size of the target population, project partners expected some other project implementation chal enges. Some project partners in Prince George’s County were concerned that the focus on supportive housing might limit the opportunities to use transitional housing for people who have immediate housing needs and/or need support to become ready for PSH. Some Montgomery County partners raised questions about whether the PFS model would optimize the delivery of housing and supportive services and about whether services could be scaled back or intensified to accommodate clients’ needs. In addition, partners in both counties had 13 The Data-Driven Justice Initiative, launched by the White House in 2016 and led by the National Association of Counties, integrates data to drive strategies that reduce reliance on criminal justice system contact and incarceration for low-level offenders with mental il nesses. Al demonstration sites except for Anchorage participated.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

2 5

concerns that were specific to working with a reentry population, in particular about the feasibility of finding housing in the community for people with criminal histories. Relatedly, opinions were mixed on the likelihood of finding service providers that were trained in service-delivery to people involved with the criminal justice system and had the capacity to serve the high needs of this population. Some stakeholders in both counties expressed concern that this would be difficult. In contrast, at least one partner in Prince George’s County expressed confidence that the RFP process would identify providers skilled in working with a reentry population. By the end of 2018, the two counties were discussing the idea of finding a single supportive housing service provider to serve both counties.

Even as the project moved into the transaction structuring phase, partners had other concerns about the financial feasibility of implementing PSH with PFS. Resolving these concerns was particularly difficult given the extensive political turnover that was expected to follow the November 2018

elections.

The partners’ concerns about the financial feasibility of the PFS project included:



Required changes in procurement laws that would al ow Montgomery County to take on debt and questions about the political will for such changes and day-to-day capacity to operationalize them.



Opportunity costs of earmarking Prince George’s County funds for future success payments, such that those funds could not be used to address other issues.



Potential adverse effects on Prince George’s County’s triple-A bond rating.



Costs of the PFS process and whether Prince George’s County should instead finance the project directly.



Doubts about whether the project would yield cost savings for Prince George’s County.

Addressing these concerns was difficult, as the project experienced significant changes in leadership across both counties’ agencies after the November 2018 elections. Partner capacity to participate in PFS planning activities in the last quarter of 2018 was reduced due to these elections. At the end of 2018, agencies in both counties were waiting for newly elected officials to appoint and hire new staff members, who would need to be briefed on the PFS project and their roles as key decisionmakers. This turnover caused delays to transaction structuring work and required renewed stakeholder engagement work with the new leadership.

2 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Rhode Island

Project partners completed the feasibility analysis phase at the end of 2017. Although the findings from the feasibility analysis were informative about the target population’s service use across systems, partners had different interpretations of the cost savings expected to be generated by the project.

Social Finance, operating as the project intermediary while the Rhode Island Coalition for the Homeless (RICH) increased staff capacity, worked on the feasibility analysis throughout 2017, with consultation and input from the Rhode Island Office of the Governor and a steering committee of key stakeholders. Social Finance produced a draft feasibility analysis report in July 2017 that used matched Homeless Management Information System and Rhode Island Department of Corrections data, as well as summary Medicaid data, to describe the target population. Although there was local support for expanding the supply of PSH units and local awareness of the needs of a homeless population that is involved with the criminal justice system before the feasibility analysis, the project’s efforts to identify and link data contributed to a better understanding of the population and its service use and costs across multiple systems.

Stakeholders interpreted the feasibility analysis findings on cost savings differently, however.

Some stakeholders were reluctant to proceed without a clear identification of potential cost savings, suggesting that the analysis should have included a model that partners could use to estimate cost savings as the project moved forward. Other stakeholders did not consider cost savings an important factor in project feasibility, believing any such savings would be neither significant nor guaranteed.

The potential cost savings to the Rhode Island Department of Corrections from implementing PSH are only the marginal costs of a night of incarceration, and they do not include fixed and capital costs to the department. The analysis found that the marginal cost of a night in a Rhode Island Department of Corrections facility, which includes costs like laundry and food, was roughly $15, yielding limited cost savings even if the project were successful. Some partners argued the project would not produce any cashable savings, but it might lead to more effective allocation of existing resources. Later, in the transaction structuring phase, partners planned to update the feasibility analysis and use the data to further refine eligibility criteria, outcomes, and success payments for the project.

Project partners spent 2018 stal ed in transaction structuring because of difficulty identifying a mechanism the state could eventual y use to pay for project outcomes. Because of this barrier, little was decided about other project features, such as the financial model, supportive housing program, or evaluation design.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

2 7

The state budget was enacted in June 2018 without the plan proposed by the PFS project. Project partners had hoped to see the creation of a government performance-improvement fund to hold funds that could be used for success payments, either from new funding or real ocated funding. It was the second year that the state budget had passed without such a fund, despite efforts from the governor’s office. Project partners were surprised the 2018 budget did not include the proposed fund, but they attributed the absence of the fund to a combination of insufficient advocacy with legislators and political deal-making at the expense of the fund. In addition, contested elections in 2018

provided considerable political uncertainty about key decisionmakers and stakeholders, and the gubernatorial primary and general election limited the capacity of the governor’s office staff to develop alternative options. Given these barriers, the project had not proceeded to considerations such as outcome measurement, success payments, or the supportive housing program model.

Project partners then began exploring other ways to secure the state funds needed for outcome payments. The option preferred by stakeholders was to create a third-party escrow fund that would be managed by the Rhode Island Commerce Corporation, a quasi-public entity, and funded (initially) by reallocating existing funds. Initially, existing funds could come from the Department of Commerce, which includes the Office of Housing and Community Development, and other housing-related funds.

By the end of 2018, no decision had been made, although leadership within the Department of Commerce was open to exploring both the opportunity of and barriers to an escrow fund. Project partners decided to resolve the question of how the state would fund outcome payments and answer other project-design questions before approaching potential investors.

Based on preliminary discussions about the project’s supportive housing model, partners expected that it would probably use scattered-site housing and an assertive community treatment service model. As of this writing, the state was awaiting a decision on its application for a 1115 waiver, which would allow service providers to bill Medicaid for services such as care coordination and case management.

If partners identify a viable mechanism for success payments in early 2019, the project will relaunch the various working groups to refine eligibility criteria, finalize the service model and outcome metrics, and select key partners (such as a service provider and evaluator), to move into the project implementation phase in 2020.

2 8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

III. Cross-Site Findings on Feasibility

Analysis and Transaction Structuring

During the second project year, 2018, most sites were in the transaction structuring phase. Much of the work they were doing in this phase was to establish a feasible financial structure—including pricing outcomes and securing investors and end payors—and to refine the project design—including programmatic elements like eligibility criteria, the service model, and the housing source(s).

Ultimately, to reach a successful transaction and launch a PFS project, the following five crucial tasks need to be accomplished:



Build collaborative partnerships among criminal justice, housing, and health agencies, as well as with financial partners. This also requires considerable work educating stakeholders about the target population, PSH, and PFS.



Identify the target population.



Establish the financial feasibility of PFS.



Establish the feasibility of implementing PSH.



Finalize the evaluation.

Exhibit 7 shows the five crucial tasks, some major subtasks, and the demonstration grant phase sites have typically conducted them in. Our discussion of cross-site findings is organized around these five crucial tasks.

As exhibit 7 shows, there was considerable overlap in the work that sites conducted during the feasibility analysis and transaction structuring phases. During transaction structuring, some stakeholders said they felt as though they were still in the feasibility analysis phase.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

2 9

EXHIBIT 7

Key Tasks for Feasibility Analysis and Transaction Structuring and Their Typical Pay for Success Phase

Feasibility Transaction

Pilot

PFS Contract

Analysis

Structuring

Period

Implementation

Establish partnerships

Educate stakeholders about PSH and PFS

X

X

Identify the target population

Confirm the need

X

Confirm the population size

X

X

Define eligibility criteria

X

X

X

X

Establish (financial) feasibility of PFS

Assess costs and possible savings

X

X

Find non-PFS funding

X

X

Secure end payors

X

X

Secure investors

X

Define success payments

X

Establish the feasibility of implementing

PSH

Confirm capacity for services

X

X

Assess housing capacity using Housing First

X

X

X

Finalize evaluation plan

Establish success metrics

X

X

Create an evaluation design

X

X

X

Involve evaluators

X

X

X

PFS = pay for success. PSH = permanent supportive housing.

Note: Non-PFS funding includes funding for Medicaid, housing vouchers, and philanthropy.

Feasibility Reports

We turn now to look more specifically at what sites included in their feasibility reports, which they were required to produce to progress to the transaction structuring phase. 14 The reports represent the culmination of the demonstration’s feasibility analysis phase and include conclusions about whether a PFS-PSH project is feasible, as well as including recommendations for moving forward. The reports were submitted to HUD and the local operating, steering, and executive committees.

By the end of 2017, three of six sites had completed and shared feasibility reports (Austin/Travis County, Pima County, and Rhode Island). By the end of 2018, the remaining sites (Anchorage, Lane County, and Montgomery and Prince George’s Counties) had shared their feasibility reports with the 14 This did not apply to Los Angeles County, which did not receive funding from the demonstration for a feasibility analysis; its assessment preceded the demonstration.

3 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Urban research team. Exhibit 8 describes what the sites included in their feasibility reports. As the exhibit shows, most reports identified the basic problem, identified a target population, and began the work of assessing the financial feasibility of a PFS project. (The financial assumptions of those reports are discussed in this report’s “Establishing Financial Feasibility of PFS” following section.) The reports also began the work of assessing the feasibility of implementing PSH by examining the housing resource landscape. Generally, sites made little progress defining the service intervention and the local capacity to implement it. Also, few reports provided details around success metrics, evaluation considerations, or investors.

During the transaction structuring phase, however, sites often found the need to revise prior assumptions and update previous analyses conducted during the feasibility analysis phase after new or different end payors became engaged and as sites concretized their project plans. Some of this revision work continues into a pilot or ramp-up phase (which tests implementation assumptions), and even into full PSH implementation if assumptions still need to be updated. This revision work left some stakeholders frustrated. One said the transaction structuring phase was less structured than the feasibility analysis phase because nontechnical conversations and decisions that needed to be resolved to advance the project occurred in a nonlinear way often and required “seemingly endless”

negotiations. Those feasibility assumptions often needed revision during transaction structuring led several stakeholders to wonder what the feasibility analyses and reports had accomplished.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3 1

EXHIBIT 8

Overview of Pay for Success Feasibility Analysis Components Austin/

Travis

Lane

Montgomery and

Pima

Anchorage,

County,

County,

Prince George’s

County,

Rhode

AK

TX

OR

Counties, MD

AZ

Island

Identify and define the problem

Describe the current

service environment

X

X

X

X

X

Describe core

components of PSH

X

X

X

X

X

X

Conduct a literature

review of PSH

X

X

X

X

X

Identify gaps in the

availability of PSH

X

X

X

X

X

Define the target population

Identify high-cost

utilizers

X

X

X

X

X

X

Assess service use of

the target

population

X

X

X

X

X

X

Monetize (assign

dollar values) to

public system service

use

X

X

X

X

X

Assess financial feasibility

Quantify benefit to

different public

systems

X

X

X

X

X

Quantify the costs of

the intervention

X

X

X

X

X

X

Quantify other PFS

transaction costs

X

Conduct a cost-

benefit analysis

X

X

X

X

X

Provide an economic

model

X

X

X

X

Defining the PSH intervention and assessing its feasibility Identify housing

resources

X

X

X

X

X

Describe core

components of the

service model

X

X

X

X

Assess the current

service provider

landscape and

capacity

X

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3 2

Austin/

Travis

Lane

Montgomery and

Pima

Anchorage,

County,

County,

Prince George’s

County,

Rhode

AK

TX

OR

Counties, MD

AZ

Island

Determine program

referral pathways

X

Establish success metrics and evaluation

Suggest specific

success metrics

X

X

X

X

Suggest other

outcomes

X

Specify how success

will be measured

(evaluation design)

X

X

Provide recommendations and next steps

Describe work to be

completed at future

phases (transaction

structuring)

X

X

X

X

X

X

Identify

organizations that

should be part of a

working group

X

X

X

Identify investors

X

PFS = pay for success. PSH = permanent supportive housing.

A. Building PFS Partnerships

Educating stakeholders and building col aborative partnerships were essential parts of the feasibility analysis and transaction structuring phases. Educating stakeholders about PFS took a significant amount of time, especially during the feasibility analysis phase. One of the main benefits of the feasibility analysis was that it allowed stakeholders to learn about how the PFS mechanism would work for each project. Nevertheless, educating partners about PFS often continued during transaction structuring. In the 2018 Annual Partnership Survey—fielded after sites had progressed to the transaction structuring phase—stakeholders were more likely to agree that partners had a shared understanding that PSH is important for the target population than to agree that partners had a shared understanding of PFS and its role in supporting PSH.

Educating stakeholders was just one part of the work needed to build strong PFS partnerships.

Although each project developed different partnership structures and communication styles, most projects had a primary partner (or set of partners) who operated as the hub for PFS work by reaching H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3 3

out to other partners and stakeholders for input on various activities. This partner coordination and engagement also varied based on the PFS phase. In the transaction structuring phase, for example, the work was less structured for most projects, with many disparate activities happening at the same time. By contrast, in the feasibility analysis phase, many projects took a step-by-step approach to the analysis. Moving from the feasibility analysis phase to the transaction structuring phase tended to involve new stakeholders, different decision points, and a broader set of partner activities. For example, a complete transaction must include commitments from end payors as well as investors.

Bringing evaluators and service providers into the process early al owed them to serve as thought partners throughout; they could sometimes help with problems accessing data, assist with analyses, and provide feedback on how the proposed eligibility criteria might affect the evaluation or the delivery of services. In one site, however, attempting to do this contributed to delays while the site worked to identify these partners. Also, the input from these partners was not always extensive.

Among sites that identified evaluators early (Anchorage, Lane County, and Montgomery and Prince George’s Counties), evaluators were involved during the early phases to varying degrees; some evaluators engaged early, but others were not truly engaged until late in transaction structuring. For service providers, the work became most intensive during full project implementation (and pilot periods), when staff members often had full-time roles on the PFS project.

End payors and investors sometimes played a role in influencing decisions about the final target population and service model after the main project partners tweaked criteria to entice them to join.

For example, during transaction structuring, potential end payors in Austin/Travis County helped shape the eligibility criteria so that the project served populations of interest to them. Similarly, key project partners in Rhode Island indicated they would delay making final decisions about eligibility criteria until they had identified end payors and decided on payment structures.

Lastly, government stakeholders’ roles and levels of engagement differed across projects, serving as end payors, data partners, subject matter experts, or local leaders in different settings.

What fol ows are summaries of the basic partnership structures, leadership, and communications styles of the PFS projects:



In Anchorage, the United Way coordinated project work during the feasibility analysis and transaction structuring phases. Key staff in the Anchorage Mayor’s Office also played a coordinating role during periods of staff turnover at the United Way. Although Agnew Beck, a local consulting agency, was an important partner during the feasibility analysis phase, the Corporation for Supportive Housing and Social Finance became more important partners as

3 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

the project moved into transaction structuring. Other stakeholders and partners were engaged through working groups, email communications, and verbal updates. Three working groups were established as part of the transaction structuring phase to focus on finance, implementation, and evaluation.



In Austin/Travis County, ECHO and CSH led the feasibility analysis phase, and Social Finance joined as a key partner during the transaction structuring phase. Although those three organizations collaborated across all project tasks, ECHO took the lead on local leadership, rallying political buy-in and leading communication with project stakeholders; CSH advised the project based on its national experience with supportive housing and PFS; and Social Finance focused on resolving legal and technical questions and on structuring the PFS

contract. ECHO and Social Finance managed most project communication outside of these three partners, communication that happened mostly in one-on-one meetings, with limited engagement across the broader group of partners.



In Lane County, the primary project partners had long-standing relationships, particularly Sponsors, Inc., Homes for Good, and the county’s parole and probation division. Key local and state-level partners met monthly from the beginning as a steering committee that facilitated data access, addressed public concerns about the project, and generated the political will to move forward. In particular, the executive director of Sponsors, Inc. (the lead service provider) had strong prior relationships with key stakeholders tapped for project financing, including the Oregon Department of Corrections, the Criminal Justice Commission, and the Oregon Community Foundation.



In Los Angeles County, CSH coordinated all activities associated with the project implementation phase. Monthly service provider meetings created opportunities for partners to col aborate on project operations, and quarterly executive-level meetings convened partners to review quarterly outcomes and success payments.



In Maryland, Montgomery County and Prince George’s County both had steering and executive committees that met monthly and quarterly, respectively, with the intermediary, AIR, facilitating the meetings. In addition, the counties formed four working groups for the transaction structuring phase focused on data and evaluation, identifying investors, contracting, and the financing structure. The project’s evaluator, ICF, facilitated a data and evaluation working group.



In Rhode Island, Social Finance led project work by completing the feasibility analysis, convening the stakeholders engaged in the steering committee and working groups, and H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3 5

educating other outside partners. RICH, the federal grantee, was unable to do this because of transitions in its senior leadership and lack of capacity during demonstration Years 1 and 2.

With a new executive director and more staff dedicated to the project, RICH anticipated taking on a more significant role with the project in the future. The governor’s office was also a key decisionmaker for the project.

Collaboration and Partnership

Stakeholders widely mentioned during interviews that increased collaboration was a benefit of the PFS process. On the one hand, some stakeholders believed that strong collaboration was already ongoing and did not attribute it to the PFS project. On the other hand, many stakeholders believed the process of constructing a PFS project led to new and improved relationships that would outlive the project. Stakeholders also reported that the PFS project brought together partners from different sectors, especially criminal justice, who do not typically engage in much col aboration. Stakeholders in several sites also mentioned that the PFS process increased awareness of the target population and educated people about PSH. Additionally, stakeholders considered increased collaboration between the public and private sectors and between nonprofit service providers and possible investors to be beneficial.

High levels of collaboration among project stakeholders were general y confirmed through the 2018 Annual Partnership Survey; respondents reported that partners had common understandings about the target population and the need for PSH, as wel as a shared vision for the project (exhibit 9).

EXHIBIT 9

Survey Responses about Collaboration

Mean

Don’t

Not

Agreement

Know

Applicable

(Scale 1 to 4)

(%)

(%)

To what extent would you agree with the following

statements about your PFS project?

I have a clear understanding of my organization’s roles and responsibilities within the PFS project

3.7

0

1

I have a clear understanding of other organizations’ roles and responsibilities within the PFS project

3.4

0

1

Would you agree with the following statements about

the PFS partnership?

Partners share a common vision

3.5

10

0

3 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Partners have a common understanding of PFS and its role

in supporting PSH

3.3

6

0

Partners have a shared understanding that PSH is important for the target population

3.8

4

0

Partners are willing to make changes to achieve shared

goals

3.4

10

0

Partners work wel together as a group

3.5

4

0

There are organizations in the community with expertise

and experience in providing supportive housing for the

target population

3.7

1

0

The PFS project is bringing local partners together across sectors to improve results for the target population

3.5

6

1

Partners are working to educate policymakers and local

elected officials about the need for supportive housing for a chronically homeless jail reentry population

3.5

16

4

The community is committed to reducing barriers to

housing for the target population, including accepting

housing applications for individuals with criminal justice histories

3.1

16

3

PFS = pay for success. PSH = permanent supportive housing.

Notes: Answers range from 1 (disagree completely) to 4 (agree completely). Means are calculated across al 71 respondents.

For a more detailed description of the respondents, see exhibit 6.

Source: 2018 Annual Partnership Survey

In addition, among those respondents who felt they were sufficiently informed (such as respondents who did not select “don’t know”), most believed operational collaboration had increased during the preceding year (exhibit 10), this is especially true of data sharing, as we discuss below.

Data Sharing and Data Use

PSH projects involve multiple sectors, and PFS projects require relevant data to assess the target population and its size and needs and to understand the costs and possible cost savings that PSH

could create. These all require considerable data and data sharing during all phases of the PFS life cycle. This topic is discussed at considerable length in our baseline implementation report (Liberman et al., 2019) and a brief, “Data Use and Challenges in Using Pay for Success to Implement Permanent Supportive Housing” (Gillespie, Batko, and Liberman, 2018). This data sharing is one aspect of the col aboration necessary for and driven by PFS projects. Stakeholder perceptions of this aspect of the col aborative work are shown in exhibit 10.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3 7

EXHIBIT 10

Survey Responses about Data Sharing

Stayed

Don’t

Not

Decreased the Same

Increased

Know Applicable

(%)

(%)

(%)

(%)

(%)

Over the past year, has collaboration

increased or decreased?

Data sharing between organizations

0

13

43

21

0

serving the target population

Number of formal agreements or

0

22

28

30

0

memoranda of understanding between

organizations serving the target

population

Frequency of meetings about effectively

1

13

39

25

0

working with the target population

Number of PFS partner organizations

0

16

14

52

6

represented on the local criminal justice

coordinating committee

PFS = pay for success.

Notes: Answers range from 1 (disagree completely) to 4 (agree completely). Means are calculated across al 71 respondents.

For a more detailed description of the respondents, see exhibit 6.

Source: 2018 Annual Partnership Survey

In several cases, the demonstration motivated sites to improve their data-sharing infrastructure, and some capitalized on other ongoing efforts. For example, Montgomery and Prince George’s Counties engaged in similar work through the Data-Driven Justice Initiative. Some stakeholders stressed that setting up a data-sharing infrastructure and getting data-sharing agreements in place would be a lasting benefit. Many survey respondents reportedly did not know whether data were used to identify the target population, however (exhibit 11). This may be attributed to the amount of time that passed in some sites after the feasibility analyses were conducted (compounded by staff turnover), as well as respondents’ particular organizational roles (see exhibit 6 in section I).

EXHIBIT 11

Survey Responses about Data Use

Don’t

Not

Mean Agreement

Know

Applicable

(Scale 1 to 4)

(%)

(%)

Would you agree with the following statements about

data sharing over the past year?

PFS partners used data to identify the target population

3.6

4.2

1.4

PFS partners have used data to understand the level of

housing and services needed by the target population

3.6

7.0

0

3 8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

PFS = pay for success.

Notes: Answers range from 1 (disagree completely) to 4 (agree completely). Means are calculated across al 71 respondents.

For a more detailed description of the respondents, see exhibit 6.

Source: 2018 Annual Partnership Survey

Time Burden

This collaborative work requires considerable investments of time by a diverse set of stakeholders, especially during the feasibility and transaction structuring phases. Most of the results about the time burdens presented here are drawn from the qualitative data we col ected. A time-cost study data collection began in the fourth quarter of 2018; upcoming evaluation reports wil include those results.

Core PFS team members, including the intermediary and government staff members, spent considerable time on the project. Most of that time was spent retrieving and analyzing data or in meetings. Intermediaries reported that they typically had the equivalent of a ful -time employee dedicated to the project, although the work was sometimes divided among several staff members.

Interviewees reported that key advisors and other members of the core project team spent between 25 and 50 percent of staff time on the project; one interviewee indicated that hiring a ful -time project manager for 1 to 2 years would have been more beneficial. According to interviewees, most government partners, (potential) end payors, and (potential) investors typically spent significantly less than 10 hours a month on the project. Some interviewees indicated, however, that getting the projects off the ground through feasibility analysis and transaction structuring took a significant amount of time for senior-level staff—executive directors, vice presidents, chief executive officers, and other higher-level investors and (potential) end-payor decisionmakers. Investors and end payors also faced periods of intense activity during contracting.

Responses to the Annual Partnership Survey, which asked how many project-related meetings respondents had attended in the past year, confirmed the reports from interviews that most people spent considerable time on the project while others spent relatively little time. About one-half of the survey respondents reported attending between 5 and 10 meetings in the previous year, while one-sixth of respondents said they attended between 1 and 4 meetings. Almost one-third of the respondents attended more than 11 PFS project meetings in the past year (exhibit 12).

EXHIBIT 12

Number of Pay for Success Project Meetings Attended in Past Year Meetings

N

Percentage

0

1

1.4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

3 9

1–4

12

16.9

5–10

35

49.3

11–20

7

9.9

More than 20

16

22.5

Total

71

100.0

Source: 2018 Annual Partnership Survey

B. Defining the Target Population

The second key step toward finalizing a PFS transaction and launching a project is defining a target population. During the feasibility analysis phase, demonstration sites had to identify a target population that met the HUD-DOJ eligibility requirements for a pattern of homelessness, criminal justice involvement, and healthcare use (see exhibit 4 in section I), and that it was of sufficient size.

Most feasibility reports sought to clearly define the nature of the problem and the characteristics and size of the target population. The Lane County feasibility report, for example, indicated a high (50

percent) rate of housing instability among the reentry population, noting constraints on the available interim housing options. Target populations were often defined in response to existing local priorities.

For example, in Austin/Travis County, multiple stakeholders cited homelessness in interviews as a primary local political priority, and PSH capacity had expanded in recent years. Similarly, Montgomery County, Maryland, had recently housed 400 people through an initiative to end chronic homelessness.

Sites conceptualized their target populations differently. The Notice of Funding Availability set a baseline for the target population: people must be homeless, involved with the criminal justice system, and have high needs. The feasibility analysis needed to identify who fit these criteria and decide how the criteria might be further refined given community context. Sites approached this in various ways. Four sites prioritized people they viewed as having the greatest needs: people who were the most vulnerable and at the highest risk of health problems and mortality while homeless (for example, according to the Vulnerability Index and the Service Prioritization Decision Assistance Tool

[VI-SPDAT] survey, which assesses medical vulnerability); another site focused on the highest utilizers across systems.

This variation across sites can be attributed to differences in data availability, methods used to merge and analyze the data across systems, and the stakeholders who were at the table and their particular interests. These variables sometimes meant that the definitions of the target populations changed during the project. For example, at the end of 2018, Rhode Island was still solidifying its criteria, but it had begun thinking early on about how to define “top utilizers”—would it do so

4 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

through HMIS or Medicaid data, or would the project use historical data to predict increases in future use? Austin/Travis County developed a two-step process to guide program enrol ment. First, it would screen for eligibility based on jail bookings, homeless history, and health care use (exhibit 13). It would then prioritize outreach to people with the highest needs based on historical health care and criminal justice system use. In this way, Austin/Travis County started with a general y high-needs population and focused on first serving people who incur the most costs to these systems. In addition, to change over time, the cross-system nature of the projects al ows for stakeholders within sites to hold different opinions about who should be prioritized: Should projects prioritize people who are seen by the homelessness, criminal justice, and healthcare systems as the highest utilizers or those seen as the most vulnerable?

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

4 1

EXHIBIT 13

Target Population Criteria by Site

Site

Criminal justice

Homelessness

“High needs”

Anchorage, AK

Two or more stays in the

Three or more

VI-SPDATa score of 9 or

Department of

encounters with a

higher

Corrections in the last 3

homeless services

years, 1 in the last 12

provider in last 3 years

months

Austin/Travis County, TX

Minimum of one jail day

Meets HUD’s definition

Minimum of one

or jail booking in last

of chronic homelessness

inpatient day or four

year, and a minimum of

emergency department

two jail bookings in the

visits in the past 18

last 3 years

months

Lane County, OR

Released from state

Homeless since most

Determined by CAT-R.

prison into Lane County

recent release from

There is a priority for

with a medium to very

prison; a history of

women, families, or

high risk of recidivating

housing instability as

clients with criminal

as defined by the LS/CMI, defined by the CAT-Rb

histories excluded from

WRNA, and/or STATIC-

other federal housing

99c

options

Los Angeles County, CA

Discharge from County

Homeless at least one

Upper third quadrant on

facility in 30–120 days,

night during the year in

VI-SPDAT, disabling

eligible for diversion

each of the last 3 years

condition

services through the

alternative court

Montgomery and Prince

A score of moderate or

A vulnerability for living

Unmet behavioral or

George’s Counties, MD

high risk to re-offend, as

in unsafe or unstable

somatic health needs:

measured by the Level of housing: measured by VI- measured by multiple Service Inventory and

SPDAT

tools

Risk (LSI-R).

Rhode Island

Frequent utilizers of the Department of Corrections, homeless shelters, and Medicaid services; specific criteria under development

CAT-R = Coordinated Assessment Tool for Reentry. LS/CMI = Level of Service/Case Management Inventory. VI-SPDAT =

Vulnerability Index and the Service Prioritization Decision Assistance Tool. WRNA = Women’s Risk Needs Assessment.

a The VI-SPDAT assigns a score based on a person’s risk of dying on the street.

b CAT-R was customized for the reentry population based on the framework of the National Al iance to End Homelessness’

Coordinated Assessment Tool, which is suited for the chronical y homeless population.

cSTATIC-99 is a risk assessment tool for adult male sexual offenders.

Source: Monthly cal s and interviews with sites

Despite this variation, there seemed to be widespread agreement across demonstration sites that participating organizations, leaders, and other staff members were aware of and committed to addressing the needs of the target population, and can be seen in responses to the 2018 Annual Partnership Survey, shown in exhibit 14.

4 2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

4 3

EXHIBIT 14

Stakeholder Understanding of Target Population

Mean

Don’t

Not

Agreement

Know

Applicable

(Scale 1 to 4)

(%)

(%)

To what extent would you agree with the following

statements about your organization’s role in the

project?

My organization plays an active role in the PFS supportive housing project

3.5

0

1

My organization has a stake in increasing supportive

housing for the target population

3.7

1

4

Leadership in my organization is aware of the issues

surrounding the target population

3.8

0

3

Leaders in my organization are committed to addressing

the needs of the target population

3.7

0

3

Staff members in my organization are aware of the issues

surrounding the target population

3.5

0

1

PFS = pay for success.

Notes: Answers range from 1 (disagree completely) to 4 (agree completely). Means are calculated across al 71 respondents.

For a more detailed description of the respondents, see exhibit 6.

Source: 2018 Annual Partnership Survey

Completing Data Analyses

Sites had to merge multiple datasets to identify the target population, and some sites experienced chal enges completing this task. For example, data-sharing agreements took a lot of time and effort to implement, and turnover in government staff sometimes complicated this. Some sites had to develop workarounds to complete the analyses in time for feasibility reports.

By matching datasets across systems and (in some sites) attaching dollar values to services used, the feasibility analyses helped assure stakeholders that the target population was large enough to merit an intervention (although stakeholders were not universally convinced) and to support a rigorous impact evaluation. In Alaska, Matanuska-Susitna Borough’s data were not included in the feasibility report, which meant that the need for the intervention in its area had not been demonstrated. This contributed to the borough’s decision to drop out of the project. How sites used data in feasibility analyses is discussed extensively in our baseline implementation report (Liberman et al., 2019) as wel as in a brief on data use (Gillespie, Batko, and Liberman, 2018).

For their feasibility analyses and reports, the Anchorage and Rhode Island sites ultimately used aggregate-level data rather than individual-level data for matching across homelessness, criminal

4 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

justice, and healthcare datasets. During the transaction structuring phase, these sites worked to rectify earlier data limitations and refine the analyses. For example, to guide program design and to help structure a transaction, Rhode Island sought to link healthcare use with shelter and corrections data on an individual level.

C. Establishing the Financial Feasibility of Pay for Success The third step toward finalizing the PFS transaction and launching the PFS project is establishing the project’s financial feasibility. For the demonstration, with the target population of high utilizers of criminal justice, healthcare, and homeless services, collaboration across systems to reduce that service burden through PSH could reduce costs considerably. Stakeholders in several sites mentioned that the PFS process increased awareness of the target population’s high-cost burden. In Los Angeles County, where some PSH was implemented before the demonstration, PFS financing seemed to be helping take PSH to a larger scale.

PFS also could help with the “wrong pockets” problem by allowing agencies (for example, Departments of Corrections) as end payors to pay for improved outcomes (and programmatic savings) generated through preventive interventions (such as PSH housing and community-based services) that might typical y fall outside of their agencies’ missions. PFS may also help different units/levels of government (such as cities, counties, and states) to more effectively col aborate financially to realize collective savings. Several stakeholders mentioned that the demonstration had prompted new and more creative ways of thinking about funding and about blending funding streams, all of which could have broader effects beyond the demonstration.

Feasibility Reports’ Estimates of Costs and Possible Savings Most demonstration feasibility reports estimated the status quo costs of serving the target population and program-implementation costs of PSH, in addition to quantifying potential cost savings to different sectors. The estimated costs varied widely across sites, however, which suggests sites used different assumptions and methods to measure and attribute costs, perhaps because they tailored them to the requests of each set of local stakeholders and intermediaries’ preferred processes. Site feasibility reports varied considerably in how detailed they were about assumptions, data sources, and calculations. For example, only two sites (Pima County and Austin/Travis County) estimated H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

4 5

nonprogram PFS costs (for example, transaction structuring costs). Exhibit 15 shows the financial considerations that were included in the sites’ feasibility reports.

4 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

EXHIBIT 15

Permanent Supportive Housing Demonstration Sites’ Target Populations and Financial Feasibility Montgomery and

Austin/Travis

Prince George’s

Pima County,

Anchorage, AK

County, TX

Lane County, OR

Counties, MD

AZ

Rhode Island

Define the target population

Identify high-

269 people with

500 high-cost

Roughly 425 adults At least 212 people

560 people

Frequent utilizers

cost utilizers

high use of

utilizers based on

in the target

eligible to enroll in

who have two

of Department of

Department of

healthcare and jail

population (based

the PFS project, based or more

Corrections facilities,

Corrections,

costs

on housing need,

on more than one jail bookings and

homeless shelters, and

frequent patterns of

criminal justice

stay in 2017

are homeless

Medicaid services

homelessness, and

involvement, and

estimated high need

risk of recidivism)

for social and health

services

Assess financial feasibility

Monetize the

Status quo cost of

Average cost of

Not included in the Status quo per person Average cost

Costs not linked; $13,239

costs of

serving each person $101,218 per

report

costs estimated

of $32,538 per

in HMIS and Department

service use to

per year is $47,413

person per year for

between $78,525

person per

of Corrections costs and

public systems

shelter, jail,

(Prince George’s

year across

$42,710 in Medicaid costs

emergency room,

County) and $87,621

jails, shelters,

per person per year

hospital, and court

(Montgomery

emergency

County)

rooms,

hospitals, and

behavioral

health

Quantify

Expected cost

Expected cost

Not included in the Expected cost

Expected cost

Expected cost avoidance

possible

avoidance across all avoidance is

report

avoidance across all

avoidance is

$5,954 in

savings to

systems per person

$78,158 per person

systems ranges from

$27,378 per

HMIS/Department of

different

per year is $22,099,

across all systems

$40,122 in Prince

person per

Corrections savings;

public systems a reduction of 47

George’s County to

year across all

$10,724 in Medicaid

percent from the

$45,035 in

systems

savings per person per

status quo

Montgomery County

year

per person per year

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

4 7

Montgomery and

Austin/Travis

Prince George’s

Pima County,

Anchorage, AK

County, TX

Lane County, OR

Counties, MD

AZ

Rhode Island

Quantify the

$22,696 per person

$28,557 per person Anticipated total

$22,018 per person

$17,425 per

$15,000 to $20,000 per

costs of the

per year for PSH

per year for

project budget is

per year (Prince

person per

person per year

intervention

housing, services,

$7,299,885 over five George’s County) to

year for

and other program

years; estimating

$22,671 per person

housing and

costs

100 PSH units,

per year

services

that’s

(Montgomery

approximately

County)

$14,600 per

participant per year

Quantify other Not included in the

Estimated at $1.6

Not included in the Not included in the

Expected to

Not included in the report

PFS

report

mil ion over five

report

report

range between

transaction

years

$350,000 and

costs

$2 mil ion over

5 years

Analyze net

$597 per person per After repaying

Not included in the $18,104 savings per

Total expected Full benefits/savings not

cost or

year cost ($48,010 in investment, $42.8

report

person per year

benefit of

quantified

savings of

PSH and system

million in savings

(Prince George’s

$20.7 million

PSH

costs minus $47,413 over 5 years

County); $22,364

over 5 years;

in system costs)

savings per person

total savings

per year

not calculated

(Montgomery

County)

HMIS = Homeless Management Information System. PFS = pay for success. PSH = permanent supportive housing.

4 8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Net Cost Savings and Other Motivations for Pay for Success Most sites initial y expected PSH to have the potential to generate net savings to the jurisdictions, a primary motivation for some stakeholders to engage with PFS. From analyses conducted during the feasibility analysis phase (exhibit 7) and transaction structuring phase, however, most sites found PSH

would not generate net savings. Despite potential savings to some agencies, those savings would be unlikely to exceed the costs of implementing PSH. For example, in Anchorage, stakeholders initially expected the project to generate net savings that could be invested in later improvements and expansions to the system. Some stakeholders were familiar with the Justice Reinvestment Initiative (JRI) projects, in which reductions in prison costs are intended to be reinvested into crime prevention efforts, potentially with net savings. 15 Cost savings would also have been an important political tool for convincing parties like the state legislature. As we discuss in the next section, however, cost savings were not always found. Anchorage’s feasibility report, for example, estimated that the PFSPSH project would be essentially cost-neutral, with the expected savings to medical system costs (62

percent), state corrections costs (26 percent), and some municipal costs (10 percent) offset by the high per-person costs of the PSH intervention. Without net cost savings, some stakeholders questioned the model’s value.

Only two other sites’ feasibility reports included information on net savings, and those two reports estimated considerable net savings (Austin/Travis County and Montgomery and Prince George’s Counties). In some sites, however, though some stakeholders expected the PSH intervention to produce net cost savings, others remained skeptical and expected PSH costs to exceed savings, either because they believed the implementation costs of PSH were underestimated or the savings were overestimated.

Partners in most sites—especially the project intermediaries—moved away from messaging about net cost savings and toward messaging about spending money more effectively and achieving better outcomes for the target population. This change disappointed some stakeholders and required them to reconceptualize the value of the project. Some stakeholders, however, believed paying for outcomes was a sel ing point for state and local politicians because it might be popular with the public, and that private capital brings valuable governance, oversight, and accountability.

15 JRI, funded by the Bureau of Justice Assistance of the U.S. Department of Justice, uses data analysis to identify costs and drivers of criminal justice populations and to develop evidence-based and cost-efficient policy options to reduce them.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

4 9

Potential Savings from Successful Permanent Supportive Housing Implementation

Another important consideration for financial transactions in the demonstration is which entities could see cost reductions from successfully implementing PSH with the target population of high utilizers. That is, under the status quo, which agencies or organizations are bearing the costs of serving this population? There are often potential savings to corrections, homelessness, and health systems, and these costs may vary across levels of government. For example, in Austin/Travis County, housing costs are borne by the city, whereas the county bears jail costs. In Alaska, the project involves local jurisdictions (Anchorage and, initially, Matanuska-Susitna Borough), although, with a unified correctional system, local jail costs are general y borne by the state.

Despite the potential for savings in criminal justice, homelessness, and health service utilization through effective PSH implementation, many sites found that reducing the service burden would not generate considerable savings that could be reallocated to fund PSH for two basic reasons. First, the marginal costs per person of some services are low, something stakeholders at several sites mentioned. For example, in corrections, the marginal costs per person for a day in jail can be very low (often just the cost of food and laundry), so little budgetary savings result from effectively reducing recidivism among the relatively small target population. Only an intervention that frees up services for many people simultaneously would create larger cost savings (for example, by closing a jail wing or reducing the number of staff members). Additionally, in some sites, some service resources that might be diverted from PSH program participants through effective program implementation are expected to be used for other clients. That could include law enforcement time, jail beds, homeless shelter beds, and emergency room resources. Stakeholders in multiple sites, including Anchorage, Rhode Island, and Austin/Travis County, mentioned this issue of excess demand.

Second, in the absence of PSH, the costs of services for the target population in some sites are partially funded by other parties, so any savings will not accrue to the local partners who could be end payors. In Medicaid expansion states, some of the target population’s health care costs may be covered by federal funds. Thus, a reduction in the use of those services wil not generate local funds to support PSH.

Finally, to structure an effective financial model and a transaction that can achieve that model’s goals, PSH implementation costs and potential savings must be estimated accurately. As one stakeholder noted, building the financial model was an iterative process. Stakeholders described several chal enges to estimating costs and savings accurately, and to generating confidence in those

5 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

estimates among site partners. First, in order to demonstrate potential cost savings to stakeholders, cost analyses require useful data and contextual information. Some stakeholders believed cost estimates were too theoretical and removed from real situations, and that cost estimates misidentified the level of government that would accrue savings (such as state- or county-level actors). In addition, some service providers worried their implementation costs were being underestimated, especially if the providers did not have previous experience working with the target population. Some service providers stressed the importance of flexibility within service budgets, arguing that this high-needs population may require more services than anticipated, which might necessitate smal er caseloads.

Finally, one stakeholder noted that PSH would not reduce al types of health care costs and that some might increase (for example, pharmacy or medication costs).

Identifying and Securing End Payors

Identifying end payors is crucial for a feasible financial transaction. Typical y, the end payors are the government agencies that expect cost savings as a result of the preventive effects of the intervention.

The basic idea of the demonstration is that the PFS transaction can help governments shift from paying for services directly to paying (only) for successful outcomes from those services. Because different government agencies (often at different levels of government) bear different costs of serving the high-utilizer target population, securing end payors can be a complex process. In the demonstration, in which PFS intermediaries are the prime grantees, some government participants typically also serve as champions of the project. As examples, local jurisdictions in Anchorage and the governor’s office in Rhode Island played this role.

In some sites, healthcare providers and hospitals—which bear considerable costs of serving the target population—seemed to be natural end payors. This involved additional complexities sometimes, however. In Austin/Travis County, some stakeholders mentioned the importance of demonstrating that healthcare system dol ars are being used to fund verifiable health services and outcomes, rather than to pay for housing. In Anchorage, there was reportedly incompatibility between the project’s financial model and hospitals’ incentive structures.

Some sites worked with multiple potential end payors to secure enough funding, although this added considerable complexity. In some cases, addressing cross-sector issues may require that different end payors pay on different success metrics. For example, at the end of 2018, Austin/Travis County had five potential end payors that might pay on different metrics, with the city paying for housing outcomes, the county paying for criminal justice outcomes, and healthcare entities paying for H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

5 1

health outcomes. Although complex, such arrangements may help address the wrong pockets problem. Outside of this demonstration, 8 of the first 25 PFS projects that were launched involved more than one government end payor (NFF, 2019a).

The Costs of Implementing Permanent Supportive Housing

Without additional financial arrangements, the costs of implementing PSH would fall to housing and health and human services agencies. Many sites attempted to find other funding sources, so investors and success payments would be needed for a smaller portion of the PSH costs. That makes PSH easier to implement, but it can also reduce the motivation to engage in the complex arrangements of PFS.

Philanthropies are playing an important role in financial feasibility in several sites by providing funding as investors, end payors, or both. For example, in Los Angeles, the Hilton Foundation provided a loan at 50 percent low interest and 50 percent no interest to support implementation as a PFS investor. This shifted more of the investment risk from a for-profit investor partner onto the philanthropic investment partner. In Lane County, the Oregon Community Foundation is providing a 36-month bridge loan to the service provider.

Federal funds have also been important; some sites have used housing vouchers and/or Medicaid waivers to lower the jurisdiction’s PSH implementation costs. HUD funds for success payments in the PFS demonstration were also considered crucial to the financial feasibility of the project to some stakeholders. In Lane County, HUD-DOJ success payment funds were key to launching the first phase of project implementation.

Some sites successful y obtained 1115 waivers that al ow some PSH service costs to be covered by Medicaid funds, and other sites were attempting to obtain such waivers. This was discussed in Anchorage, Austin/Travis County, and Montgomery and Prince George’s Counties. When states also participate in the Affordable Care Act’s Medicaid expansion, this al ows more federal dollars (and fewer state dollars) to fund those services. (As of 2018, all participating sites were in Medicaid expansion states, except Austin/Travis County.)

Some participating sites expected or hoped to cover some PSH costs through federal housing vouchers. For example, in Anchorage, three scenarios were being considered: one in which vouchers would provide 30 percent of the needed housing, one in which they would provide 50 percent, and

5 2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

one in which they would cover all the housing. 16 Similarly, Rhode Island is exploring whether funds that have historical y been used for capital construction could be reallocated to cover rental subsidies and services for the PFS project. Some stakeholders have raised concerns, however, about whether these housing resources should be used for a small target population when they might otherwise be used for more people. This tension is one consequence of using funds for a high-needs, high-utilizer target population, and some stakeholders may not be convinced of the severity of the cost burden that the target population is imposing. Relatedly, some stakeholders also expressed concerns about reserving scarce resources for the reentry population that might otherwise be used for people who do not have prior involvement with the criminal justice system.

Several stakeholders mentioned that securing permanent funding for PSH beyond the length of the demonstration was a challenge, raising issues of sustainability. Some sites had begun planning or thinking about this long-term need, and some stakeholders believed that if the project’s evaluation shows success in meeting payment benchmarks, the local government would take over long-term funding of the project.

Finding Investors and Managing Risk

Until success benchmarks can be achieved, investors are needed for PFS transactions to cover implementation costs (besides what is funded through other means, such as housing vouchers).

Demonstration sites differed on when and how they engaged investors, dependent on local circumstances. Most sites focused first on securing end payors, establishing the project’s viability, and setting bounds on the project’s design before approaching potential investors.

Despite the necessity of finding funding for implementation costs upfront to launch the project, few stakeholders spontaneously mentioned investors when asked about their considerations for financial feasibility; finding investors did not seem to be a primary concern. Relatedly, few government stakeholders—who are typically potential end payors—discussed the risk of funding an ineffective program and a desire to transfer that risk to investors as a motivation for engaging in PFS.

More stakeholders expressed concern about paying interest to investors, on the assumption that the program would be successful. Paying interest was seen as politically risky (and financially costly), and some stakeholders questioned whether investors should be making money off of the provision of 16 In the Arizona demonstration site, the City of Tucson offered to contribute housing vouchers to the project.

And after the feasibility analysis phase, Pima County, which includes Tucson, decided to directly fund PSH

without PFS.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

5 3

services to vulnerable populations. This perceived risk seemed heightened by the apparent complexity of the anticipated PFS financial arrangements.

Lane County moved into the first phase of implementation without finding investors willing to assume the risk that PSH would not meet success benchmarks. Instead, the Oregon Community Foundation is providing a 36-month bridge loan to the service provider. Success payments will allow loan repayment using funds from the grant; if success metrics are not achieved, the service provider will have to repay the loan directly.

Political and Legal Complexities

The pre-implementation phases of feasibility analysis and transaction structuring are complex—

general y involving considerable education about PFS and complicated negotiations—and can be lengthy. In 2018, political cycles interrupted the PFS process in three sites, where elections led to considerable uncertainty about the stability of key decisionmakers and stakeholders. In Anchorage, corrections are a state function, and the Alaska gubernatorial race was contested. Ultimately, the governing party changed, and at the end of 2018, commitment from the new administration to either PFS or PSH was uncertain. In Rhode Island, the governor had been a champion of the PFS PSH project and was running for reelection in a contested race. Although she was reelected, the election was a focus of some important stakeholders and injected considerable uncertainty into the project, uncertainty that ultimately caused a sizable delay. In Montgomery and Prince George’s Counties, the county executives were term-limited. The newly elected leadership in both counties were from the same party as the previous county executives, so project partners were optimistic that the new leadership would be supportive. Nonetheless, there was turnover among some key government stakeholders, and by the end of 2018, some positions remained unfil ed. Thus, although these three sites had very different elections, project implementation was delayed considerably in each, and pending elections reduced the ability and/or willingness of key stakeholders to undertake long-term commitments to PSH or to make financial commitments to bring PFS transactions to a conclusion.

Several sites were concerned about legal restrictions on states and jurisdictions committing money in the future, restrictions that required considerable political will to address sometimes. In addition, some government stakeholders were concerned that taking on debt could negatively affect their jurisdictions’ credit ratings. Stakeholders in Austin/Travis County also mentioned the Texas legislative requirement that government entities be able to terminate a contract for cause, convenience, or failure to appropriate. How that might be reconciled with a PFS contract is not yet

5 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

clear, but it might require contract clauses that allow for the possibility of early termination with direct payment of services provided to date. Such issues sometimes required legislative fixes at the state or local levels or creative financing arrangements, such as ones in which the funds for success payment are paid in the short term to a third party that holds the funds until success metrics are evaluated and payments are due.

The financial situations and structures of state and local governments also affected the PFS

projects’ financial feasibility. Those include budget cycles and the likelihood of receiving appropriations. For example, Oregon’s biennial state budgetary process required the Lane County project partners to consider other payment sources. And in Alaska, where state government expenses had traditionally been funded by oil revenue and without an income tax, a decline in oil revenue affected discussions of the Anchorage project’s financing.

D. Establishing the Feasibility of Implementing

Permanent Supportive Housing

The fourth key step toward establishing a feasible PFS project is to determine the feasibility of implementing the intervention. Throughout the feasibility analysis and transaction structuring phases, demonstration sites planned for the key components of what would eventual y be an implemented PSH model for the target population. Important topics to address regarding the implementation of PSH during the feasibility analysis and transaction structuring phases included:



Enrollment pathways.



Housing availability.



Services components.



Service provider capacity—both the skills for serving the target population and the resources for the required number of clients.



Potential training and technical assistance needs.

As we discussed earlier, feasibility reports addressed some of these topics, especial y the availability of housing and resources. As sites moved through the transaction structuring phase, some created a dedicated working group (and in some instances engaged technical assistance providers) to discuss and make decisions about remaining PSH model components. Among the issues that still need to be resolved were participant identification, enrollment pathways, and service provider H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

5 5

capacity and skil s gaps, particularly around the integration of the Housing First philosophy and criminal justice services.

Participant Identification and Enrollment Pathways

During the feasibility analysis phase, sites used locally available data to determine whether a sufficient number of people would meet the target population criteria. The feasibility analyses were seen as confirming previous assumptions about the existence of a population and its overlapping system involvement and needs. When moving toward implementation, sites were required to more clearly define program eligibility criteria, prioritization, and pathways for identification and enrol ment.

The two sites—Los Angeles County and Lane County—that had begun project implementation by late 2018 are addressing different target populations. Los Angeles County is serving people in jail who are candidates for diversion from the criminal justice system, 17 whereas Lane County is targeting people being released from state prisons (making it unique among demonstration sites).

How eligible people wil be identified and enrol ed in the program is a crucial design decision; it affects which people among those who meet the HUD-DOJ demonstration criteria are ultimately enrol ed in the program. These methods varied considerably among project sites. For example, Los Angeles County used jail in-reach to identify participants, which meant that people were incarcerated when they first interacted with this program. In designing its referral pathways, Anchorage considered using coordinated entry, a process HUD requires for al communities receiving federal homelessness assistance funds. Coordinated entry is intended to ensure that all people experiencing a housing crisis have fair and equal access and are quickly identified, assessed for, referred, and connected to housing and assistance based on their strengths and needs. Although stakeholders in Anchorage agreed that supporting and working within the existing homelessness system was important, there was a concern that only a fraction of the unsheltered population appears in the homelessness system’s data. The project team was figuring out how to incorporate other sources of information, including the municipality’s Mobile Intervention Team, to identify participants for the project.

17 The eligibility criteria for clients include that (1) they are currently within the county criminal justice system; (2) are expected to be discharged in 30–120 days or are eligible for diversion services through specialty courts, or are identified through other referral sources under the auspices of the Office of Diversion and Reentry.

5 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Pilot Periods

Some sites used pilot periods to answer important questions about enrollment pathways and confirm assumptions about PSH implementation. Two sites—Austin/Travis County and Lane County—

launched pilots during the transaction structuring phase. 18 Neither used performance-based payments, nor were they funded by the PFS project itself. 19 In Austin/Travis County, services were funded through philanthropy with rental assistance from the city government; in Lane County, the pilot leveraged project partners’ existing housing and service budgets. As of the end of 2018, through these pilots, Austin/Travis County had placed 24 clients and Lane County had placed 92. Anchorage, meanwhile, was planning to launch a ramp-up period in 201920 that would be funded entirely with philanthropic dollars.

The questions explored during the pilot periods pertained to service delivery and project design, testing assumptions about recruitment and enrollment pathways and defining relationships and divisions of labor among project partners. Austin/Travis County also used its pilot to build provider capacity and test its approach to managing housing placement. In contrast, Lane County used its pilot to help predict program performance and use this to inform the PFS project’s structure.

Austin/Travis County and Lane County learned useful lessons during the pilot period, such as its population’s high need for medical services; the value in separating navigation, housing, and supportive services case management; the need to ensure consistent eligibility criteria; characteristics of the service population; and what data are needed for developing a performance baseline and helping set targets.

Anchorage planned to use its ramp-up period to help further define the target population and enrollment pathways. Partners discussed several options for these pathways. The first was to identify and prioritize participants through administrative data held by the local university. A backup option was through the outreach and services team embedded in the local fire and rescue department, which was seen as a potential referral source. Simultaneously, the Continuum of Care was planning to refine the coordinated entry processes and hoped to consolidate referrals to the project within that process.

18 Los Angeles did not launch a pilot specifical y for this demonstration; however, they were able to pul lessons learned from the first wave of the Just in Reach program to inform their expansion during Just in Reach 2.0.

19 The HUD-DOJ NOFA allowed some funding to be used for ramp-up periods that are included in the success payments.

20 Los Angeles County conducted a pilot study before applying to participate in the demonstration; its pilot program is summarized in section IV.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

5 7

Anchorage used its implementation work group to discuss potential enrollment pathways and planned to refine this process during the ramp-up period.

Stakeholders in one site noted some tension between the pilot and the full project, and they described the need to emphasize that the pilot was for enrol ment and service delivery for PSH, not of PFS.

Service Provider and Housing Capacity

Most feasibility reports did not answer the question of whether local service providers could implement PSH, although partners in the sites frequently mentioned that as a concern during interviews in 2018. These capacity-related questions included whether local service providers could implement a PSH project of the sizes being discussed, whether areas had enough skil ed workers who could be hired by local service providers to meet the service needs of the population, and whether the programs had the skil s necessary to implement a Housing First PSH program. In one site, some stakeholders questioned whether service providers could scale service responses for the target population. Stakeholders indicated that the local economy might not be able to support the skil ed workers these programs would need to serve participants.

Every site planned to leverage local housing resources for its project. In some sites, concerns were expressed about the ability of providers and the local housing market to house the target population.

In Lane County, the pilot relied on a project-based location called The Oaks at 14th—a housing complex that will henceforth be called The Oaks. To meet enrollment targets for implementation, the program needed to expand housing options, and Lane County ultimately incorporated public housing and housing choice vouchers. Some stakeholders questioned whether some people reentering the community from prison—such as those who have been convicted of a sex crime—would be able to secure units through these housing types. Stakeholders in Rhode Island expressed concerns about low housing inventory. Stakeholders in Montgomery and Prince George’s Counties expressed similar concerns about the cost of housing in the jurisdiction and the potential for project-based units to encounter NIMBY resistance. The scattered-site model for PSH was seen as less likely to face that issue. In Anchorage, stakeholders indicated that uncertainty around the turnover of housing vouchers was a concern for meeting enrol ment targets. Austin was also concerned about overall voucher availability.

5 8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Housing First

In several sites, stakeholders raised concerns about the challenge for the service providers who are implementing a Housing First approach to PSH for the first time. In Los Angeles County, some contracted service providers did not have Housing First experience before this project. In Anchorage, stakeholders indicated that only one organization in the area had operated PSH using a Housing First approach and that familiarity with Housing First varied across the site. Stakeholders in both Montgomery and Prince George’s Counties also suggested that gaps in knowledge of Housing First practice in their county remain a concern. In this period, Anchorage has worked with the Corporation for Supportive Housing to close their knowledge gap through training, site visits, and a capacity assessment.

Permanent Supportive Housing and Criminal Justice

ONGOING CRIMINAL JUSTICE REFORMS

In several sites, the demonstration project was designed—or would be designed—to complement ongoing state- and local-level interventions and efforts to reduce incarceration and present more alternatives to incarceration. Such interventions include the fol owing:



In Anchorage, before the demonstration, there were efforts to change the way the state deals with the incarceration of the reentry population. There was legislative work through the state’s criminal justice reform and JRI to shape conversations about alternatives to incarceration.



In Austin/Travis County, there were several concurrent efforts to reduce the local jail population and present alternatives to incarceration. These included a new sobering center (to divert the estimated 4,000 to 5,000 public intoxication arrests), diversion programs for people with serious mental illnesses, extended funding for mobile outreach crisis teams, and expansion of the use of citations as alternatives to arrests.



In Lane County, PFS complements JRI on the state level (where PSH and wraparound services have been seen as justice reinvestment strategies), as well as the Department of Corrections’

interest in reducing the size of state’s prison population. PFS also aligns with Lane County’s efforts to implement PSH and reduce barriers for people with criminal histories reentering the community.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

5 9



In Los Angeles County, significant legislative changes on the state level reclassified offenses and reallocated resources to criminal justice reform (including Proposition 47, Senate Bill 678, and Assembly Bill 109).



Montgomery County and Prince George’s County both previously pursued initiatives to support diversion and reentry into the community. Montgomery County built a Pre-Release and Reentry Services facility, and Prince George’s County received support from DOJ’s “Smart Reentry: Focus on Evidence-Based Strategies for Successful Reentry from Incarceration to Community” program.



In Rhode Island, the governor established the Justice Reinvestment Working Group in 2015 to focus on issues related to probation. This effort lead to the implementation of six bills in 2017

to reform the statewide parole and probation system. Rhode Island also participates in the Data-Driven Justice Initiative to use data to help develop strategies that divert people with low-level offenses and mental illnesses from the criminal justice system and that employ alternatives to pretrial incarceration.

OTHER IMPLEMENTATION ISSUES RELATED TO CRIMINAL JUSTICE

Stakeholders in several sites noted barriers to providing PSH to the reentry population. Those barriers included the following:



People with certain criminal histories could be excluded from services. For example, someone who had been convicted of a sex offense may not be able to live in some areas and might be ineligible for certain services, including Section 8 vouchers or public housing.



Traditional case management may need to be intensified to meet the reentry population’s needs.



Service providers can experience high burnout rates when working with particularly vulnerable populations.



Coordinating housing and other interventions for people who cycle between the jail and the community is difficult, especially when a person’s release date is unclear.



Some landlords are unwilling to rent to people with criminal records.



Additional time and resources are required if service providers need to communicate with a judge, prosecutor, public defender, or probation officer.

6 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON



Austin/Travis County and Los Angeles County were dealing with remodeling old facilities and building new ones, projects that have received significant pushback from the community and advocates. Sites struggle to balance local reform efforts to reduce the jail population and increase alternatives to incarceration with concurrent efforts to expand jail capacity.



Identifying who is homeless among people involved with the criminal justice system can be difficult. For example, some correctional facilities have agreements with hotels or motels to house people recently released, but discharging someone from incarceration to a hotel does not identify them as homeless.

HOUSING FIRST AND CRIMINAL JUSTICE

Housing First, a key component of the demonstration projects, emphasizes housing provision without conditions and a voluntary approach to service provision. Participants are held to the same requirements as a typical tenant and are not evicted for failure to comply with a services plan. Many in the target population, however, may have conditions to meet as part of community supervision, and parole and probation officers are responsible for ensuring their supervisees are complying. A few respondents mentioned this as an issue in establishing a feasible PSH project, especially if the collaboration between community supervision and homelessness services is new. Will service providers be required to report participant activities that may violate their parole/probation but would not lead to their eviction from a Housing First program? How would such requirements affect staff relationships with participants? This issue arose during project implementation (as discussed in the next section), but because only one site was significantly into the project implementation phase by the end of 2018, that discussion is preliminary.

E. Establishing Success Metrics and Evaluation

The fifth and final key step toward producing a feasible PFS project is establishing success metrics and planning for the evaluation. Demonstration sites worked on establishing success metrics and setting payment outcome triggers during the feasibility analysis and transaction structuring phases. As noted in the baseline implementation report (Liberman et al., 2019), by the end of 2017, most sites were considering measuring outcomes across all three domains of criminal justice, homelessness, and health. Exhibit 16 provides an overview of the proposed outcome areas for each site.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

6 1

Success Payments

During the feasibility analysis phase, sites started discussions about which outcomes would inform success payments. During the transaction structuring phase, sites operationalized these outcomes and decided on specific metrics and benchmarks that must be met to trigger success payments.

EXHIBIT 16

Outcome for Success Payments at Demonstration Sites

Outcomes

Status of

Housing

Criminal Justice

Health Care

Outcome Plan

Anchorage, AK

Increased housing Reduced prison or Reduced

Proposed

stability

jail days

psychiatric

hospitalizations or

other service

utilization

Austin/Travis County, TX

Increased housing Reduced jail bed

Reduced

Proposed

stability

days or bookings

emergency room

visits and reduced

inpatient days

Lane County, OR

Stable housing at

New incarceration

NA

Final

6 and 12 months

owing to felony

conviction

avoidance

Los Angeles County, CA

Housing retention Jail avoidance

NA

Final

at 6 and 12

months

Montgomery and Prince

Increased housing Reduced prison or Reduced

Proposed

George’s Counties, MD

stability

jail days;

emergency room

reduced arrests

and inpatient use

Rhode Island

Increased housing Reduced prison or Reduced inpatient

Proposed

stability

jail days

days

NA = not available.

At the end of 2018, four sites—Anchorage, Austin/Travis County, Montgomery and Prince George’s Counties, and Rhode Island—were still finalizing their payment outcomes. The two sites that had begun implementation—Lane County and Los Angeles County—had established outcomes and success metrics in a PFS contract. (Those success metrics are described in more detail in the

“Implementing PSH” section.) As sites worked to develop their success metrics and payment outcomes, the following common themes emerged:



End payors were an important part of the success outcome discussion. One reason that most discussions about successful outcomes and payment triggers occurred in the transaction structuring phase is that end payors typically had not yet committed during the

6 2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

feasibility analysis stage. As one respondent noted, “ We’re not really driving the outcomes or measurement; funders are because they to some extent are deciding what to pay for and what matters to them to achieve.”



Metrics should be easy to understand, but nuanced. A few sites expressed concern about making payment metrics too complicated. Sites felt most comfortable designing metrics for housing stability and using established criteria in defining positive housing exits, most notably relying on HUD’s categories. There was more variation in the criminal justice metrics; avoidance of arrests, bookings, and jail or prison returns were all considered.



The literature supports some measures and benchmarks but is unclear on others.

Research has demonstrated the relationship between PSH and housing stability for the target population and provides clear benchmarks—for example, 85 percent housed at 6 months (Aidala et al., 2014). The research base is less robust on criminal justice metrics and even less so on health outcomes.

Evaluation

Evaluation is at the core of PFS, and evaluation partners play important roles in the project. Evaluation partners joined the PFS project teams at various points in the demonstration. Al the sites are required to select an evaluation partner and launch an evaluation that uses “experimental, quasi-experimental methods, or other scientific methodology to be executed by an evaluator,” according to the HUD

NOFA. The primary evaluation objectives are for third-party evaluators to:



Monitor outcome measures that are outlined and agreed upon in the PFS contract.



Assess the operationalization and implementation of the PFS intervention, ensuring it is consistent with HUD’s program guidelines.



Determine whether the outcomes outlined are achieved and provide documentation to trigger the release of success payments.

Sites may select the third-party evaluator through a Request for Proposals (RFP) process or other procurement pathways. Los Angeles County already had its evaluation partner (RAND) when they applied for the HUD-DOJ grant.

Although not required by HUD-DOJ, some sites involved evaluators during the pre-implementation phases, during which evaluators can provide important supports. During the H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

6 3

feasibility analysis stage, evaluators can serve as PFS thought partners, especial y in understanding the feasibility of demonstrating success. Although some sites selected their evaluators early, cost considerations reduced their level of involvement during the feasibility analysis phase. During the transaction structuring phase, evaluator involvement was more common; evaluators helped refine success outcomes and program design to help ground the final program negotiations that wil produce the PFS contract and the parameters by which success payments wil be made.

Urban provides consultation services to local evaluation partners for each PFS project, including on the collection of baseline outcome data, the viability of proposed local data collection efforts, and the local outcome evaluation plans. By the end of 2018, Urban had provided consultation to Austin/Travis County and Lane County about evaluation ideas and draft plans, and they had also reviewed the evaluation plans for Los Angeles County and Lane County.

Sites engaged evaluators at different stages. Exhibit 17 provides an overview of where sites were in the process of selecting an evaluator as of late 2018. This is fol owed by a brief summary of the planned evaluation approach that wil be taken by each site.

EXHIBIT 17

Evaluation Partners and Methods

Process

Evaluator

Methods

Anchorage, AK

Previous partner

NPC Research

Under development

Austin/Travis County, TX

RFP

Abt Associates

Quasi-experimental

Lane County, OR

Previous partner

New York University

Natural experiment

Los Angeles County, CA

Onboard at time of

RAND

Quasi-experimental

HUD-DOJ application

design

Montgomery and Prince

Onboard at time of

ICF International, Inc.

Under development

George’s Counties, MD

HUD-DOJ application

Rhode Island

TBD

TBD

Under development

NPC Research = Northwest Professional Consortium, Inc. TBD = to be determined.



Anchorage. Northwest Professional Consortium, Inc. (NPC Research), a small research firm in Multnomah County, Oregon, was selected as the evaluator for the Anchorage site. An evaluation-planning working group, chaired by the lead evaluator from NPC Research, had regularly been meeting and was focused on drafting the evaluation plan, preparing the Institutional Review Board proposal, and working on data-sharing agreements. A draft evaluation plan should be ready at the end of September 2019.



Austin/Travis County. With help from CSH, the site launched an RFQ for evaluators in May 2018 but did not find the right project partner. After making some changes (for example,

6 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

clarifying expectations and reducing the maximum response length from 20 to 10 pages), the project rereleased the revised RFQ in July 2018 with broader distribution and targeted outreach. This yielded more responses, which resulted in first-round interviews with four potential evaluators in late August and early September 2018. Most interviewed stakeholders did not express a strong evaluation design preference. There was a general disinterest in a randomized controlled trial, however, primarily because of concerns over cost and insufficient sample size. Several stakeholders referenced a quasi-experimental design, and at least one suggested that the pre-post would be sufficient. Importantly, the core team indicated the importance of having a process evaluation component. Ultimately, Abt Associates was selected as the evaluator.



Lane County. Evaluation planning took place throughout the feasibility analysis phase because the evaluator, based at New York University, had previously evaluated other Sponsors, Inc. programming and had participated in project planning meetings as a content expert on the population and service models. The evaluators will provide information for the purposes of success payments on two metrics: housing stability and felony conviction avoidance. For the evaluation, some key partners would have preferred a random assignment study because, as one interviewed state stakeholder indicated, it would carry more weight with state policymakers and agency officials. Ultimately, however, the evaluators proposed a natural experiment. The target population is reentering the community from state prison.

Because of natural fluctuation in availability, PSH units will be available when some people return from prison (the proposed treatment group) but not available as others return (the proposed control group), and this availability should be unrelated to participant characteristics. The evaluation will include a broad examination of the impact of PSH on those served, consider arrests and convictions for both felony and nonfelony offenses, assess healthcare utilization, and consider arrests and convictions for felony and nonfelony offenses, healthcare use, and employment. Data will be collected through administrative data analysis and brief quarterly interviews with all members of the treatment and comparison groups. In-depth interviews will be conducted with a subset of people identified through a modified success case study method.



Los Angeles County. RAND was brought on early in the project’s planning phases before the demonstration application was submitted. The evaluation team received funding from the Irvine Foundation to draft an evaluation plan to share with potential investors. RAND has been responsible for tracking success payments and verifying success metrics for the PFS-PSH

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

6 5

Demonstration, and it has verified the success payment metrics for the PFS-PSH

Demonstration for two quarters.



Montgomery and Prince George’s Counties. ICF International, Inc., a global consulting firm, was selected as the evaluator in 2018. Overall progress was affected by the turnover of local government staff members and elected officials after the 2018 elections. ICF International, Inc. staff members also coordinated the evaluation subgroups of the steering committees in both counties.

CHALLENGES WITH EVALUATIONS

In addition to reviewing the evaluation plans, Urban asked key stakeholders about their plans for the evaluation and some of the chal enges they may have faced launching this component of the project.

Stakeholders’ chal enges included the following:



Uneasiness with randomized controlled trials. Stakeholders in nearly all demonstration sites expressed hesitation about implementing an RCT. Among the reasons was a belief that RCTs make projects considerably more expensive, concerns that sample sizes would be too smal to accommodate both the treatment and control groups, a perception that RCTs raise the data-collection burden, and ethical concerns about withholding services from the control group. Some intermediaries shared some of these concerns for their particular sites and did not advocate for RCT in their projects.



Paying for impact versus paying for outcomes. All the sites that drafted evaluation plans included a quasi-experimental approach for their evaluation. This meets HUD-DOJ’s requirement for the grant, but it should be noted that the success payments for those sites will not be based on the results of a quasi-experimental study that compares outcomes of participants with a comparison group. Instead, they will be based only on the observed outcomes of program participants. Setting payment triggers based on impact is not specifically required by HUD-DOJ, but researchers have highlighted it as an important component of PFS (Milner and Walsh, 2016). Using one evaluation method to understand broader impacts and focusing on outcomes for payment triggers may result in circumstances in which investors are paying for success that is not supported in the broader impact evaluation. This could make the PFS model less credible.



Process studies will be required to understand the impact. Stakeholders at several sites noted the importance of including a process or implementation study in the evaluation. This can serve two purposes: (1) to document the model, and (2) to help evaluators understand

6 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

and interpret what is driving the end results. Including a process study was not specifically required by HUD-DOJ as a part of the PFS evaluation and supporting it might require sites to raise additional resources. For example, Montgomery and Prince George’s Counties had been in discussion about dedicating more resources to enhance their evaluation with a process study component.



Data concerns. Although all the sites during the feasibility analysis stage went through a process of accessing the data resources needed to carry out the PFS project, chal enges with data fragmentation and data quality persisted, and those will make pulling together the data needed to evaluate and monitor the PFS outcomes difficult. For example, automating a data dashboard on implementation metrics (such as program enrollments, signed leases, and program exits) is a critical component of understanding project progress (which investors want to track); it is also crucial to being able to improve program performance and make midcourse corrections. Los Angeles County noted some challenges with how long it took to pul together data from different sources, ensure that they were reliable, and report on the results. Other sites were still in the early stages of creating implementation dashboards, and this area could require technical assistance.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

6 7

IV. Implementing PSH

This section provides an overview of Los Angeles County’s and Lane County’s implementation efforts in 2018. During the implementation phase, clients are being actively enrolled and served, and their outcomes are being col ected to inform future decisions related to success payments. The implementation phase should reflect the lessons from any pilot or ramp-up period in a clearly defined program model. The project components of each site’s implementation process are expected to vary depending on local needs, priorities, and political and funding contexts.

Los Angeles County Implementation

Before the HUD-DOJ demonstration, Los Angeles County conducted the Just in Reach pilot, which was designed to focus on the hardest-to-serve population: people experiencing homelessness and with repeated criminal justice system contact (incarcerated three times over 3 years and with three episodes of homelessness in 5 years). The initial pilot lasted from 2008 to 2010, with funding from the Los Angeles Sheriff’s Department Community Transition Unit and the Robert Wood Johnson Foundation. Just in Reach 2.0 launched in 2014 to serve a similar population. Simultaneously, the Los Angeles County Board of Supervisors passed a motion directing the chief executive officer (CEO) to develop a blueprint for PFS within the county. In July 2015, the board approved the county CEO’s recommendation that Just in Reach 2.0 be prioritized as Los Angeles’s first PFS project.

Los Angeles County entered the program implementation phase in October 2017, with 2018

being the first full year of enrollment. As of December 2018, the county had placed 182 people in housing. Using a jail in-reach model, the project targeted people who were within the county criminal justice system and had been sentenced to probation, had experienced homelessness, and had other needs, such as substance use disorders or serious mental il nesses. Upon release from jail, clients were matched with a service provider for supportive housing.

During project implementation, interview and survey respondents indicated that service providers spent the most time on the project. Stakeholders in other roles, including the evaluator and sheriff’s department, indicated that they had spent smal amounts of time on the project through 2018.

6 8

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Participant Enrollment and Housing

In Los Angeles County, clinicians identify people in jail who are candidates for diversion from the criminal justice system and who have a qualifying score on the Vulnerability Index–Service Prioritization Decision Assistance Tool (VI-SPDAT), 21 defined as scoring in the upper third quadrant, meaning some of the highest-needs people for whom Los Angeles deems PSH a viable option. There is some subjectivity as to who is referred for the program because, ultimately, a judge decides whether someone can participate in the diversion programs offered by the Office of Diversion and Reentry (ODR), including this program. During project implementation, the courts decided to no longer refer participants who had only a substance use issue and no accompanying mental health disorder.

When the courts refer a potential participant to the program, jail discharge coordination begins between the sheriff’s department and the ODR service provider. The service provider is responsible for ensuring that, upon leaving jail, the participant enters an interim housing program in the community. The structure of interim housing placements varies considerably, from large programs that have congregate living spaces to apartment-like living spaces in smaller programs. Overall, stakeholders reported that the interim housing placements have more rules and requirements for participants than the ultimate PSH placement. Stakeholders reported that potential participants remain in the interim housing placements for roughly 6 to 9 months. During that time, participants work on becoming more stable and securing the documentation necessary to lease an apartment.

When participants are deemed ready to be moved into a PSH placement by their intensive case management service provider, they are placed on the “housing ready” list. Service providers have discretion about how participants qualify as “housing ready” and whether they are placed on the list. 22

When participants are placed on the list, Brilliant Corners (the housing search agency for the project) begins pairing them with potential units. The first 12 potential participants housed each month are enrolled in the PFS-PSH project. Any remaining potential participants housed each month are enrolled in the ODR, non-PFS Just in Reach project, in which the supportive housing intervention is very similar to the PFS program model.

At the start of the project implementation phase, clients transitioned from interim housing to a permanent unit within 3 to 6 months. As the year progressed, however, that time frame stretched to 9

21 The VI-SPDAT score covers a range of vulnerabilities, including health needs or diagnoses and risks.

22 Screening participants for housing readiness is not aligned with a Housing First approach. This is a variation on the model that site partners have implemented.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

6 9

to 12 months due to the time necessary to address the needs of this high-acuity population. The type of interim housing (such as single-site or project-based) varied by the provider.

After leaving interim housing, participants receive a PSH subsidy paid for through the PFS

transaction. At the end of the PFS period, the county will assume responsibility for paying the subsidy.

The subsidies are administered through the Los Angeles County Department of Health Services, which also administers the county’s Flexible Housing Subsidy Pool. Finding housing units for clients in the county’s tight market was a significant challenge. All the project housing in Los Angeles County is scattered-site housing financed through tenant-based subsidies. In addition to being difficult to locate, available units might not be a good match for clients because of location (for example, they might have negative gang-affiliation implications) or the ability of the client (such as whether the unit is accessible or not). And housing options for clients who required more support at home were scarce because the vast majority of permanent supportive housing in Los Angeles is scattered-site.

Service Provision

In Los Angeles County, four service providers under contract with ODR provide intensive case management services. The services are paid for through existing county programs, and the providers serve both participants in the PFS project and clients in the county’s non-PFS Just in Reach project.

For service providers, the two programs are operationally one, and providers do not know which program clients are assigned to. Intensive case management service providers fol ow potential participants from discharge from jail, through interim housing, and into their PSH unit.

Project partners consistently met enrollment targets of 12 clients a month and continued to work toward a final enrol ment target of 300 clients. As project implementation ramped up and the number of clients in PSH increased, service providers experienced high levels of staff burnout and turnover, struggling to keep sufficient staffing levels for ongoing program enrol ment. The project responded to these challenges in two ways: (1) by increasing the budget for service providers to hire a licensed clinician to respond to clients’ serious mental health needs, and (2) by decreasing the staff-to-client ratio from 1:20 to 1:15 to support more intensive engagement.

One provider was a reentry service provider that had not implemented a Housing First model before this project. This provider had additional requirements for participants before they were placed on the “housing ready” list.

7 0

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Implementation Challenges and Criminal Justice Involvement Service providers reported the following implementation chal enges that stemmed from the target population’s criminal justice involvement:



Compared with other clientele, the reentry population required more attention and one-on-one interaction with case managers. Typical case management techniques were intensified and adjusted for the reentry population. Service providers taught basic life skills, including how to wash clothes and shop for groceries.



Working with the reentry population required working with local criminal justice partners like probation, the sheriff’s department, and the court system. This strengthened partnerships between service providers and the justice system, but it also necessitated a higher level of coordination and exchange of information across agencies.



Caseworkers were required to provide documentation to the courts on participants’ activities, specifically their compliance with court orders. It was unclear whether participants knew that or whether it affected the relationship between participants and the service providers.



Although securing buy-in for the program from clients while they were still in jail was not a challenge, clients sometimes lost touch after their release because they were no longer in need of a diversion program as an alternative to incarceration.



Clients’ unpredictable jail release dates made planning work difficult. ODR assisted by becoming involved in court cases and helping track and manage conditional releases.

However, the churn of people in and out of jail made anticipating when clients would need bridge housing and coordinating permanent housing units a challenge.



Finding permanent housing was more difficult for the program’s reentry population because landlords often created barriers for, or screened out, people with criminal records, particularly those who had been convicted of sex offenses.

Success Payments

Los Angeles County’s project designed a success payment process based on the achievement of verified housing and justice client outcomes. This process is distinct from the program evaluation planned by RAND, which involves a comparison of client outcomes to the outcomes of the quasi-experimental comparison group. To detect outcomes that triggered success payments, project partners reported client-level data in an existing county database for county-funded supportive H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

7 1

housing. Bril iant Corners reported on client move-ins to housing and the cost of housing subsidies; the four other service providers reported on case management metrics (including exits from housing), and the sheriff’s department reported on rearrests. ODR pul ed these data on a quarterly schedule and shared them with CSH, the project’s intermediary. CSH then used the data to determine project outcomes for 6-month and 12-month housing retention and to calculate the associated success payments. RAND independently verified the outcomes calculated by CSH. In 2018, the project reported on housing retention outcomes for three quarters, resulting in more than $1.3 million in success payments, as summarized in exhibit 18. These payments were based on verified housing outcomes. Success payment in future years will be made on housing and jail avoidance outcomes.

EXHIBIT 18

Los Angeles County Success Payment Outcomes Reported in 2018

Total for

Three

Q4 2017

Q1 2018

Q2 2018

Quarters

Eligible participants for 6-month housing

39

36

42

117

retention

Participant met outcome

36

34

37

107

Negative exit

3

1

3

7

Positive exit

0

0

1

1

Deceased

0

0

1

1

Higher level of care (for example, assisted

0

1

0

1

living)

Payment for 6-month housing retention at

$218,736

$206,584

$224,812

$650,132

$6,076 per person

Eligible participants for 12-month housing

NA

NA

39

39

retention

Participant met outcome

NA

NA

27

27

Negative exit

NA

NA

8

8

Positive exit

NA

NA

4

4

Payment for 12-month housing retention at

NA

NA

$707,130

$707,130

$26,190 per person

Total payment

$218,736

$206,584

$931,942

$1,357,262

NA = not available. Q = quarter.

RAND anticipated starting to work on the parallel evaluation in 2021. Through county-level administrative data, it planned to track client service use from before and after housing placement, although the specific outcomes to be evaluated had not been determined by the end of 2018. RAND

also plans to complete a cost analysis. Because of a lack of funding, RAND wil not be conducting an implementation evaluation.

7 2

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

Lane County Implementation

The full implementation of the Lane County PSH project began in September 2018. What follows is a description of the various components of the project implementation phase. Participant identification is a collaboration among the service provider (Sponsors, Inc.), Lane County Parole and Probation, and the local housing authority (Homes for Good). In addition to relying on validated risk/needs assessments, the site is using a modified version of a homeless prioritization tool (the Coordinated Assessment Tool), but has adapted it to assess (and prioritize) the needs of people who are released from prison and, as a result of recent long-term incarceration, would not meet HUD’s definition of

“chronically homeless.” When potential participants are referred to the project, they have 6 months following their prison release date to enter a lease in a project-based unit or a unit subsidized by a project-issued Section 8 Housing Choice voucher. If they do not do so during that time, they are to be placed in the comparison group for the evaluation. Partners expected the pace of participant lease placement to be determined by the availability of housing for the project. The project site does not have any shelter or interim housing placement process for participants while they await a lease placement. As of December 2018, Lane County determined 20 referrals to be eligible and consented for housing, and five people were successful y placed in project-based housing.

Housing

The local public housing authority, Homes for Good, is providing housing for the project. The site is using turnover within a project-based affordable housing project (The Oaks at 14th), targeted toward criminal-justice-involved people, public housing, and housing choice vouchers to provide housing to participants. All participants who enter the program alone (without family) are eligible for housing at The Oaks at 14th. If they do not have an exclusionary conviction (for example, a sex offense or conviction for manufacturing methamphetamine in a federal y funded housing facility), they are also eligible for a housing choice voucher. If they meet the HUD definition of “elderly or disabled,” and they do not have an exclusion conviction, they are also eligible for a one-bedroom public housing unit. Participants who enter with family members and who do not have an exclusionary conviction are eligible for public housing and housing choice placement. The project does not currently have any available housing options for people with exclusionary convictions who seek to live with others.

During the first few months of implementation, turnover was slower than expected at The Oaks at 14th and in the designated public housing facilities, which slowed project enrollment. Additionally, participants struggled to secure leases with housing vouchers in the private market, which project stakeholders attributed to an extremely low vacancy rate (2 percent) and to a reluctance among H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

7 3

landlords to rent to people with criminal histories. As of December 31, 2018, the site was developing strategies for improving the housing search and landlord relationship process, but the slow turnover at The Oaks at 14th and limitations on public housing and on leasing with housing choice vouchers are expected to make enrol ment slower than the partners had predicted during the transaction structuring phase.

Services

The service provider for the project, Sponsors, Inc., has traditionally been focused on providing transitional housing for people with criminal justice histories. Sponsors, Inc. is primarily funded through fee-for-service contracts that draw from state Grant in Aid and JRI dollars. Many stakeholders credit the project’s success to date to its executive director’s experience and vision. Services offered to project participants by Sponsors, Inc. are provided alongside ongoing supervision from parole and probation. This project is the first experience of Sponsors, Inc. with a Housing First model, and because of that, the organization heavily focused on preparing for that shift during the period before full implementation. For example, a lot of consideration was given to the transition to serving a population with no income; Sponsors, Inc. perceived that as the biggest difference between the project’s population and the people it had previously served. Sponsors, Inc. engaged Seattle’s Downtown Emergency Service Center for Housing First training in late 2018, and it will seek continuous training and technical assistance on Housing First for its staff and their counterparts from the housing authority and from parole and probation. In the first few months of implementation, especially because enrol ment was slower than anticipated, Sponsors, Inc. and other project partners focused on ensuring that their model was implemented with fidelity.

7 4

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

V. Conclusion

In the 3 years since HUD and DOJ launched this demonstration, two sites successfully signed a PFS

contract and launched a PSH program, four other sites completed the feasibility phase and are working on structuring the PFS transaction, and one left the demonstration. The two sites currently in the implementation phase, Los Angeles County and Lane County, have been enrol ing and housing homeless single adults. As of December 2018 (3 months into implementation), Lane determined 20 of 43 referrals to be eligible and consented for housing, and five people were successful y placed in project-based housing. Over 14 months from the start of implementation Los Angeles County had placed 182 people. In 2018, the Los Angeles County project reported on housing retention outcomes for three quarters, for a total of over $1.3 million in success payments. Lane County was scheduled to receive the first success payment in mid-2019.

Four sites are in the transaction and structuring phase—Anchorage; Austin/Travis County; Montgomery and Prince George’s Counties; and Rhode Island. Sites continued to define the target population during that phase. At the end of 2018, the sites were still finalizing their payment outcomes, which were focused on metrics for housing stability and recidivism (for example, arrests, bookings, jail days, or prison returns). A few sites hope to include health outcomes as wel , but that is an ongoing challenge, largely because of a lack of research on what outcomes to expect. End payors are an important part of the success outcome discussion, and because many of these sites had not yet confirmed specific end payor organizations or agencies, their success metrics were not final either.

Some sites have had difficulty establishing end payors because of concerns about (and roadblocks to) signing a PFS contract. Some of these chal enges include legislative restrictions on committing future funds, local government concerns that taking on debt could negatively affect credit ratings, and competing priorities for legislators. In some sites, these chal enges are exacerbated by political turnover, which can stal momentum.

Evaluations are a core component of the PFS process, and evaluation partners play an important role in the project. But sites have been reluctant to use random assignment designs, instead opting for quasi-experimental designs with nonrandomized comparison groups. For the two sites with PFS

contracts (Los Angeles County and Lane County), payment triggers are based on the outcomes of program participants alone, without reference to the comparison groups. The quasi-experimental studies, meanwhile, are being conducted over a longer time frame to understand the programs’

impacts, but those studies are not being used for success payments. Using one method to understand H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

7 5

broader impacts and another for payment triggers may result in circumstances in which investors are paying for success that is not ultimately supported in the broader impact evaluation. This could result in decreased credibility of the PFS model.

A central question for Urban’s formative evaluation is: What are the benefits and costs (or advantages and disadvantages) of launching PSH for the target population of high utilizers through a PFS process?

Stakeholders broadly reported that two of the demonstration’s benefits were that it catalyzed partnership and col aboration and that it forced sites to improve cross-sector data sharing. The cross-sector collaboration was necessary for the successful implementation of PSH for the target population and involved different agencies and different levels of government. The HUD-DOJ grants helped motivate this col aborative work. Some stakeholders believed that the PFS process, in particular, served to engage partners and to bring additional stakeholders to the work and that it was increasing awareness of the target population and its high-cost load.

In addition, some stakeholders believed that the PFS project was contributing to positive system change that might extend beyond the demonstration, both by changing traditional thinking about funding and blending funding streams and by encouraging a shift to think about program outcomes—the “success” in PFS—rather than program inputs .

Collaboration across service sectors, levels of government, and public and private stakeholders, however, comes with a considerable burden in terms of time cost. Launching a PSH project, whatever the financing structure, inevitably requires a significant time commitment, as does developing a PFS

project and getting to a transaction. The current report includes qualitative information about stakeholders’ perceptions of the time demands of the project. Although we are not able to quantify the time spent through a rigorous time study, we are asking PFS project partner staff for estimates of the time they are contributing. Respondents’ perceptions seem to suggest that the projects have involved intense time commitments for a core group at each site. Almost three-fourths of respondents to the Annual Partnership survey reported attending more than 5 meetings in the past year, with almost one-third of that group reporting more than 20 meetings attended. In the last quarter of 2018, Urban began collecting data on time costs of the PFS project, using information that each site submits to HUD and quarterly interviews with organizations that do not report their hours to HUD. Results from that work will be presented in future reports.

Whether the benefits from PFS warrant those time costs wil partly depend on whether the sites are successful in launching PSH projects through the demonstration. We wil continue to monitor the

7 6

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

strengths of partnerships through an Annual Partnership Survey and to collect additional data on time use. Future reports will integrate that data with information on the sites’ progress as more of them move into program implementation and on the challenges and successes that each site experiences.

H U D - D OJ PAY F OR S U CC E S S PE R M A NE N T S U PP OR TI V E HO U SI NG D E M ON S TR AT I ON

7 7

Appendix—Stakeholder Benefits

Survey Descriptives

A. Respondent Information

EXHIBIT A.1

What kind of organization do you work for?

Frequency

Percent

Criminal justice

8

11.3

Health and human services

13

18.3

Health or Behavioral Health provider

5

7.0

Housing

7

9.9

Housing provider

3

4.2

Other evaluation or technical assistance organization

11

15.5

Other service providers

4

5.6

Other: please describe

13

18.3

Philanthropy/Foundation

7

9.9

Total

71

100.0

EXHIBIT A.2

Which of the following best describes your organization’s role in the PFS project?

Frequency

Percent

Criminal justice—referral source for the target population 6

8.5

Direct service for the target population

15

21.1

Evaluator

2

2.8

Financial partner

11

15.5

Housing partner for the target population

10

14.1

Intermediary

11

15.5

Other roles: please describe

8

11.3

Other technical assistance

8

11.3

Total

71

100.0

PFS = pay for success.

7 8

A PP E N DI X

EXHIBIT A.3

How long have you been in this position?

Frequency

Percent

Less than 1 year

2

2.8

1–2 years

19

26.8

2–5 years

27

38.0

5–10 years

13

18.3

10–15 years

4

5.6

More than 15 years

6

9.0

Total

71

100.0

EXHIBIT A.4

How long have you been participating in the

PFS Supportive Housing Project?

Frequency

Percent

Less than 6 months

9

12.7

6–12 months

12

16.9

12–18 months

11

15.5

18–24 months

12

16.9

24–30 months

9

12.7

More than 30 months

18

25.0

Total

71

100.0

PFS = pay for success.

EXHIBIT A.5

In the past calendar year, how many meetings have

you attended about the PFS Supportive Housing Project?

Frequency

Percent

0 meetings

1

1.4

1–4 meetings

12

16.9

11–20 meetings

7

9.9

5–10 meetings

35

49.3

More than 20 meetings

16

22.5

Total

71

100.0

PFS = pay for success.

EXHIBIT A.6

What is the current phase of this PFS project?

Frequency

Percent

Feasibility assessment

5

7.1

Project implementation

30

42.9

Transaction structuring

35

50.0

Total

70

100.0

7 9

PFS = pay for success.

8 0

A PP E N DI X

B. To what extent would you agree with the following

statements about your Pay for Success project?

EXHIBIT B.1

I have a clear understanding of my organization’s

roles and responsibilities within the PFS project.

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

2

2.8

*Agree somewhat

15

21.1

Agree completely

52

73.2

Not applicable

1

1.4

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT B.2

I have a clear understanding of other organizations’

roles and responsibilities within the PFS project.

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

6

8.5

*Agree somewhat

24

33.8

Agree completely

39

54.9

Not Applicable

1

1.4

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT B.3

The PFS project is meeting its aims and objectives

for its current phase.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

11

15.5

*Agree somewhat

26

36.6

Agree completely

24

33.8

Don’t Know

9

12.7

Not Applicable

1

1.4

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

8 1

8 2

A PP E N DI X

EXHIBIT B.4

The PFS project’s work is being guided by information

and data.

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

0

0.0

*Agree somewhat

21

29.6

Agree completely

44

62.0

Don’t Know

4

5.6

Not Applicable

1

1.4

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

C. Pay for Success Organization. To what extent would

you agree with the following statements about your

organization’s role in the project?

EXHIBIT C.1

My organization plays an active role in the

PFS Supportive Housing Project.

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

7

9.9

*Agree somewhat

16

22.5

Agree completely

46

64.8

Not applicable

1

1.4

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT C.2

My organization has a stake in increasing supportive

housing for the target population.

Frequency

Percent

Disagree completely

2

2.8

*Disagree somewhat

3

4.2

*Agree somewhat

8

11.3

Agree completely

54

76.1

Don’t Know

1

1.4

Not Applicable

3

4.2

8 3

Total

71

100.0

*Unlabeled choices on the survey 4-point scale

8 4

A PP E N DI X

EXHIBIT C.3

Leadership in my organization is aware of the issues

surrounding the target population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

2

2.8

*Agree somewhat

10

14.1

Agree completely

57

80.3

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT C.4

Leaders in my organization are committed to

addressing the needs of the target population.

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

1

1.4

*Agree somewhat

14

19.7

Agree completely

53

74.7

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT C.5

Staff members in my organization are aware of the

issues surrounding the target population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

7

9.9

*Agree somewhat

20

28.2

Agree completely

43

60. 6

Not applicable

1

1.4

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

8 5

D. Pay for Success Partners. Would you agree with the following statements about the PFS partnership?

EXHIBIT D.1

Partners Share a Common Vision

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

5

7.0

*Agree somewhat

21

29.6

Agree completely

38

53.5

Don't Know

7

9.9

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT D.2

Partners Have a Common Understanding of PFS

and Its Role in Supporting PSH

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

9

12.7

*Agree somewhat

26

36.6

Agree completely

32

45.1

Don't Know

4

5.6

Total

71

100.0

PFS = pay for success. PSH = permanent supportive housing.

* Unlabeled choices on the survey 4-point scale

EXHIBIT D.3

Partners Have a Shared Understanding That PSH

Is Important for Target Population

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

1

1.4

*Agree somewhat

10

14.1

Agree completely

57

80.3

Don't Know

3

4.2

Total

71

100.0

PSH = permanent supportive housing.

* Unlabeled choices on the survey 4-point scale

8 6

A PP E N DI X

EXHIBIT D.4

Partners Share Understanding That PFS Is a Promising Approach to Establish and/or Support PSH for Target Population

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

12

16.9

*Agree somewhat

15

21.1

Agree completely

39

54.9

Don't Know

5

7.0

Total

71

100.0

PFS = pay for success. PSH = permanent supportive housing.

* Unlabeled choices on the survey 4-point scale

EXHIBIT D.5

Partners Are Willing to Make Changes to Achieve Shared Goals

Frequency

Percent

Disagree completely

0

0.00

*Disagree somewhat

10

14.08

*Agree somewhat

20

28.17

Agree completely

34

47.89

Don't Know

7

9.86

Total

71

100.00

* Unlabeled choices on the survey 4-point scale

EXHIBIT D.6

Partners Have Mutual Trust and Respect

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

5

7.0

*Agree somewhat

29

40.9

Agree completely

33

46.5

Don't Know

4

5.6

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT D.7

Partners Work Well Together as a Group

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

4

5.6

*Agree somewhat

24

33.8

Agree completely

39

54.9

Don't Know

3

4.2

Total

71

100.0

8 7

* Unlabeled choices on the survey 4-point scale

8 8

A PP E N DI X

EXHIBIT D.8

Partners Make High-Level Decisions through a Collaborative Process

Frequency

Percent

Disagree completely

2

2.8

*Disagree somewhat

7

9.9

*Agree somewhat

30

42.3

Agree completely

27

38.0

Don't Know

5

7.0

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

E. Community Support. Would you agree with the

following statements about the community support for

the project?

EXHIBIT E.1

Housing the target population is a high priority for the community.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

3

4.2

*Agree somewhat

31

43.7

Agree completely

33

46.5

Don't Know

4

5.6

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT E.2

There are organizations in the community with expertise and experience in providing supportive housing for the target population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

5

7.0

*Agree somewhat

13

18.3

Agree completely

52

73.2

Don't Know

1

1.4

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

8 9

EXHIBIT E.3

The PFS project is bringing local partners together

across sectors to improve results for the target population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

8

11.3

*Agree somewhat

15

21.1

Agree completely

43

60.6

Don't Know

4

5.6

Not applicable

1

1.4

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT E.4

Over the past year, the community has acquired new sources of funding for rent, operating subsidies, or supportive services for the target population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

13

18.3

*Agree somewhat

15

21.1

Agree completely

29

40.9

Don't Know

12

16.9

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

F. Collaboration. Would you agree with the following

statements about Permanent Supportive Housing

providers in your community?

EXHIBIT F.1

Over the past year, local supportive housing providers were able to collaborate with other agencies to promote housing stability and well-being for the target population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

6

8.5

*Agree somewhat

26

36.6

Agree completely

27

38.0

Don't Know

10

14.1

Not applicable

2

2.8

9 0

A PP E N DI X

Total

71

100.0

*Unlabeled choices on the survey 4-point scale

9 1

EXHIBIT F.2

Over the past year, local supportive housing providers have been working with health and behavioral health providers to facilitate access to benefits and services.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

8

11.3

*Agree somewhat

32

45.1

Agree completely

22

31.0

Don't Know

8

11.3

Not applicable

1

1.4

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

G. Collaboration. Over the past year, how much have the

following systems collaborated with each other in

working with the target population?

EXHIBIT G.1

Law Enforcement with the Homeless Assistance System

Frequency Percent

No collaboration

0

0.0

*Little collaboration

12

16.9

*Occasional collaboration

21

29.6

Extensive collaboration

19

26.8

Don’t know

17

23.9

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT G.2

Courts with the Homeless Assistance System

Frequency Percent

No collaboration

1

1.4

*Little collaboration

14

19.7

*Occasional collaboration

16

22.5

Extensive collaboration

11

15.5

Don’t know

26

36.6

Not applicable

3

4.2

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

9 2

A PP E N DI X

EXHIBIT G.3

Jail(s) with the Homeless Assistance System

Frequency Percent

No collaboration

1

1.4

*Little collaboration

12

16.9

*Occasional collaboration

25

35.2

Extensive collaboration

12

16.9

Don’t know

18

25.4

Not applicable

3

4.2

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT G.4

Law Enforcement with Health/Behavioral Health Service Providers

Frequency Percent

No collaboration

1

1.4

*Little collaboration

9

12.7

*Occasional collaboration

25

35.2

Extensive collaboration

14

19.7

Don’t know

20

28.2

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT G.5

Courts with Health/Behavioral Health Service Providers

Frequency Percent

No collaboration

1

1.4

*Little collaboration

9

12.7

*Occasional collaboration

18

25.4

Extensive collaboration

15

21.1

Don’t know

25

35.2

Not applicable

3

4.2

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT G.6

Homeless Assistance with Health/Behavioral Health Service Providers

Frequency Percent

No collaboration

0

0.0

*Little collaboration

6

8.5

*Occasional collaboration

28

39.4

Extensive collaboration

21

29.6

Don’t know

14

19.7

Not applicable

2

2.8

9 3

Total

71

100.0

*Unlabeled choices on the survey 4-point scale

9 4

A PP E N DI X

H. Collaboration. Over the past year, has collaboration

increased or decreased?

EXHIBIT H.1

Data Sharing Between Organizations Serving the Target Population

Frequency

Percent

Decreased

0

0.0

Stayed the same

13

18.3

Increased

43

60.6

Don’t know

15

21.1

Total

71

100.0

EXHIBIT H.2

Number of Formal Agreements or Memoranda of Understanding Between Organizations Serving the Target Population

Frequency

Percent

Decreased

0

0.0

Stayed the same

22

31.0

Increased

28

39.4

Don’t know

21

29.6

Total

71

100.0

EXHIBIT H.3

Frequency of Meetings about Effectively Working with the Target Population

Frequency

Percent

Decreased

1

1.4

Stayed the same

13

18.3

Increased

39

54.9

Don’t know

18

25.4

Total

71

100.0

EXHIBIT H.4

Number of PFS Partner Organizations Represented

on the Local Criminal Justice Coordinating Committee

Frequency

Percent

Decreased

0

0.0

Stayed the same

16

22.5

Increased

14

19.7

Don’t know

37

52.1

Not applicable

4

5.6

Total

71

100.0

9 5

PFS = pay for success.

9 6

A PP E N DI X

I. Collaboration. Over the past year, did any of the

following pose a serious problem for PFS partners

collaborating?

EXHIBIT I.1

Competition for Resources or “Turf Issues”

Frequency

Percent

Not a problem

17

23.9

*Minor problem

20

28.2

*Moderate problem

9

12.7

Serious problem

3

4.2

Don’t know

20

28.2

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT I.2

Conflicting Priorities and Visions

Frequency

Percent

Not a problem

19

26.8

*Minor problem

18

25.4

*Moderate problem

15

21.1

Serious problem

5

7.0

Don’t know

13

18.3

Not applicable

1

1.4

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT I.3

Lack of Trust among Agencies

Frequency

Percent

Not a problem

30

42.3

*Minor problem

19

26.8

*Moderate problem

11

15.5

Serious problem

1

1.4

Don’t know

9

12.7

Not applicable

1

1.4

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

9 7

EXHIBIT I.4

Absence of Established Working Relationships

Frequency

Percent

Not a problem

24

33.8

*Minor problem

27

38.0

*Moderate problem

7

9.9

Serious problem

1

1.4

Don’t know

10

14.1

Not applicable

2

2.8

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT I.5

Skepticism or Disagreement about the Housing First Approach

Frequency

Percent

Not a problem

29

40.9

*Minor problem

20

28.2

*Moderate problem

7

9.9

Serious problem

4

5.6

Don’t know

10

14.1

Not applicable

1

1.4

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

J. Collaboration. Over the past year, how often has your

organization engaged in the following activities with

other organizations serving as partners in the PFS

project?

EXHIBIT J.1

We collaborated with PFS partners to leverage resources (for example, write grants together).

Frequency

Percent

Never

12

17.1

*Rarely

14

20.0

*Occasionally

17

24.3

Frequently

19

27.1

Don’t know

3

4.3

Not applicable

5

7.1

Total

70

100.0

9 8

A PP E N DI X

PFS = pay for success.

*Unlabeled choices on the survey 4-point scale

9 9

EXHIBIT J.2

We collaborated with PFS partners to provide

training or educational opportunities.

Frequency

Percent

Never

15

21.1

*Rarely

10

14.1

*Occasionally

20

28.2

Frequently

13

18.3

Don’t know

8

11.3

Not applicable

5

7.0

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT J.3

We shared resources (for example, materials or

equipment) with PFS partners.

Frequency

Percent

Never

7

9.9

*Rarely

9

12.7

*Occasionally

18

25.4

Frequently

25

35.2

Don’t know

4

5.6

Not applicable

8

11.3

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT J.4

We provided information about our programs or

services to PFS partners.

Frequency

Percent

Never

2

2.8

*Rarely

5

7.0

*Occasionally

19

26.8

Frequently

39

54.9

Don’t know

1

1.4

Not applicable

5

7.0

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

1 0 0

A PP E N DI X

EXHIBIT J.5

We met with PFS partners to share information

about services, resources, or clients.

Frequency

Percent

Never

3

4.2

*Rarely

11

15.5

*Occasionally

16

22.5

Frequently

31

43.7

Don’t know

2

2.8

Not applicable

8

11.3

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

EXHIBIT J.6

We collaborated with PFS partners to provide direct

services to individuals.

Frequency

Percent

Never

15

21.1

*Rarely

6

8.5

*Occasionally

8

11.3

Frequently

21

29.6

Don’t know

4

5.6

Not applicable

17

23.9

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

K. Data Sharing. Would you agree with the following

statements about data sharing, over the past year?

EXHIBIT K.1

PFS Partners Used Data to Identify the Target Population

Frequency

Percent

Disagree completely

1

1.4

*Somewhat disagree

3

4.2

*Somewhat agree

15

21.1

Agree completely

48

67.6

Don’t know

3

4.2

Not applicable

1

1.4

Total

71

100.0

1 0 1

PFS = pay for success.

*Unlabeled choices on the survey 4-point scale

1 0 2

A PP E N DI X

EXHIBIT K.2

PFS partners have used data to understand the level

of housing and services needed by the target population.

Frequency

Percent

Disagree completely

1

1.4

*Somewhat disagree

2

2.8

*Somewhat agree

19

26.8

Agree completely

44

62.0

Don’t know

5

7.0

Total

71

100.0

PFS = pay for success.

* Unlabeled choices on the survey 4-point scale

L. Performance Data. Would you agree with the

following statements about the use of evidence to

manage supportive housing providers over the past

year?

EXHIBIT L.1

Supportive housing providers have used performance

data to identify activities needing improvement.

Frequency

Percent

Disagree completely

0

0.0

*Somewhat disagree

6

8.5

*Somewhat agree

23

32.4

Agree completely

15

21.1

Don’t know

21

29.6

Not applicable

6

8.5

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

1 0 3

EXHIBIT L.2

Supportive housing providers have followed evidence-based models of service delivery (for example, Housing First, harm reduction, motivational interviewing).

Frequency

Percent

Disagree completely

0

0.0

*Somewhat disagree

3

4.2

*Somewhat agree

22

31.0

Agree completely

28

39.4

Don’t know

13

18.3

Not applicable

5

7.0

Total

71

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT L.3

Government agencies have used performance data to assess whether supportive housing programs are improving client outcomes.

Frequency

Percent

Disagree completely

1

1.4

*Somewhat disagree

12

17.1

*Somewhat agree

16

22.9

Agree completely

19

27.1

Don’t know

16

22.9

Not applicable

6

8.6

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT L.4

Does your organization directly serve the target population?

Frequency

Percent

No

39

54.9

Yes

32

45.1

Total

71

100.0

1 0 4

A PP E N DI X

M. Data Sharing. Over the past year, how often has your

organization shared or received information about the

clients in the target population?

EXHIBIT M.1

We have shared information with other organizations

about clients in the target population.

Frequency

Percent

Never

2

6.5

*Rarely

1

3.2

*Occasionally

6

19.4

Frequently

19

61.3

Don’t know

3

9.7

Total

31

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT M.2

We have received information from other organizations

about clients in the target population.

Frequency

Percent

Never

0

0.0

*Rarely

4

12.9

*Occasionally

11

35.5

Frequently

14

45.2

Don’t know

2

6.5

Total

31

100.0

* Unlabeled choices on the survey 4-point scale

1 0 5

N. Data Sharing. Over the past year, has your organization shared criminal justice information with

other organizations serving the target population?

EXHIBIT N.1

A Client’s Criminal History

Frequency

Percent

Never

4

12.5

*Rarely

3

9.4

*Occasionally

6

18.8

Frequently

10

31.3

Don’t know

5

15.6

Not applicable

4

12.5

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT N.2

Whether a Client Was Recently Released from Jail

Frequency

Percent

Never

2

6.3

*Rarely

2

6.3

*Occasionally

7

21.9

Frequently

14

43.8

Don’t know

3

9.4

Not applicable

4

12.5

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT N.3

Whether a Client Was Recently Released from State or Federal Prison

Frequency

Percent

Never

5

15.6

*Rarely

4

12.5

*Occasionally

5

15.6

Frequently

10

31.3

Don’t know

3

9.4

Not applicable

5

15.6

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

1 0 6

A PP E N DI X

EXHIBIT N.4

Whether a Client is under Community Supervision (for example, Probation, Parole)

Frequency

Percent

Never

2

6.3

*Rarely

2

6.3

*Occasionally

6

18.8

Frequently

14

43.8

Don’t know

4

12.5

Not applicable

4

12.5

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

O. Data Sharing. Over the past year, has your

organization shared client information with other

organizations serving the target population?

EXHIBIT O.1

Client Referrals

Frequency

Percent

Never

1

3.1

*Rarely

2

6.3

*Occasionally

7

21.9

Frequently

19

59.4

Don’t know

2

6.3

Not applicable

1

3.1

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT O.2

Whether a Client Is Being Served by Other Organizations

Frequency

Percent

Never

2

6.5

*Rarely

0

0.0

*Occasionally

7

22.6

Frequently

17

54.8

Don’t know

3

9.7

Not applicable

2

6.5

Total

31

100.0

* Unlabeled choices on the survey 4-point scale

1 0 7

EXHIBIT O.3

What Services a Client Is Receiving from Other Organizations

Frequency

Percent

Never

3

9.4

*Rarely

1

3.1

*Occasionally

8

25.0

Frequently

14

43.8

Don’t know

4

12.5

Not applicable

2

6.3

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT O.4

Client Assessments Conducted by Your Organization or Another Organization

Frequency

Percent

Never

2

6.3

*Rarely

4

12.5

*Occasionally

5

15.6

Frequently

15

46.9

Don’t know

4

12.5

Not applicable

2

6.3

Total

32

100.0

*

EXHIBIT O.5

Client Outcomes (for example, Days in Housing, Connection to Health Services)

Frequency

Percent

Never

3

9.4

*Rarely

7

21.9

*Occasionally

6

18.8

Frequently

11

34.4

Don’t know

4

12.5

Not applicable

1

3.1

Total

32

100.0

* Unlabeled choices on the survey 4-point scale

1 0 8

A PP E N DI X

P. Service Challenges. Over the past year, has your PFS

project experienced any of the following as serious

problems for providing services to the target

population?

EXHIBIT P.1

Access to Clients in Jail

Frequency

Percent

Not a problem

13

18.6

*Minor problem

14

20.0

*Moderate problem

6

8.6

Serious problem

0

0.0

Don’t know

19

27.1

Not applicable

18

25.7

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT P.2

Coordination of Entry for Homeless Services

Frequency

Percent

Not a problem

15

21.4

*Minor problem

17

24.3

*Moderate problem

7

10.0

Serious problem

4

5.7

Don’t know

14

20.0

Not applicable

13

18.6

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT P.3

Organizational Policies about Data Sharing

Frequency

Percent

Not a problem

17

24.3

*Minor problem

16

22.9

*Moderate problem

13

18.6

Serious problem

4

5.7

Don’t know

12

17.1

Not applicable

8

11.4

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

1 0 9

EXHIBIT P.4

Obtaining Client Releases to Share Information across Organizations

Frequency

Percent

Not a problem

16

22.9

*Minor problem

17

24.3

*Moderate problem

3

4.3

Serious problem

3

4.3

Don’t know

18

25.7

Not applicable

13

18.6

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT P.5

Accessing Reliable Assessment Information

Frequency

Percent

Not a problem

10

14.3

*Minor problem

14

20.0

*Moderate problem

14

20.0

Serious problem

1

1.4

Don’t know

18

25.7

Not applicable

13

18.6

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT P.6

Accessing Other Relevant Data

Frequency

Percent

Not a problem

11

15.7

*Minor problem

13

18.6

*Moderate problem

16

22.9

Serious problem

1

1.4

Don’t know

16

22.9

Not applicable

13

18.6

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT P.7

Data Systems Compatibility

Frequency

Percent

Not a problem

6

8.6

*Minor problem

10

14.3

*Moderate problem

13

18.6

Serious problem

10

14.3

Don’t know

22

31.4

Not applicable

9

12.9

1 1 0

A PP E N DI X

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

1 1 1

EXHIBIT P.8

Other Technological Limitations

Frequency

Percent

Not a problem

8

11.4

*Minor problem

11

15.7

*Moderate problem

8

11.4

Serious problem

6

8.6

Don’t know

25

35.7

Not applicable

12

17.1

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT P.9

Limited Time and Resources

Frequency

Percent

Not a problem

3

4.3

*Minor problem

11

15.7

*Moderate problem

24

34.3

Serious problem

9

12.9

Don’t know

14

20.0

Not applicable

9

12.9

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

Q. Service Challenges. Over the past year, have any of

the following posed a serious problem for providing

services to the target population for your PFS project?

EXHIBIT Q.1

Waiting Lists for Services for the Target Population (for example, Housing, Substance Abuse Treatment, Mental Health Programs)

Frequency

Percent

Not a problem

6

8.7

*Minor problem

4

5.8

*Moderate problem

13

18.8

Serious problem

17

24.6

Don’t know

16

23.2

Not applicable

13

18.8

Total

69

100.0

* Unlabeled choices on the survey 4-point scale

1 1 2

A PP E N DI X

1 1 3

EXHIBIT Q.2

Lack of Access to Housing for the Target Population

Frequency

Percent

Not a problem

1

1.5

*Minor problem

4

5.8

*Moderate problem

21

30.4

Serious problem

19

27.5

Don’t know

11

15.9

Not applicable

13

18.8

Total

69

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT Q.3

Lack of Access to Substance Abuse Treatment for the Target Population

Frequency

Percent

Not a problem

4

5.8

*Minor problem

10

14.5

*Moderate problem

10

14.5

Serious problem

15

21.7

Don’t know

16

23.2

Not applicable

14

20.3

Total

69

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT Q.4

Lack of Access to Mental Health Programs for the Target Population

Frequency

Percent

Not a problem

4

5.8

*Minor problem

13

18.8

*Moderate problem

10

14.5

Serious problem

13

18.8

Don’t know

15

21.7

Not applicable

14

20.3

Total

69

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT Q.5

Rigid Eligibility Requirements for Housing for the Target Population

Frequency

Percent

Not a problem

7

10.1

*Minor problem

8

11.6

*Moderate problem

13

18.8

Serious problem

11

15.9

Don’t know

17

24.6

Not applicable

13

18.8

1 1 4

A PP E N DI X

Total

69

100.0

* Unlabeled choices on the survey 4-point scale

1 1 5

EXHIBIT Q.6

Policies Excluding Certain Kinds of Offenders from Housing or Services

Frequency

Percent

Not a problem

6

8.7

*Minor problem

7

10.1

*Moderate problem

10

14.5

Serious problem

14

20.3

Don’t know

19

27.5

Not applicable

13

18.8

Total

69

100.0

* Unlabeled choices on the survey 4-point scale

R. Community Infrastructure. Would you agree with the

following statements about your community?

EXHIBIT R.1

The community is committed to reducing barriers to

housing for the target population, including accepting

housing applications for individuals with criminal justice histories.

Frequency

Percent

Disagree completely

2

2.9

*Disagree somewhat

7

10.0

*Agree somewhat

32

45.7

Agree completely

16

22.9

Don’t know

11

15.7

Not applicable

2

2.9

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT R.2

The community has a process to prioritize new and

turnover housing units for the target population.

Frequency

Percent

Disagree completely

3

4.3

*Disagree somewhat

16

22.9

*Agree somewhat

15

21.4

Agree completely

12

17.1

Don’t know

22

31.4

Not applicable

2

2.9

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

1 1 6

A PP E N DI X

1 1 7

EXHIBIT R.3

Supportive housing providers have staff dedicated to

landlord recruitment and relations.

Frequency

Percent

Disagree completely

1

1.4

*Disagree somewhat

13

18.6

*Agree somewhat

20

28.6

Agree completely

17

24.3

Don’t know

16

22.9

Not applicable

3

4.3

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT R.4

Supportive housing providers are able to bill Medicaid

for covered services.

Frequency

Percent

Disagree completely

5

7.1

*Disagree somewhat

20

28.6

*Agree somewhat

9

12.9

Agree completely

5

7.1

Not at all

1

1.4

Don’t know

27

38.6

Not applicable

3

4.3

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

EXHIBIT R.5

Partners are working to educate policymakers and

local elected officials about the need for supportive

housing for a chronically homeless jail reentry population.

Frequency

Percent

Disagree completely

0

0.0

*Disagree somewhat

6

8.6

*Agree somewhat

18

25.7

Agree completely

32

45.7

Don’t know

11

15.7

Not applicable

3

4.3

Total

70

100.0

* Unlabeled choices on the survey 4-point scale

1 1 8

A PP E N DI X

References

Aidala, Angela A., William McAllister, Maiko Yomogida, and Virginia Shubert. 2014. Frequent Users Service

Enhancement “FUSE” Initiative: New York City FUSE II Evaluation Report. New York, NY: Columbia University Mailman School of Public Health.

Anchorage Daily News. 2018. “Federal Disaster Declared for Southcentral Earthquake,” Anchorage Daily News, December 2.

Brooks, James. 2018. “Gov. Dunleavy’s Top Public Safety Offices Remain Empty as State Layoffs Take Effect,”

Anchorage Daily News, December 4.

Culhane, Dennis. 2018. Chronic Homelessness. Rockvil e, MD: Center for Evidence-based Solutions to Homelessness, Abt Associates.

Cunningham, Mary, Ruth Gourevitch, Mike Pergamit, Sarah Gil espie, Devlin Hanson, Tracey O’Brien, Christine Velez, Daniel Brisson, Gary Sanford, and Abby Magnus. 2018. From Homelessness to Housed: Interim Lessons

from the Denver Supportive Housing Social Impact Bond Initiative. Washington, DC: Urban Institute.

DeMarban, Alex, and Annie Zak. 2018. “Walker Team ‘Strongly’ Objects as Dunleavy Transition Asks All At-Will

Alaska State Workers for Resignation Letters,” Anchorage Daily News, November 19.

Gal oway, Ian. 2015. “Pay for Success: Unlocking Government Funding for Prevention.” National Council on Crime and Delinquency blog, January 27.

Gillespie, Sarah, Samantha Batko, and Akiva Liberman. 2018. “Data Use and Challenges in Using Pay for Success

to Implement Permanent Supportive Housing: Lessons from the HUD-DOJ Demonstration.” Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Henry, Meghan, Anna Mahathey, Tyler Morril , Anna Robinson, Azim Shivji, and Rian Watt. 2018. The 2018 Annual

Homelessness Assessment Report (AHAR) to Congress, Part 1: Point-in-Time Estimates of Homelessness.

Washington, DC: U.S. Department of Housing and Urban Development, Office of Community Planning and Development.

Kaeble, Danielle, and Mary Cowhig. 2018. Correctional Populations in the United States, 2016. Washington, DC: U.S. Department of Justice, Office of Justice Programs, Bureau of Justice Statistics.

Liberman, Akiva, Mary Cunningham, Sarah Gillespie, Samantha Batko, Brian Bieretz, Matt Eldridge, Evelyn McCoy, Nicole DuBois, and Alexandra Ricks. 2019. Evaluation of the HUD-DOJ Pay for Success Permanent Supportive

Housing Demonstration: Baseline Report. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Milner, Justin, and Kel y Walsh. 2016 . Measuring Success in Pay for Success: Randomized Controlled Trials as the

Starting Point. Washington, DC: Urban Institute.

Nonprofit Finance Fund (NFF). 2019a. Pay for Success: The First 25. New York, NY: Nonprofit Finance Fund.

———. 2019b. “Pay for Success Activity Map.” New York, NY: Nonprofit Finance Fund.

Roman, John. 2015. Solving the Wrong Pockets Problem: How Pay for Success Promotes Investment in Evidence-Based Best Practices. Washington, DC: Urban Institute.

U.S. Department of Housing and Urban Development (HUD). 2015 . Pay for Success Permanent Supportive Housing

Demonstration. Notice of Funding Availability FR-5900-N-26. Washington, DC: U.S. Department of Housing and Urban Development.

U.S. Government Accountability Office (GAO). 2015. Pay for Success: Collaboration among Federal Agencies Would

Be Helpful as Governments Explore New Financing Mechanisms. Report GAO-15-646. Washington, DC: U.S.

Government Accountability Office.

U.S. Interagency Council on Homelessness (USICH). 2016. “Reducing Criminal Justice System Involvement among

People Experiencing Homelessness . ” Washington, DC: U.S. Interagency Council on Homelessness.

1 2 0

A PP E N DI X

U.S. Department of Housing and Urban Development Office of Policy Development and Research

Washington, DC 20410-6000

January 2021

Document Outline

- Foreword

- Contents

- Executive Summary

- Background

- Cross-Site Findings on Feasibility Analysis and Transaction Structuring

- Feasibility Reports

- Building Pay for Success Partnerships

- Defining the Target Population

- Establishing the Financial Feasibility of Pay for Success

- Establishing the Feasibility of Implementing Permanent Supportive Housing

- Establishing Success Metrics and Evaluation

- Implementing PSH

- Los Angeles County

- Lane County

- The Benefits and Costs of Using Pay for Success to Launch Permanent Supportive Housing

- I. Introduction

- Overview

- Background

- Permanent Supportive Housing

- Expanding Permanent Supportive Housing Using Pay for Success

- Pay for Success Partners

- The HUD-DOJ Pay for Success Permanent Supportive Housing Demonstration

- Demonstration Theory of Change

- Demonstration Requirements and Phases

- Demonstration Sites

- Year 2 Evaluation Questions and Methods

- Data Collection Activities

- II. Progress at Sites in the Transaction Structuring Phase

- Anchorage, Alaska

- Austin/Travis County, Texas

- Lane County, Oregon

- Montgomery and Prince George’s Counties, Maryland

- Rhode Island

- III. Cross-Site Findings on Feasibility Analysis and Transaction Structuring

- Feasibility Reports

- A. Building PFS Partnerships

- Collaboration and Partnership

- Data Sharing and Data Use

- Time Burden

- B. Defining the Target Population

- Completing Data Analyses

- C. Establishing the Financial Feasibility of Pay for Success

- Feasibility Reports’ Estimates of Costs and Possible Savings

- Net Cost Savings and Other Motivations for Pay for Success

- Potential Savings from Successful Permanent Supportive Housing Implementation

- Identifying and Securing End Payors

- The Costs of Implementing Permanent Supportive Housing

- Finding Investors and Managing Risk

- Political and Legal Complexities

- D. Establishing the Feasibility of Implementing Permanent Supportive Housing

- Participant Identification and Enrollment Pathways

- Pilot Periods

- Service Provider and Housing Capacity

- Housing First

- Permanent Supportive Housing and Criminal Justice

- Ongoing Criminal Justice Reforms

- Other Implementation Issues related to Criminal Justice

- Housing First and Criminal Justice

- E. Establishing Success Metrics and Evaluation

- Success Payments

- Evaluation

- Challenges with Evaluations

- IV. Implementing PSH

- Los Angeles County Implementation

- Participant Enrollment and Housing

- Service Provision

- Implementation Challenges and Criminal Justice Involvement

- Success Payments

- Lane County Implementation

- Housing

- Services

- V. Conclusion

- Appendix—Stakeholder Benefits Survey Descriptives

- A. Respondent Information

- B. To what extent would you agree with the following statements about your Pay for Success project?

- C. Pay for Success Organization. To what extent would you agree with the following statements about your organization’s role in the project?

- D. Pay for Success Partners. Would you agree with the following statements about the PFS partnership?

- E. Community Support. Would you agree with the following statements about the community support for the project?

- F. Collaboration. Would you agree with the following statements about Permanent Supportive Housing providers in your community?

- G. Collaboration. Over the past year, how much have the following systems collaborated with each other in working with the target population?

- H. Collaboration. Over the past year, has collaboration increased or decreased?

- I. Collaboration. Over the past year, did any of the following pose a serious problem for PFS partners collaborating?

- J. Collaboration. Over the past year, how often has your organization engaged in the following activities with other organizations serving as partners in the PFS project?

- K. Data Sharing. Would you agree with the following statements about data sharing, over the past year?

- L. Performance Data. Would you agree with the following statements about the use of evidence to manage supportive housing providers over the past year?

- M. Data Sharing. Over the past year, how often has your organization shared or received information about the clients in the target population?

- N. Data Sharing. Over the past year, has your organization shared criminal justice information with other organizations serving the target population?

- O. Data Sharing. Over the past year, has your organization shared client information with other organizations serving the target population?

- P. Service Challenges. Over the past year, has your PFS project experienced any of the following as serious problems for providing services to the target population?

- Q. Service Challenges. Over the past year, have any of the following posed a serious problem for providing services to the target population for your PFS project?

- R. Community Infrastructure. Would you agree with the following statements about your community?

- References

Table of Contents

Foreword

Contents

Executive Summary

Background

Cross-Site Findings on Feasibility Analysis and Transaction Structuring

Feasibility Reports

Building Pay for Success Partnerships

Defining the Target Population

Establishing the Financial Feasibility of Pay for Success

Establishing the Feasibility of Implementing Permanent Supportive Housing

Establishing Success Metrics and Evaluation

Implementing PSH

Los Angeles County

Lane County

The Benefits and Costs of Using Pay for Success to Launch Permanent Supportive Housing

I. Introduction

Overview

Background

Permanent Supportive Housing

Expanding Permanent Supportive Housing Using Pay for Success

Pay for Success Partners

The HUD-DOJ Pay for Success Permanent Supportive Housing Demonstration

Demonstration Theory of Change

Demonstration Requirements and Phases

Demonstration Sites

Year 2 Evaluation Questions and Methods

Data Collection Activities

II. Progress at Sites in the Transaction Structuring Phase

Anchorage, Alaska

Austin/Travis County, Texas

Lane County, Oregon

Montgomery and Prince George’s Counties, Maryland

Rhode Island

III. Cross-Site Findings on Feasibility Analysis and Transaction Structuring

Feasibility Reports

A. Building PFS Partnerships

Collaboration and Partnership

Data Sharing and Data Use

Time Burden

B. Defining the Target Population

Completing Data Analyses

C. Establishing the Financial Feasibility of Pay for Success

Feasibility Reports’ Estimates of Costs and Possible Savings

Net Cost Savings and Other Motivations for Pay for Success

Potential Savings from Successful Permanent Supportive Housing Implementation

Identifying and Securing End Payors

The Costs of Implementing Permanent Supportive Housing

Finding Investors and Managing Risk

Political and Legal Complexities

D. Establishing the Feasibility of Implementing Permanent Supportive Housing

Participant Identification and Enrollment Pathways

Pilot Periods

Service Provider and Housing Capacity

Housing First

Permanent Supportive Housing and Criminal Justice

Ongoing Criminal Justice Reforms

Other Implementation Issues related to Criminal Justice

Housing First and Criminal Justice

E. Establishing Success Metrics and Evaluation

Success Payments

Evaluation

Challenges with Evaluations

IV. Implementing PSH

Los Angeles County Implementation

Participant Enrollment and Housing

Service Provision

Implementation Challenges and Criminal Justice Involvement

Success Payments

Lane County Implementation

Housing

Services

V. Conclusion

Appendix—Stakeholder Benefits Survey Descriptives

A. Respondent Information

B. To what extent would you agree with the following statements about your Pay for Success project?

C. Pay for Success Organization. To what extent would you agree with the following statements about your organization’s role in the project?

D. Pay for Success Partners. Would you agree with the following statements about the PFS partnership?

E. Community Support. Would you agree with the following statements about the community support for the project?

F. Collaboration. Would you agree with the following statements about Permanent Supportive Housing providers in your community?

G. Collaboration. Over the past year, how much have the following systems collaborated with each other in working with the target population?

H. Collaboration. Over the past year, has collaboration increased or decreased?

I. Collaboration. Over the past year, did any of the following pose a serious problem for PFS partners collaborating?

J. Collaboration. Over the past year, how often has your organization engaged in the following activities with other organizations serving as partners in the PFS project?

K. Data Sharing. Would you agree with the following statements about data sharing, over the past year?

L. Performance Data. Would you agree with the following statements about the use of evidence to manage supportive housing providers over the past year?

M. Data Sharing. Over the past year, how often has your organization shared or received information about the clients in the target population?

N. Data Sharing. Over the past year, has your organization shared criminal justice information with other organizations serving the target population?

O. Data Sharing. Over the past year, has your organization shared client information with other organizations serving the target population?

P. Service Challenges. Over the past year, has your PFS project experienced any of the following as serious problems for providing services to the target population?

Q. Service Challenges. Over the past year, have any of the following posed a serious problem for providing services to the target population for your PFS project?

R. Community Infrastructure. Would you agree with the following statements about your community?

References