Using HUD

Administrative Data to

Estimate Success Rates

and Search Durations

for New Voucher

Recipients

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

Disclaimer

The contents of this report are the views of the authors and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S.

Government. This research does not represent the institutional views of research funders, New York University, NYU School of Law, or the Wagner Graduate School of Public Service. Funders do not determine research findings or recommendations in research and policy reports by the NYU Furman Center.

Using HUD Administrative

Data to Estimate Success

Rates and Search Durations

for New Voucher Recipients

Prepared for

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Prepared by

Ingrid Gould El en

Katherine O’Regan

Sarah Strochak

Final Report | December 2021

Acknowledgments

We thank the Melvil e Charitable Trust, the Department of Housing and Urban Development’s Research Partnership, and the Russell Sage Foundation for their generous support.

Foreword

The Housing Choice Voucher program is HUD’s largest rental housing subsidy, serving over 2.3

mil ion households. However, the program only has enough funding to support approximately one in four eligible households. Low-income families wait, sometimes for many years, for their name to come to the top of the waiting list. When finally offered a voucher, recipients must find a unit that they like and that meets all program requirements. Additionally, the landlord must be wil ing to accept the voucher and rent the unit to them, all within 180 days. The “voucher success rate” indicates how often families successfully navigate this process.

The voucher success rate is a key metric for assessing the performance of the Housing Choice Voucher program. HUD has commissioned three previous large-scale studies of voucher success rates (in 1985, 1994, and 2000). These studies required costly data collection directly from public housing authorities (PHAs). This report examines whether voucher success rates can be calculated solely using HUD administrative data, which would enable HUD to monitor this important metric on an ongoing basis, for individual PHAs.

This paper identifies a promising approach for assessing the quality of data that PHAs submit to HUD. In 2019, an estimated two-thirds of PHAs (1,379 PHAs) had data of sufficient quality to calculate a voucher success rate. For those PHAs, the voucher success rate in 2019 was 61

percent, with a median search time of 60 days. The report also provides insight on success rates related to different search periods (60, 90, 180, 240-day periods). This study analyses the pre-COVID housing market, but with rents having increased significantly in most parts of the country, it is likely that it is now even tougher for voucher holders to find a unit to lease.

HUD also recently announced a new cohort of the Moving to Work Demonstration program that wil evaluate landlord incentives and their impact on participation in the voucher program. To understand the effectiveness of this and other efforts to improve the Housing Choice Voucher program wil require timely and high-quality data. If data and methods in this paper are validated and expanded, HUD could track voucher success rates in nearly real time for individual PHAs and across the nation. This information wil help the voucher program to better reach its full potential.

Solomon Greene

Principal Deputy Assistant Secretary for Policy Development and Research U.S. Department of Housing and Urban Development

iii

Contents

Acknowledgments ................................................................................................................. iv

Foreword ............................................................................................................................... iii

List of Exhibits ........................................................................................................................ v

Executive Summary .............................................................................................................. vi

Background ............................................................................................................................ 1

Terminology ........................................................................................................................... 1

Data Sources ......................................................................................................................... 2

Approach ............................................................................................................................... 3

Sample ................................................................................................................................... 5

Success Rate Results ............................................................................................................ 6

Recommendations ............................................................................................................... 10

Appendix A: Data Cleaning .................................................................................................. 12

Appendix B: Detailed Definitions of Outcomes .................................................................... 17

Appendix C: Issuance Share Sensitivity Testing ................................................................. 19

Appendix D: Entrance Timeline Sensitivity Testing ............................................................. 21

References .......................................................................................................................... 22

iv

List of Exhibits

Exhibit 1: Action Code Glossary ................................................................................................... 2

Exhibit 2: Successful Search ........................................................................................................ 3

Exhibit 3: Failed Search ................................................................................................................ 4

Exhibit 4: Geographic Characteristics of PHAs by Data Quality ................................................... 5

Exhibit 5: Distribution of PHA Size by Data Quality ...................................................................... 6

Exhibit 6: Demographic Characteristics of Voucher Holders by PHA Data Quality ...................... 6

Exhibit 7: Success Rates in 2019 ................................................................................................. 6

Exhibit 8: Distribution of Success Rates Across Public Housing Agencies .................................. 7

Exhibit 9: Success Rates by Search Window ............................................................................... 8

Exhibit 10: Distribution of Median Search Duration Across PHAs ................................................ 8

Exhibit 11: Success Rates and Search Duration by Rural Characterization ................................. 9

Exhibit 12: Success Rates and Search Duration by Region ......................................................... 9

Exhibit 13: Success Rates and Search Duration by Public Housing Agency (PHA) Size ........... 10

Exhibit 14: Success Rates and Search Durations by Median Rent Quartile ............................... 10

Exhibit A.1: Duplicate or Multiple Issuances ............................................................................... 12

Exhibit A.2: Incomplete Expirations ............................................................................................ 13

Exhibit A.3: Inaccurate Expirations ............................................................................................. 13

Exhibit A.4: True Expiration ........................................................................................................ 14

Exhibit A.5: False Expiration ....................................................................................................... 14

Exhibit A.6: Extensions ............................................................................................................... 15

Exhibit A.7: Extensions of Existing Search Events ..................................................................... 15

Exhibit A.8: Unsuccessful Search Events ................................................................................... 15

Exhibit A.9: Frequency of Additional Issuance Classifications .................................................... 16

Exhibit B.1: Examples of False Positives .................................................................................... 17

Exhibit B.2: Frequency for Detailed Outcomes ........................................................................... 18

Exhibit C.1: Success Rates (180-day) Within 10 Percent Issuance Share Bands, 2019 ............ 19

Exhibit C.2: Success Rates (180-day) Within 5 Percent Issuance Share Bands for PHAs with an

Issuance Share of 50 Percent and Higher, 2019 ..................................................................... 19

Exhibit C.3: Correlation Between 2018 and 2019 Success Rates (180-day), by 2019 Issuance

Share Band .............................................................................................................................. 20

Exhibit D.1: Success Rates for Different Variations of the 7-Day Date Threshold ...................... 21

v

Executive Summary

Housing choice vouchers are the largest federal rental assistance program in the United States, serving more than 2 mil ion households annually. Although this program is funded federally, it is administered by local public housing agencies (PHAs). Unlike other federal programs, vouchers require participants to search for and lease a unit on the private rental market, and not all participants succeed in doing so. Success rates, defined as the share of searches that result in a lease-up, have rarely been estimated on a national scale, and the few previous analyses have required collecting data directly from PHAs. The goal of this study is to estimate individual PHA-level and national success rates for recipients of housing choice vouchers at non-Moving to Work (MTW) PHAs, solely using administrative data already collected by the U.S. Department of Housing and Urban Development (HUD). We also seek to evaluate the quality of these administrative data, providing key data quality metrics at the PHA level.

Our approach draws on administrative data including the issuance of search vouchers, expirations of search vouchers, and movements into and out of the voucher program. We cleaned and organized these data into search events. We developed two metrics that assess the quality of each PHA’s data to calculate success rates: the share of entrances to the program that are preceded by a search voucher and the share of participants who enter the program shortly after receiving a search voucher; these metrics are designed to identify PHAs that are likely recording all, or most, search vouchers that are issued.

After accounting for data quality standards, the resulting analysis sample includes 1,379 PHAs across the country. Incorporating all PHAs that meet data quality standards, we estimate that 61

percent of searches initiated in 2019 succeeded, using a 180-day search window. If that timeline is extended to 240 days, the estimated success rate rises to 63 percent. The median time for a successful search, or search duration, is 60 days.

Success rates and search durations vary widely across PHAs. Success rates are lower in rural counties, which also tend to have much shorter search durations. Search durations vary markedly depending on PHA size, with larger PHAs recording much longer search durations.

Search durations are also associated with median rent in the surrounding county, with longer search durations in higher rent counties.

In this report, we walk through our data sources and methodology for calculating success rates.

We then describe our sample in detail, comparing PHAs that meet data quality standards to those that do not. We then present our results on success rates and search duration for the aggregate sample and by PHA and county characteristics. The report ends with recommendations on validating the methodology and improving data col ection.

vi

Background

Low lease-up rates are frequently cited as one of the biggest shortcomings of the voucher program (El en, 2017). Many voucher recipients do not succeed in finding and leasing a unit that meets all criteria for the program within the allowable search time, which varies across public housing agencies (PHAs). Once potential participants are selected off the waitlist at their local PHA and issued a search voucher, they must find a unit that falls within the payment standard for their jurisdiction. The unit must pass a housing inspection, and the landlord must accept the voucher. Researchers have estimated success rates for voucher recipients on a large scale three times previously. In all cases, success rates were estimated for a sample of large metropolitan area PHAs. From 1985 through 1987, the estimated success rate was 68 percent (Leger and Kennedy, 1990). A 1994 study estimated a higher success rate of 81 percent in 1993 (Kennedy and Finkel, 1994). Most recently, a study of 48 randomly selected, representative PHAs yielded an estimate of 69 percent in 2000 (Finkel and Buron, 2001). All three studies relied on new data collected in partnership with selected PHAs solely for the analysis. National estimates were constructed by weighting PHAs by size and probability of selection for the study.

Terminology

HUD data1 on voucher holders are collected at the “action” level; four primary actions are relevant for calculating success rates. New recipients of vouchers are issued a search voucher (action code 10), which grants them permission to search for a unit that meets program criteria and guarantees the subsidy once they find a unit that passes the housing quality inspection. If a voucher recipient is unable to find a unit in the allotted time frame, the search voucher expires (action code 11).

If a recipient succeeds in leasing a unit, they are admitted into the program (action code 1).

Once in the program, participants wil experience additional actions, including annual reassessment and updates to reflect changes in income or household size, moving, or porting in and leasing a unit in a different PHA. When a participant leaves the program, they receive an end of participation code (action code 6).

The upcoming sections use the icons shown in exhibit 1 to represent different actions and how we combine actions into search events to determine outcomes:

1 HUD data refers to the IMS/PIC data, described in detail in the next section.

1

Exhibit 1: Action Code Glossary

Source: Form HUD-50058, ilustration by the authors

Data Sources

The Office of Public and Indian Housing (PIH) at HUD collects administrative data from PHAs through the Inventory Management System/PIH Information Center (IMS/PIC). This system requires PHAs to electronically submit information to HUD on the status of voucher recipients. 2

Whereas previous large-scale studies of success rates (Finkel and Buron, 2001; Kennedy and Finkel, 1994; Leger and Kennedy, 1990) collected data directly from PHAs participating in the study, the methodology of this report uses only IMS/PIC administrative data already collected by HUD.

This report draws on three main sources from the IMS/PIC data: issuance actions, new admissions actions, and an annual cross-sectional database. Al three datasets are derived directly from HUD’s voucher transactions database, which contains every action for all households issued a search voucher or admitted to the program. The issuance data contain all search vouchers (action code 10) and expirations of those search vouchers (action code 11).

We use issuance data starting in 2014 and going through 2020. These records contain the action code, effective date, and a HUD-generated household ID used to link across datasets.

The data also indicate if the search voucher is issued through a special program, such as Enhanced Vouchers or Welfare to Work vouchers.

The new admissions data are also pulled from the transactions database and include all action codes representing new admissions. 3 We use data from 2014 through the second half of 2021; these records include the key fields from the issuance data and distinguish between tenant-based and project-based vouchers. Any actions in this database may represent a participant entering the program.

Lastly, we combine those two sources with the cross-sectional data, an annual cross-section of voucher holders containing the most recent observation from the transactions database for every household in every year from 2003 through 2021. That database is used to distinguish between those looking to enter the program and those in the program looking to move.

2 https://www.hud.gov/program_offices/public_indian_housing/systems/pic/about.

3 Of the new admissions, 99.7 percent are recorded as an action code 1 (admission). A very small number of 4s (moves), 7s (port-ins), or 14s (special adjustment) may be miscoded, but they are established as a new admission to the program through the data-cleaning process.

2

Approach

Search Events

This report groups issuance actions into “search events” and determines success or failure at the search event level. A search event starts with the issuance of a search voucher. Once a search event has started, any subsequent search vouchers issued within the next 180 days are considered part of the same search event, instead of counting each search voucher as a separate attempt; this criterion addresses differences across PHAs in their choice to grant timeline extensions—not included in HUD data—versus issuing a new search voucher. Most search events contain only one issuance.

Of all successful events in our 2019 sample, only 1.4 percent had multiple search vouchers grouped into one search event; the share is 2.2 percent for failed events. Most households—95

percent—had only one search event between 2015 and 2019. Less than 1 percent of households had more than one search event in 2019. For more detailed information on how we grouped actions into search events, see appendix A.

Defining Outcomes

This report defines a “success” as lease-up and program admission within a set period (e.g., 180 days) after search voucher receipt. This is depicted in exhibit 2.

Exhibit 2: Successful Search

Source: Form HUD-50058, il ustration by the authors

A search is considered a failure if the voucher recipient is not admitted into the program within a given search duration (exhibit 3). A search is typically identified as a failure when a voucher is issued to a recipient that does not lease up before the voucher expires. Expirations are observed at different durations; most occur 60, 90, 120, or 150 days after a search voucher was issued. Notably, PHAs are not consistent in recording expirations in the HUD data. If a search event does not have an expiration recorded but the household is not admitted into the voucher program during the specified period, then a failure is imputed in place of an expiration. Around 33 percent of failures were imputed in the 2019 sample.

3

Exhibit 3: Failed Search

Source: Form HUD-50058, il ustration by the authors

A small portion of events have outcomes that involve other actions, but they represent only 2

percent of successful searches and less than 1 percent of failed searches. Those cases are detailed in appendix B.

Data Quality

For each PHA, we calculate two metrics to assess the accuracy and completeness of the issuance data, which are foundational to estimating success rates. The first is the share of entrances to the program that are preceded by a search voucher, known as the “issuance share.”

A complete search should include the issuance of a search voucher, followed by either admission to the program or expiration of that search voucher. However, we sometimes observe admissions to the program that are not preceded by a search voucher issuance. This pattern is problematic because it is an indication that the search voucher issuance data are not complete and may not include all of the searches that are unsuccessful.

We address the lack of complete issuance data in two ways. First, we exclude new admissions that are not preceded by a search voucher issuance, because their inclusion would most likely inflate success rates. 4 Second, we calculate success rates only for PHAs with nearly complete issuance data—specifically, those for which at least 80 percent of admissions are preceded by an issuance (‘issuance share’). For sensitivity testing around this threshold, see appendix C.

The second data quality criterion concerns timelines for successful searches. Some PHAs admit a large share of participants to the program within just a few days of their search voucher issuance. This pattern could indicate that the PHA is not entering issuance actions into HUD

data until admission is about to happen, and that the issuance data entry is incomplete, particularly for failed searches.

To address this issue, we do not estimate annual success rates for PHAs in years in which they admit more than 15 percent of new entrances to the program within 7 days of search voucher 4 With this decision, this report implicitly assumes that search voucher issuances are missing at the same rate for failed and successful searches. If more search vouchers are missing from failed searches than from successful searches, the estimates will be inflated.

4

issuance. Among PHAs with consistent and complete search voucher issuance and expiration data, less than 5 percent of successful searches had a duration of 7 days or less; sensitivity testing around this threshold is reported in appendix D.

Sample

After data quality filters are applied, the final sample includes 1,379 PHAs in 2019, representing just under 63 percent of the 2,161 non-MTW PHAs with active tenant-based voucher holders in 2019. 5 Those 1,379 PHAs collectively served 78 percent of tenant-based voucher holders at PHAs that were present in the issuance data in 2019. Note that this excludes voucher holders at MTW PHAs.

To link PHAs with a geographic area, we perform a crosswalk at the county level. For each PHA, we use the longitudinal voucher database to identify the county where the majority of voucher holders in 2019 reside. If a PHA has voucher holders spread across counties, with no single county housing a majority of its voucher holders, we exclude the PHA from any geographic analysis. Of the 1,379 PHAs with high data quality in 2019, more than 95 percent had the majority of their voucher holders concentrated in one county.

Overall, the sample is comparable to the population of voucher recipients as a whole; exhibits below show that the sample is representative of the population along demographic and geographic characteristics. PHAs from all 50 states and 3 additional territories are included.

PHAs that pass data quality standards are less likely to be in rural counties compared with the universe of non-MTW PHAs observed in the data (exhibit 4). PHAs vary in size; the smaller ones are underrepresented among PHAs that pass quality standards, whereas mid-sized and larger are overrepresented (exhibit 5). The characteristics of voucher holders in this sample are similar to the overall voucher population (exhibit 6).

Exhibit 4: Geographic Characteristics of PHAs by Data Quality

Regional Distribution

Number

States

Characteristic

of PHAs

Represented

West

Midwest

South

Northeast Rural Share

Passes quality

standards

1,379

53

12.5%

26.5%

33.9%

23.5%

19.5%

Does not pass

quality standards

588

46

9.2%

24.5%

37.6%

24.8%

30.4%

All PHAs

2,161

54

11.0%

25.7%

34.5%

24.8%

23.7%

Notes: “All PHAs” include all non-MTW PHAs with tenant-based voucher holders in 2019. The other two categories include only PHAs present in the issuance data with at least one search event in 2019. Values are not weighted and represent a share of PHAs. Counties are considered rural if more than 50 percent of their population lives in a rural area, as classified by the Census Bureau’s Urban and Rural Classification.

Source: IMS/PIC

5 This number represents 70 percent of the 1,967 non-MTW PHAs that are present in the issuance database and had at least one search event in 2019.

5

Exhibit 5: Distribution of PHA Size by Data Quality

Share of PHAs, by Number of Voucher Holders

Number of

501 to

1,251 to

5,251 to

10,001 to

Characteristic

PHAs

1 to 250

251 to 500

1,250

5,250

10,000

60,000

Passes quality

standards

1,379

32.1%

20.7%

24.5%

18.9%

2.3%

1.5%

Does not pass

quality standards

588

54.4%

17.9%

16.3%

8.5%

1.0%

0.7%

All PHAs

2,161

41.8%

19.2%

20.6%

14.6%

1.8%

1.1%

Notes: “All PHAs” include all non-MTW PHAs with tenant-based voucher holders in 2019. The other two categories include only PHAs present in the issuance data with at least one search event in 2019. Values are not weighted and represent the share of PHAs within each row.

Source: IMS/PIC

Exhibit 6: Demographic Characteristics of Voucher Holders by PHA Data Quality

Disabled

Head of

Median Age

Median

Mean

Number of

Non-White

Household

of Household

Annual

Household

Characteristic

PHAs

Share

Share

Head

Income

Size

Passes quality standards

1,379

66.6%

26.3%

49

$11,844

2.32

Does not pass quality

standards

588

63.3%

24.7%

51

$12,570

2.31

All PHAs

2,161

65.7%

25.7%

50

$11,952

2.28

Notes: “All PHAs” include all non-MTW PHAs with tenant-based voucher holders in 2019. The other two categories include only PHAs that are present in the issuance data and had at least one search event in 2019. Values are weighted by number of voucher holders.

Source: IMS/PIC

Success Rate Results

In 2019, the 180-day success rate for all public housing agencies with high data quality was 61

percent (exhibit 7). The distribution of success rates across PHAs, shown in exhibit 8, displays significant variation in success rates across PHAs.

Exhibit 7: Success Rates in 2019

Success Rate

Median Search

Search Events

Successes

Failures

(180 day)

Duration

194,788

118,854

75,934

61.0%

60

Note: Search duration is calculated as the median search duration of searches successful within 240 days.

Source: IMS/PIC

6

Exhibit 8: Distribution of Success Rates Across Public Housing Agencies

Note: Distribution is weighted by number of search events.

Source: IMS/PIC

Search Duration

PHAs have some discretion in deciding on the amount of time they grant to voucher recipients for their housing search. We calculate search duration as the median number of days between search voucher issuance and admission to the program for all successful searches within a 240-day search window. The median search duration for successful searches was 60 days6 but varied substantially across PHAs. That variation has two critical implications for this analysis.

First, we compute success rates for different ‘search windows’—60, 90, 180, and 240 days—to account for the wide variation in allowable search times across PHAs (exhibit 9). For each interval length, a search event is only considered successful if the household leases up within the respective period. Across the sample, fewer than one-half of voucher searches were successful within 90 days of issuance of a search voucher. Success rates increased considerably—to 61 percent—using a 180-day search interval. Extending that interval another 60 days—to 240 days—results in only a 2-percentage-point increase in the estimated success rate, to 63.2 percent.

6 If we shortened the search window and considered searches successful within 180 days, the median search duration would be 57 days.

7

Exhibit 9: Success Rates by Search Window

Source: IMS/PIC

Second, given the variation in search duration, a success rate alone tells an incomplete story about voucher program searches; median search durations must be examined alongside success rates. Two PHAs may have the same 180-day success rate of 60 percent, but in one, the median search time is 45 days, and in the other, it is 145 days. Saying that the voucher recipients have a comparable experience in these two PHAs would not be accurate. As shown in exhibit 10, more variation occurs in search times than in success rates, perhaps because PHAs extend search times in more challenging markets to boost success rates.

Exhibit 10: Distribution of Median Search Duration Across PHAs

Note: Distribution is weighted by number of search events.

Source: IMS/PIC

8

Success Rates and Search Duration by Public Housing Agency (PHA) characteristics As noted, considerable variation exists in both success rates and—in particular—search durations across PHAs. This section analyzes whether these differences correlate with PHA characteristics. Exhibit 11 shows that success rates are lower and search durations shorter for PHAs in predominantly rural counties. Majority-rural counties are also home to smaller PHAs with fewer search events in 2019.

Exhibit 11: Success Rates and Search Duration by Rural Characterization

Median Number of

Success Rate

Median Search

Characterization

Number of PHAs

Search Events

(180 day)

Duration

Majority urban

1,054

69

62.3%

65

Majority rural

256

39

58.5%

41

Note: Includes only PHAs with the majority of their voucher holders in one county.

Source: IMS/PIC

Exhibit 12 shows success rates and search durations by census region. Variation in success rates is small, with success rates slightly lower than the national average in the West and Midwest, and higher in the Northeast and South. Variation in search durations is slightly higher and search durations are longest in the Northeast.

Exhibit 12: Success Rates and Search Duration by Region

Success Rate

Median Search

Region

Number of PHAs

Search Events

(180 day)

Duration

Midwest

366

51,979

60.0%

57

Northeast

324

26,912

62.2%

65

South

467

75,831

62.4%

61

West

173

36,793

58.5%

56

Source: IMS/PIC

Success rates do not vary widely by PHA size either. The smallest and largest PHAs, by number of voucher holders in 2019, have slightly lower success rates than mid-sized PHAs.

Median search duration, however, increases sharply with PHA size. PHAs with 250 or fewer voucher holders had a median search duration of 43 days, compared with 92 days for PHAs with more than 1,000 voucher holders.

9

Exhibit 13: Success Rates and Search Duration by Public Housing Agency (PHA) Size

Number of

Median Number

Success Rate

Median Search

Voucher Holders

Number of PHAs of Search Events

(180 day)

Duration

1 to 250

442

16

59.3%

43

251 to 500

286

49

61.3%

45

501 to 1,250

338

108

59.7%

52

1,251 to 5,250

260

226

63.4%

64

5,251 to 10,000

32

471

60.3%

74

10,001 to 60,000

20

1,282

57.9%

92

Source: IMS/PIC

Lastly, PHAs are divided into quartiles based on the median gross rent in the county. 7 Although PHAs in counties with higher rents—specifically, those in the top quartile—have a somewhat higher success rate on average, they also exhibit much higher search durations. Worth noting is that 60- and 90-day success rates are lower for PHAs in higher rent counties; only when success rates are calculated at the 180-day window do their success rates meet or outperform PHAs in lower rent counties.

Exhibit 14: Success Rates and Search Durations by Median Rent Quartile

Median Rent Range in

Success Rate

Median Search

Median Rent Quartile

Quartile ($)

Number of PHAs

(180 day)

Duration

1

275 to 727

328

59.6%

44

2

728 to 861

328

60.6%

58

3

862 to 1,112

327

62.3%

68

4

1,113 to 1,970

327

64.4%

74

Source: IMS/PIC

Recommendations

This report describes an approach to estimating success rates that relies solely on administrative data HUD already collects. This approach can be used for more than 60 percent of non-Moving to Work (MTW) PHAs, which collectively serve 78 percent of—non-MTW—HCV

households. Following are three sets of recommendations to HUD for next steps to build on this work—specifically, to improve the quality and scope of these measures.

Validation and Improvements

Perhaps the most important next step is to validate this approach—or a variation of it—in the field, using PHA data. The few previous studies of success rates used PHA data that include 7 We use median gross rent from the American Community Survey 2015–2019 5-year estimates. We do not calculate success rates by vacancy rates as there is no high quality source of vacancy rates at the county level.

10

information on extensions of search vouchers and are presumably more complete in numerous ways. Some validation might be possible through data HUD has already collected as part of completed demonstrations or studies that included success rates or, more likely, from existing demonstrations currently in the field. HUD could also leverage data collected as part of current demonstrations, such as the MTW landlord cohort, in which the evaluation design includes calculating success rates with PHA data. Finally, HUD could undertake this validation as part of a new study of Section 8 success rates, as directed by Congress in HUD’s 2021 appropriations.

Regardless of the PHA data source, the approach taken in this report differs from how PHAs may calculate success rates. This report calculates annual success rates for each search event—not every search voucher issuance—that started that year over a 180-day period.

Simply comparing our estimated success rates to PHA self-reported success rates is not appropriate.

As part of any field validation, it would be useful to assess the data quality thresholds applied in this report. For example, is the issuance rate cutoff of 80 percent useful or more restrictive than necessary? Do PHA data reveal that short timelines are accurate and not indicative of selective entry of successful searchers? The threshold and analysis in this report are based on all successful searches, regardless of whether the voucher recipient changed housing units. HUD

data on all voucher actions include a variable indicating whether the household has moved in the past year. Preliminary work with this variable suggests that it is well populated and that very short searches are more prevalent for voucher recipients who did not change units. HUD—or others—could refine the data quality threshold to apply strictly to new admissions who change units, for which such short durations are least likely; this data, too, can be validated in the field.

Expand to Moving to Work Public Housing Agencies

The success rates in this report have been calculated exclusively for non-MTW PHAs because those were the data available to the researchers. HUD could easily apply the approach to MTW

PHAs (after validation).

Improving HUD’s Data Quality

A more forward-looking recommendation to improve HUD’s ability to calculate success rates from the administrative data provided by PHAs is the development of a feedback system for PHAs regarding their data quality. We have produced a series of tailored reports for each PHA in our analysis sample; these “report cards” assess the quality of the PHA’s voucher issuance data and present the resulting estimates for success rate and search duration. HUD could publish similar reports—or share them directly with PHAs—on an ongoing basis to identify and correct data quality problems.

Those next steps would enable HUD and Congress to obtain more accurate and timely information on success rates in the Housing Choice Voucher program.

11

Appendix A: Data Cleaning

Additional Issuance Data Classifications

For this report, we cleaned and grouped issuance and new admissions actions into search events to calculate success rates. Several common problems have been identified in this process, and a set of rules engineered to address them and determine which issuance actions were part of each search event. This appendix details some of the most prevalent and consequential data issues. Exhibit A.1 indicates how prevalent each classification was in 2019.

Duplicate or Multiple Issuances

The issuance data contain duplicates; some households may also see multiple search vouchers issued in a short period.

Exhibit A.1: Duplicate or Multiple Issuances

Source: Form HUD-50058, il ustration by the authors

We solved the duplication problem by removing exact duplicates, as well as any search vouchers issued within 45 days of a previous search voucher, with no other actions in between.

In the latter case, we remove the second duplicate, so that the start of the search event is defined by the earliest effective date. In 2019, 6.5 percent of search vouchers were exact duplicates, and 5.7 percent were issued within 45 days of another search voucher. Across all years, 13 percent of search vouchers were removed in this stage.

Incomplete Expirations

Some search events do not have recorded expirations, even if the participant was never admitted to the program and no further actions occurred after the initial search voucher; this result is relatively common, with 33 percent of failed searches in 2019 lacking an expiration.

12

Exhibit A.2: Incomplete Expirations

Source: Form HUD-50058, il ustration by the authors

In these search events, we impute a failure when the participant is not observed in the new admissions or cross-sectional data within a given time frame from the beginning of the search event. Although we can be reasonably sure that the participant did not successful y lease up into a unit, we cannot determine a timeline for how long the search lasted or if the public housing agency issued an expiration.

Inaccurate Expirations

A related problem is inaccurate expirations. Expirations are sometimes followed by another search voucher in a short period of time, or maybe followed by admission to the program. An example of this sequence is depicted in exhibit A.3, in which an expiration is issued after 60

days of an initial search voucher and a new search voucher is issued not long after—in this example, 4 days later.

Exhibit A.3: Inaccurate Expirations

Source: Form HUD-50058, il ustration by the authors

13

We solve for this by distinguishing between a true expiration and a false expiration. If a new search voucher is issued within 30 days of the expiration of an existing voucher, it is treated as an extension. If a new search voucher is issued more than 30 days after an expiration, we consider this a new search voucher, and the beginning of a new search event for this household; those potential combinations and relevant solutions are depicted in exhibits A.4 and A.5.

Exhibit A.4: True Expiration

Source: Form HUD-50058, il ustration by the authors

Exhibit A.5: False Expiration

Source: Form HUD-50058, il ustration by the authors

Extensions

Extensions of search vouchers are not recorded in the IMS/PIC data. We cannot distinguish between a new search or an extension, but we can see that some households receive multiple issuances in a short time period. Whether or not multiple search vouchers are grouped together into one search event wil affect success rates.

14

Exhibit A.6: Extensions

Source: Form HUD-50058, il ustration by the authors

Our solution is to consider a new search voucher issuance within 180 days of the original search voucher as an extension. However, if a new search voucher is issued more than 180

days after the first search voucher, with no other actions in between, it is considered a new search event, and the original search event is unsuccessful; both potential outcomes are depicted in exhibits A.7 and A.8, below.

Exhibit A.7: Extensions of Existing Search Events

Source: Form HUD-50058, il ustration by the authors

Exhibit A.8: Unsuccessful Search Events

Source: Form HUD-50058, il ustration by the authors

Defining Entrants to the Program

Our success rate calculation includes only those who are entering the program, not those who are looking to move or port in to another PHA. Those actions are not always labeled correctly, so we use the cross-sectional data to track the participation of households over time.

Specifically, we use PHA codes and action codes to determine which households were not currently in the program during each search event.

15

A significant share of search vouchers are issued to households currently in the program. In 2019, 32 percent of search vouchers were issued to active participants.

Extraneous Expirations

We do not consider any search vouchers issued to participants while they are in the program, in order to distinguish between those looking to enter the program and those looking to move.

There are cases where the participant may be issued a search voucher while they are actively in the program, and then the search voucher expires after the participant has been issued an end of participation code. Those expirations are considered extraneous and are not included in a search event; this circumstance is rare, as less than 1 percent of expirations in 2019 are extraneous.

Issuances for Project-Based Vouchers

In the case of project-based vouchers (PBV), some PHAs issue search vouchers even though PBVs generally do not require the same search process as tenant-based vouchers. Although we cannot identify PBVs in the issuance data, we can identify them once the household has been admitted. We solve this problem by removing search voucher issuances associated with a PBV. To the extent that any issuances associated with a PBV fail, this wil bias our success rates downward. We are essential y assuming that none of the search vouchers issued for PBVs result in an unsuccessful search.

Exhibit A.9: Frequency of Additional Issuance Classifications

Share of Search

Share of Al

Voucher

Share of

Classification

Count

Issuance Actions

Issuances

Expirations

Original issuance

227,724

69.6%

86.9%

--

Additional issuance

29,349

9.0%

11.2%

--

Issuance extension

4,901

1.5%

1.9%

--

True expiration

49,692

15.2%

--

76.4%

False expiration

10,203

3.1%

--

15.7%

Late expiration

4,414

1.3%

--

6.8%

Extraneous expiration

608

0.2%

--

0.9%

Lead off expiration

151

0.0%

--

0.2%

Note: Exhibit includes the full universe of actions in 2019.

Source: IMS/PIC

16

Appendix B: Detailed Definitions of Outcomes

This analysis distinguishes between observed and imputed failures and successes. An observed success is an event in which a participant receives a search voucher and then is admitted to the program within 180 days. A success with a false expiration occurs when an event contains a search voucher and an expiration of that search voucher but is followed by admission to the program. Successes with false expirations make up just 2 percent of all successes among the sample in 2019.

Multiple patterns are classified as failures. An observed failure is when an event contains a search voucher and an expiration for that search voucher. Observed failures constitute the majority, 66 percent, of failed searches among the sample in 2019. However, events can stil be classified as failures in the absence of an expiration if the household is never admitted to the program and never found in the new admissions or longitudinal database. This is one of the three patterns that constitutes an imputed failure, accounting for 34 percent of failed searches in 2019.

The imputed failure category contains two other patterns that fall in the category of a false positive. In these situations, the household receives a search voucher then receives an action indicating that they were admitted to the program but receives an end of participation code within 30 days of admission. Similarly, some households receive a search voucher and then receive an end of participation code in lieu of an expiration; both patterns are classified as false positives, which make up less than 1 percent of failed searches in 2019.

Exhibit B.1: Examples of False Positives

Source: Form HUD-50058, il ustration by the authors

17

Exhibit B.2: Frequency for Detailed Outcomes Outcome

Number of Events Share of Successes Share of Failures

Successes

Standard success

123,207

98.00%

--

Success with false expiration

2,509

2.00%

--

Failures

Standard failure

45,768

--

66.26%

Imputed failure

23,203

--

33.59%

False positive

79

--

0.11%

False positive with false expiration

22

--

0.03%

-- = not applicable.

Note: Includes only 2019 events for PHAs that meet data quality standards.

Source: IMS/PIC

18

Appendix C: Issuance Share Sensitivity Testing

In selecting an issuance share cutoff, we examined two metrics: the 180-day success rate within different bands of the issuance share and the correlation between 180-day success rates in 2018 and 2019 within different issuance share bands. The goal for both metrics was to identify the issuance share threshold below which success rates appeared to look more volatile. On the basis of the results in exhibits C.1 through C.3, we selected an 80-percent issuance share threshold.

Exhibit C.1: Success Rates (180-day) Within 10 Percent Issuance Share Bands, 2019

Number of

Number of

Success Rate

Issuance Share Band (10%)

Events

PHAs

(180-day)

(0.5,0.6]

1,995

47

68.7%

(0.6,0.7]

4,649

78

64.9%

(0.7,0.8]

12,428

145

59.2%

(0.8,0.9]

36,581

315

61.8%

(0.9,1]

344,002

1,464

61.4%

Source: IMS/PIC

Exhibit C.2: Success Rates (180-day) Within 5 Percent Issuance Share Bands for PHAs with an Issuance Share of 50 Percent and Higher, 2019

Number of

Number of

Success Rate

Issuance Share Band (5%)

Events

PHAs

(180-day)

(0.5,0.55]

834

14

74.7%

(0.55,0.6]

1,161

33

64.4%

(0.6,0.65]

1,766

22

61.2%

(0.65,0.7]

2,883

56

67.1%

(0.7,0.75]

3,560

69

64.9%

(0.75,0.8]

8,868

82

56.9%

(0.8,0.85]

12,906

113

59.0%

(0.85,0.9]

23,675

217

63.2%

(0.9,0.95]

59,974

354

62.5%

(0.95,1]

284,028

1,290

61.2%

Source: IMS/PIC

19

Exhibit C.3: Correlation Between 2018 and 2019 Success Rates (180-day), by 2019 Issuance Share Band

Number of

Weighted

Unweighted

Issuance Share Band (10%)

PHAs

Correlation

Correlation

(0.5,0.6]

28

0.19

0.51

(0.6,0.7]

53

0.44

0.31

(0.7,0.8]

100

0.73

0.59

(0.8,0.9]

203

0.79

0.67

(0.9,1]

1,403

0.77

0.61

Note: Weighted correlations are weighted by number of events.

Source: IMS/PIC

20

Appendix D: Entrance Timeline Sensitivity Testing Exhibit D.1 shows the 180-day success rates for PHAs that fail different versions of the date threshold. For instance, the first row contains PHAs that admit more than 20 percent of their entrants within 10 days.

Exhibit D.1: Success Rates for Different Variations of the 7-Day Date Threshold Number of

Number of

Success Rate

Share Limit

PHAs

Search Events

(180 day)

20%

194

10,673

66.9%

15%

271

14,927

64.3%

10%

400

26,447

63.4%

7%

514

38,773

62.1%

None

1,639

202,001

62.2%

Note: Includes only PHAs that meet the 80-percent issuance share threshold.

Source: IMS/PIC

21

References

Elen, Ingrid Gould. 2017. What Do We Know About Housing Choice Vouchers? Working paper.

New York: New York University, Furman Center.

HousingChoiceVouchers_WorkingPaper_IngridGouldEl en_14AUG2017.pdf

(furmancenter.org).

Finkel, Meryl, and Larry Buron. 2001. Study on Section 8 Voucher Success Rates. Volume I, Quantitative Study of Success Rates in Metropolitan Areas. Cambridge, MA: Abt Associates.

Leger, Mirielle, and Stephen Kennedy. 1990. Final Comprehensive Report of the Freestanding Housing Voucher Demonstration. Washington, DC: U.S. Department of Housing and Urban Development.

Kennedy, Stephen, and Meryl Finkel. 1994. Section 8 Rental Voucher and Rental Certificate Utilization Study: Final Report. Cambridge, MA: Abt Associates.

22

U.S. Department of Housing and Urban Development Office of Policy Development and Research

Washington, DC 20410-6000

February 2023

Document Outline

Using HUD Administrative Data to Estimate Success Rates and Search Durations for New Voucher Recipients

Disclaimer

Acknowledgments

Foreword

Contents

List of Exhibits

Executive Summary

Background

Terminology

Data Sources

Approach

Search Events

Defining Outcomes

Data Quality

Sample

Success Rate Results

Search Duration

Success Rates and Search Duration by Public Housing Agency (PHA) characteristics

Recommendations

Validation and Improvements

Expand to Moving to Work Public Housing Agencies

Improving HUD’s Data Quality

Appendix A: Data Cleaning

Additional Issuance Data Classifications

Duplicate or Multiple Issuances

Incomplete Expirations

Inaccurate Expirations

Extensions

Defining Entrants to the Program

Extraneous Expirations

Issuances for Project-Based Vouchers

Appendix B: Detailed Definitions of Outcomes

Appendix C: Issuance Share Sensitivity Testing

Appendix D: Entrance Timeline Sensitivity Testing

References