Final Report

FY 2015 Improper Payment for Quality Control

for Rental Subsidy Determination Study

FINAL

FY 2015 Final Report

Improper Payment for Quality Control

for Rental Subsidy Determination Study

Prepared for:

Office of Policy Development and Research

Department of Housing and Urban Development

Washington, DC 20410

Prepared by:

ICF International

August 31, 2016

Table of Contents

Table of Contents

Executive Summary ................................................................................................................... ES-i

I. Introduction ...............................................................................................................................I-1

II. Methodology ...........................................................................................................................II-1

III. Study Objectives and Analytic Methods .............................................................................. III-1

IV. Findings ................................................................................................................................ IV-1

V. Recommendations ...................................................................................................................V-1

Appendix A: Rent Calculations .................................................................................................. A-1

Appendix B: Weighting Procedure ..............................................................................................B-1

Appendix C: Source Tables

Source Tables Based on Quality Control Data .................................................................C-1

Source Tables Based on Tenant File Data ......................................................................C-34

Appendix D: Consistency and Calculation Errors ...................................................................... D-1

Appendix E: Project Staff Questionnaire Descriptive Analysis...................................................E-1

Appendix F: Multivariate Analysis ..............................................................................................F-1

List of Exhibits

Exhibit ES-1

Frequency of Rent Error by Program Type . . . . . . . . . . . . . . . . . . . . . . . . ES-iv

Exhibit ES-2

Subsidy Dollar Error   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ES-v

Exhibit ES-3

Estimates of Error in Program Administrator Income and Rent

Determinations (in $1,000s)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ES-v

Exhibit ES-4

Comparative 2000 Through FY 2015 Gross Erroneous Payments* . . . . ES-vii

Exhibit ES-5

Rent Components Responsible for the Largest Dollar Error for

Households With Rent Error   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ES-x

Exhibit ES-6

Impact of Changes in the Error Threshold on Frequency and Estimates

of Error (in $1,000s)  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ES-xi

Exhibit II-1

Change in Frame Population Totals Used to Pull the Study Sample Over

Time   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . II-3

Exhibit III-1

PHA-administered Section 8 Unit Size Standards  . . . . . . . . . . . . . . . . . . .III-4

Exhibit III-2

Summary of Study Objectives   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .III-7

HUDQC Final Report for FY 2015

TOC-i

Table of Contents

Exhibit IV-1

Percentage of Households With Proper Payments  . . . . . . . . . . . . . . . . . . . IV-3

Exhibit IV-2

Percentage of Households With Error, Average Dollars in Error, and

Dollar Error Rate for All Households With Error . . . . . . . . . . . . . . . . . . . . IV-4

Exhibit IV-3a

Underpayment Households: Percentage of Households and Average

Monthly Dollar Amount of Error  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-5

Exhibit IV-3b

Overpayment Households: Percentage of Households and Average

Monthly Dollar Amount of Error  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-5

Exhibit IV-4

Gross and Net Dollar Rent Error (Monthly) for All Households  . . . . . . . . IV-6

Exhibit IV-5

Gross and Net Dollar Error Rates (Monthly) for All Households   . . . . . . . IV-7

Exhibit IV-6

Certifications and Recertifications by Administration Type*  . . . . . . . . . . . IV-8

Exhibit IV-7a

Percentage of Newly Certified Households Meeting

Certification Criteria  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-10

Exhibit IV-7b

Percentage of Newly Certified Households Meeting Certification

Criteria by Program Type  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-10

Exhibit IV-8

Average Monthly Underpayment and Overpayment: Dollar Amounts

Averaged Across All Households  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-11

Exhibit IV-9a

Positive Subsidy Households (Tenant Underpayment) Percentage of

Households and Average Monthly Dollar Amount of Error  . . . . . . . . . . . IV-11

Exhibit IV-9b

Negative Subsidy Households (Tenant Overpayment) Percentage of

Households and Average Monthly Dollar Amount of Error  . . . . . . . . . . . IV-12

Exhibit IV-10

Average Monthly Underpayment and Overpayment: Dollar Amount

Averaged Across All Households  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-12

Exhibit IV-11

Rent Components Responsible for the Average Annual Largest

Dollar Error for Households With Rent Error . . . . . . . . . . . . . . . . . . . . . . IV-14

Exhibit IV-12

Total and Annual Component Dollars in Error for Households

With Rent Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-14

Exhibit IV-13

Percentage of Households With Rent Component Error by

Payment Type  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-15

Exhibit IV-14

Percentage of Households With Elderly/Disabled Allowances

and Dependent Allowances* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-16

Exhibit IV-15

Findings With and Without Information Obtained From Sources

Other Than the Tenant File  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-17

Exhibit IV-16

Percentage of Households With Calculation and Consistency Errors   . . . IV-18

Exhibit IV-17

Form HUD-50058/50059 Rent Calculation Error Compared With

QC Rent Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-19

Exhibit IV-18

Percentage of Households Fully Verified by Either the

PHA/Owner or ICF   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-20

Exhibit IV-19

Verification of Form HUD-50058/50059 Rent Components by

PHA/Owners** . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-22

TOC-ii

August 31, 2016

Exhibit IV-20

Verification of Form HUD-50058/50059 Rent Components by

PHA/Owner Staff, by Form Type  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-23

Exhibit IV-21

QC Error Households With Missing Verification in the Tenant File   . . . . IV-24

Exhibit IV-22a

Form HUD-50058/50059 Administrative Error: Percentage of

Households, Average Dollars in Error (Non-MTW Households) . . . . . . . IV-25

Exhibit IV-22b  Form HUD-50058/50059 Administrative Error: Percentage of

Households,

Average Dollars in Error (Households With QC Rent Error)  . . . . . . . . . . IV-26

Exhibit IV-23

Administrative Error: Percentage of Households Average Dollars in

Error for All Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-27

Exhibit IV-24a

Percentage of Households in Units With the Correct Number of

Bedrooms (According to Study Guidelines) . . . . . . . . . . . . . . . . . . . . . . . IV-28

Exhibit IV-24b  Percentage of All Households in FY 2014 by Number of Bedrooms

and Number of Household Members*  . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-28

Exhibit IV-24c

Percentage of All Households in FY 2015 by Number of Bedrooms

and Number of Household Members*  . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-29

Exhibit IV-25

PHAs by Predominant Rent Reasonableness Method*   . . . . . . . . . . . . . . IV-30

Exhibit IV-26a

Rent Reasonableness Documentation for New Admissions and

Annual Recertifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-31

Exhibit IV-26b  Type of Rent Reasonableness Documentation for New Admissions

and Annual Recertifications*   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-32

Exhibit IV-27

Timing of Most Recent Rent Reasonableness Determination: New

Admissions and Annual Recertifications*   . . . . . . . . . . . . . . . . . . . . . . . . IV-32

Exhibit IV-28a

Types of Documents Used by PHAs to Identify Utilities and Calculate the

Utility Allowance Value* . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .IV-34

Exhibit IV-28b  Comparison of Utility Allowance on the Form HUD-50058 to the

Utility Allowance Worksheet  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-35

Exhibit IV-28c

Availability of All Information to Enable QC Utility Allowance (UA)

Calculation*   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-35

Exhibit IV-28d  QC Utility Allowance Compared to Form HUD-50058

Utility Allowance  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-36

Exhibit IV-29a

Number and Percentage of Households With Payment Standard

Discrepancies   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-38

Exhibit IV-29b  Payment Standard Compared With the Fair Market Rent   . . . . . . . . . . . . IV-39

Exhibit IV-29c

Details of Cases Falling Outside 90%–110% of the Fair

Market Rent*   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-39

Exhibit IV-29d  Comparison of the FY 2014 to FY 2015 Payment Standard Analysis  . . . IV-40

Exhibit IV-30a

PIC/TRACS Data by Program Type and Average Gross Dollars in

Error for All Households . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-42

HUDQC Final Report for FY 2015

TOC-iii

Table of Contents

Exhibit IV-30b  PIC/TRACS Data by Program Type and Average Gross Dollars in

Error for Households in Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-42

Exhibit IV-30c

Average Gross Dollars in Error by Payment Type and

PIC/TRACS Data  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-43

Exhibit IV-30d  Average Net and Gross Dollars in Error by Administration Type

and PIC/TRACS Data for All Households  . . . . . . . . . . . . . . . . . . . . . . . . IV-43

Exhibit IV-30e

Percentage of Matched and Non-Matched Dollar Amounts for Key

Variables Matching Variables From the Form HUD-50058/50059

and PIC/TRACS  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-44

Exhibit IV-31a

Administrative Errors in the 20 Largest PHAs  . . . . . . . . . . . . . . . . . . . . . IV-51

Exhibit IV-31b  Dollar Rent Errors in the 20 Largest PHAs*   . . . . . . . . . . . . . . . . . . . . . . IV-52

List of Figures

Figure ES-1

Comparative 2000 through FY 2015 Gross Erroneous Payments

Over Time  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ES-viii

Figure IV-1

Payment by Program Type  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-6

Figure IV-2

Percentage of Cases by Case Type  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-8

Figure IV-3

Example of the Impact of Component Errors . . . . . . . . . . . . . . . . . . . . . . IV-13

Figure IV-4

PIC/TRACS Data Present by Program Type for All Households

Over Time  . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . IV-41

Figure IV-5

Conceptual Model Used in Multivariate Analysis  . . . . . . . . . . . . . . . . . . IV-48

TOC-iv

August 31, 2016

Executive Summary

The U.S. Department of Housing and Urban Development (HUD) Quality Control for Rental

Assistance Subsidy Determinations (HUDQC) Study provides national estimates of the extent,

severity, costs, and sources of rent errors in tenant subsidies for the largest housing programs

administered by the Office of Housing and the Office of Public and Indian Housing (PIH). These

programs account for nearly all of HUD’s current housing assistance outlays administered by

the Office of Housing and PIH, as well as the majority of rental units assisted by HUD. This

study was designed to measure the extent of administrator income and rent determination error by

housing providers. It does not involve an audit of individual Public Housing Authorities (PHAs)

or projects, nor does it monitor the implementation of housing programs. Its singular focus is to

identify households for which an error was made in the calculation of the amount of the household’s

rent and to provide nationally representative findings related to those errors.

The errors evaluated by ICF in this study affect the rent contributions that tenants should have been

charged. The findings presented in this report are derived from data collected from December 2015

through May 2016 for actions taken by PHA and project staff during Federal fiscal year (FY) 2015

(October  2014  through  September  2015).  These  findings  show  that  73  percent  of  households

nationally paid the correct amount of rent in FY 2015. In 13 percent of cases, households paid too

much rent, and in 14 percent of cases, households paid too little.

HUD administers its rental housing assistance programs through third-party program administrators,

including PHAs, public and private project owners, and contracted management agents. In the

programs examined, eligible tenants are generally required to pay 30 percent of their adjusted

income toward shelter costs (i.e., contract rent plus utilities), with HUD providing the balance of

the rental payment. New program applicants are required to provide information on household

characteristics, income, assets, and expenses, and this information is used to determine the amount

of rent they need to pay. In most instances, current tenants must certify this information annually

and, in some circumstances, they must recertify the information when there are significant changes

in the household’s income or composition. Applicant or tenant failure to correctly report income

may result in HUD’s over- or underpayment of housing assistance. The failure of the responsible

program administrator to correctly interview the tenant or process and calculate the tenant’s rental

assistance may also result in an improper payment.

In 2000, HUD established a baseline error measurement to cover the three major types of rental

housing assistance payment errors: (1) program administrator income and rent determination error,

(2) intentional tenant misreporting of income (the Income Match Study), and (3) errors in program

administrator billings for assistance payments (the HUDQC billing studies). Fourteen studies have

been conducted to identify program administrator income and rent determination error. In addition

to the 2000 study, studies were conducted in FY 2003 through FY 2015. The study referenced in this

report covers FY 2015 and updates the FY 2014 measurement of errors in program administrator

income and rent determinations. Separate reports will be provided for the other studies conducted

in FY 2015.

For the purposes of this study, “error” is defined as any rent calculation or eligibility determination

that differs from what would have occurred if the PHA or other program administrator had followed

all HUD income certification and rent calculation requirements during the initial certification or

HUDQC Final Report for FY 2015

ES-i

Executive Summary

annual recertification conducted in FY 2015. When appropriate, study findings are compared with

findings from the previous studies.

Financial Impact of Identifying Rent Error. Reduction in the rent error associated with the

programs included in this study does not mean there will be an overall savings in the costs associated

with administering these programs. Given the large number of eligible households on waiting lists,

if a household leaves the program because it is no longer eligible for a subsidy, another household

will take its place, and the replacement household may be entitled to a smaller or a larger subsidy

than the household that left the program. Therefore, the most direct benefit of identifying households

with rent error is ensuring that the households eligible for the program are receiving the correct

subsidy, rather than reducing the funds needed to administer the programs. The most appropriate use

of this study is as a tool for strengthening HUD’s procedures for ensuring administrative compliance

with regulations. The implementation of the recommendations presented in this report may require

greater resources in order to provide HUD, PHAs, and owners with the written policy guidelines,

training, standardized forms, and ongoing monitoring needed to ensure program compliance. The

HUDQC Study assists the agency’s objective of providing the right subsidies to the right families

to sustain and support quality rental assistance programs for communities.

A. Methodology

HUD Requirements and Study Standards. Using the  Code of Federal Regulations and official

HUD handbooks and notices, we consolidated all HUD rules relevant to the determination of rent

into a set of HUD requirements. We invited program experts to participate in establishing and

reviewing the standards used in this study.

The Sample. A nationally representative sample of 583 projects in the United States and Puerto

Rico was selected for this study. These projects were selected from the universe of the three program

types covered by the study:

● Public Housing

● PHA-administered Section 8 (Vouchers and Moderate Rehabilitation)

● Owner-administered Section 8, Section 202 Project Rental Assistance Contract (PRAC),

Section 811 PRAC, and Section 202/162 Project Assistance Contract

A random sample of four households was selected for most projects, but more households

were  selected  from  unusually  large  projects.  The  final  study  data  set  includes  responses  from

2,400 households.

Out-of-Scope Projects. Certain programs were excluded from the study because their eligibility

and rent calculation rules differed from the standards, including the Owner-administered Rental

Assistance Payment, Rental Supplement Program, Section 236, and Below Market Interest Rate

programs. Since the FY 2012 study, Moving to Work (MTW) agencies have been included in

the sampling frame and the HUDQC Study sample. Owner-administered Rental Assistance

Demonstration properties were excluded from the FY 2015 sample. Universe files requested from

HUD either excluded out-of-scope projects, or the projects were identified by HUD for easy removal.

ES-ii

August 31, 2016

Executive Summary

Weighting. Updated population counts per program were calculated based on the assisted housing

universe files provided by HUD in June 2015 to compile weights for the study. The same population

totals per program, provided by HUD in the FY 2005 statement of work, were used from FY 2006

through FY 2010. Starting in FY 2011, the population totals were updated based on the FY 2012

HUDQC sample universe to better reflect the current population. The same population totals were

used from FY 2012 to FY 2014, and were then updated for the FY 2015 study. In general, when the

population totals are adjusted, the changes seen in total gross dollar error may be due to a change in

the assisted housing population, not necessarily an increase in average dollar error. When comparing

dollar error between years in which the population size has been adjusted, it is appropriate to

compare average dollar error, as it is not affected by changes in population size.

The Data Collection Process. The data collection effort included creating and automating more

than 30 data collection instruments, contacting and obtaining information from PHA/Owner staff,

hiring and training 73 field interviewers, and selecting the project and household sample. Field

interviewers  obtained  data  from  tenant  files  and  interviewed  tenants  using  computer-assisted

personal interviewing software developed for this study. The automated data collection process

included built-in consistency and edit checks that prompted interviewers to probe inconsistent and

anomalous responses. Collected data were electronically transferred daily to ICF headquarters

for review. Requested third-party verifications related to income, assets, and expenses were also

processed at ICF’s office in Rockville, MD.

Calculation of Rent Error. A quality control (QC) rent was calculated for each household in the

sample, using the information reported by the PHA/Project, household, Social Security match,

and third-party verification. Rent error was calculated by subtracting the QC rent from the actual

paid tenant rent (the rent from Forms HUD-50058/50059 that was calculated by the project staff).

A discrepancy of $5 or less between the actual and QC rents was not counted as an error. This $5

differential was used to eliminate rounding differences and minor calculation discrepancies that

have little effect on program-wide subsidy errors.

B. Major Rent Error Findings

National Rent Error Estimates. The analysis of the FY 2015 tenant files, household interviews,

and income verification data indicates that:1

● Seventy-three percent of all households paid the correct amount of rent within $5 (61 percent

paid the exact amount).

● Fourteen percent of all households paid at least $5 less than they should have (with an

average error of $66 per month).

● Thirteen percent of all households paid at least $5 more than they should have (with an

average error of $35 per month).

1   Note that results may not total to 100 percent due to rounding.

HUDQC Final Report for FY 2015

ES-iii

Rent Error Estimates by Program Type. The rate of rent underpayments was highest, at 15 percent, in the PHA-administered Section 8 program, followed by the Public Housing program at 13 percent

and the Owner-administered program at 12 percent. The PHA-administered Section 8 program had

the highest overpayment rate, at 15 percent, followed by Public Housing programs at 12 percent

and Owner-administered programs with 11 percent. Exhibit ES-1 summarizes this information.

Exhibit ES-1

Frequency of Rent Error by Program Type

Rent Underpayment

Rent Overpayment

Program

(Subsidy Overpayment)

(Subsidy Underpayment)

Public Housing

13%

12%

PHA-administered Section 8

15%

15%

Owner-administered

12%

11%

Total

14%

13%

Dollar Error Effect of Rent Errors. All summary error estimates represent the summation of net

case-level errors, meaning that a case was determined to have a net overpayment error, no error, or

a net underpayment error. Major findings are as follows:2

● Rent underpayments of approximately $499.3 million annually (up from $458.4 million

in FY 2014). For tenants who paid less monthly rent than they should have paid

(14 percent), the average monthly underpayment was $66. For purposes of generalization,

spreading total underpayment errors across all households (including those with no error

and overpayment error) produces a program-wide average monthly underpayment error of

$8.94 ($107 annually). Multiplying and weighting the $107 by the approximately 4.7 million

units represented by the study sample resulted in an overall annual underpayment dollar

error of approximately $499.3 million per year.

● Rent overpayments of approximately $248.5 million annually (down from

$260.3 million in FY 2014).  For tenants who paid more monthly rent than they should have

paid (13 percent), the average monthly overpayment was $35. When this error is spread

across all households, it produces an average monthly overpayment of $4.45 ($53 annually).

Multiplying and weighting the $53 by the approximately 4.7 million units represented by

the study sample resulted in an overall annual overpayment dollar error of approximately

$248.5 million per year.

● Aggregate net rent error of $250.9 million annually. When combined, the average Gross

Rent Error per case was $13.39 ($8.94 + $4. 45). Overpayment and underpayment errors

partly offset each other; the net overall average monthly rent error was -$4.49 (-$8.94

+ $4.45). HUD subsidies for Public Housing and PHA-administered Section 8 programs

equal the allowed expense level or payment standard minus the tenant rent, which means

2   National annual totals in the text and exhibits were calculated using exact values and were weighted.

Although household-level numbers are presented below, using them to calculate national annual totals

will result in different amounts due to both rounding and weighting. Similarly, the source tables in

Appendix C were rounded to the nearest integer for formatting purposes.

ES-iv

August 31, 2016

Executive Summary

that rent errors have a dollar-for-dollar correspondence with subsidy payment errors,

except in the Public Housing program in years in which it is not fully funded (in which

case, errors have slightly less than a dollar-for-dollar effect). The study found that the net

subsidy cost of the under- and overpayments was approximately $250.86 million per year

($499.34 million – $248.48 million).

Subsidy overpayment and underpayment dollars are summarized in Exhibit ES-2.3 This information

identifies the various types of errors, error rates, and related estimated variances.

Exhibit ES-2

Subsidy Dollar Error

Subsidy

Subsidy

Type of Dollar Error

Overpayment

Underpayment

$66

$35

Average Monthly per Tenant Error for Households with Errors

(14% of cases)

(13% of cases)

Average Monthly per Tenant Error Across All Households

$8.94

$4.45

Total Annual Program Errors

$499.3 million

$248.5 million

Total Annual Errors (95% Confidence Interval)

$374.2 – 624.5 million

$178.3 – 318.7 million

Exhibit ES-3 provides estimates of program administrator error by program type. These data provide

estimates of national-level net costs for total errors and major error types, provide information on

the extent to which errors are concentrated in projects and programs, and estimate total positive

and negative errors in terms of HUD subsidies.

Exhibit ES-3

Estimates of Error in Program Administrator Income and Rent Determinations (in $1,000s)

Subsidy

Subsidy

Net Erroneous

Gross Erroneous

Program

Overpayment

Underpayment

Payment

Payment

Public Housing

$98,861

$84,426

$14,435

$183,286

PHA-administered Section 8

$287,452

$113,326

$174,126

$400,778

Total PHA-administered

$386,313

$197,752

$188,561

$584,064

Owner-administered

$113,025

$50,732

$62,293

$163,758

Total

$499,338

$248,484

$250,854

$747,822

95% Confidence Interval

±$125,137

±$70,208

±$136,035

±$150,569

Comparison with Prior Studies.  Thirteen prior studies (the 2000 baseline study and the annual

studies since FY 2003) estimated erroneous payments attributed to program administrator rent

calculation and processing errors. The current FY 2015 study used similar methodology, sampling

procedures, and sample sizes as all previous studies. Although the FY 2003 and FY 2004 studies

demonstrated significant reductions in erroneous payments attributed to program administrator

income and rent determinations, the studies since that time have shown less dramatic changes in

gross error.

3  Estimates should be viewed in conjunction with 95 percent confidence intervals. Based on the sample,

estimates may vary from year to year. Variations in estimates may not be statistically significant.

HUDQC Final Report for FY 2015

ES-v

The total gross erroneous payments decreased from FY 2014 to FY 2015, but this change was not

statistically significant. There were also no statistically significant overall changes or significant

changes within program type. The average dollar error decreased from $13.55 in FY 2014 to

$13.39 in FY 2015. The decrease in total gross dollar error may have been caused by the change

in population size. Additionally, in a departure from previous years, newfound sources of Social

Security income were treated as intentional tenant misreporting of income in FY 2015, and was not

included in the QC rent calculation.4 This income source was instead included in the Income Match

Study. Finally, sampling error contributes to the variation of estimates from year to year, as a new

sample is drawn for each study in order to best represent the housing population at that point in time.

Exhibit ES-4 presents a review of the gross erroneous payments for the QC studies from 2000 to

FY 2015. Figure ES-1 shows the progression of gross erroneous payments over time.

4   For further details on intentional tenant misreporting of income, please refer to the  FY 2015 Income

Match Draft Report, delivered to HUD on August 20, 2016.

ES-vi

August 31, 2016

Executive Summary

Exhibit ES-4

Comparative 2000 Through FY 2015 Gross Erroneous Payments*

Administration Type

Gross Erroneous

PHA-

Payments

Public

administered

Total PHA-

Owner-

(in $1,000s)

Housing

Section 8

administered

administered

Total

FY 2015†

$183,286

$400,778

$584,064

$163,758

$747,822

±$150,569

FY 2014

$247,580

$392,317

$639,897

$129,452

$769,349

±$167,657

FY 2013

$177,908

$324,293

$502,201

$105,628

$607,829

±$112,660

FY 2012‡

$190,849

$430,716

$621,566

$177,234

$798,800

±$148,415

FY 2011§

$139,885

$436,156

$576,041

$119,168

$695,209

±$108,728

FY 2010

$141,033

$341,515

$482,548

$167,719

$650,266

±$137,235

FY 2009

$130,268

$440,288

$570,556

$209,455

$780,011

±$162,116

FY 2008

$183,305

$400,248

$583,553

$191,723

$775,276

±$153,447

FY 2007

$149,364

$435,012

$584,376

$199,104

$783,480

±$157,292

FY 2006

$172,824

$520,020

$692,844

$261,324

$954,168

±$192,264

FY 2005

$220,464

$456,240

$676,704

$248,580

$925,232

±$164,206

FY 2004

$242,076

$521,220

$763,292

$224,460

$987,744

±$131,201

FY 2003

$316,116

$730,956

$1,047,072

$368,796

$1,415,844

±$163,000

2000

$602,556

$1,096,524

$1,699,092

$539,160

$2,238,252

±$275,000

Percent Reduction

from 2000 to FY 2015

69.58%

63.45%

65.62%

69.48%

66.55%

* Gross Rent Error is the sum of the absolute value of positive and negative rent error.

† For FY 2015, the population totals were updated to reflect the population in FY 2015.

‡ For FY 2012, the population totals were updated to reflect the population in FY 2012. In addition, the MTW program was included in the HUDQC study for the first time.

§ For FY 2011, the population totals were updated to reflect the population in FY 2011.

Note: Numbers may not add exactly due to rounding.

HUDQC Final Report for FY 2015

ES-vii

Figure ES-1

Comparative 2000 through FY 2015 Gross Erroneous Payments Over Time

A sharp decline in erroneous payments occurred from 2000 to FY 2004, from $2.2 billion to

$988 million. From FY 2004 through FY 2015, the Gross Rent Error was relatively less varied and

showed a general plateau.

C. Sources of Errors

Rent errors are often a result of a mix of different types of errors. In addition to dollar errors,

this study also examined administrative and component errors. For the purposes of this study,

administrative errors are analyzed separately from specific component errors.

Administrative Errors. Administrative errors are errors that result from administrative mistakes.

They consist of the following:

● Consistency errors, which are errors in logical conformity between elements within Form

HUD-50058/50059

● Calculation errors, which are arithmetic errors within subsections of Form HUD50058/50059

● Transcription errors, which are errors in transferring information from documentation in

the tenant file to Form HUD-50058/50059

● Failure to conduct a recertification in a timely manner

● Failure to verify information

ES-viii

August 31, 2016

Executive Summary

Component Errors. Component errors are related to the income and expense components used

to calculate rent. The income components are employment income, Social Security benefits and

pensions, public assistance, other income, and asset income. The expense and allowance components

are the elderly/disabled allowance, dependent allowance, medical allowance, child care allowance,

and disability allowance. Component errors often occur when project staff do not conduct a

thorough tenant interview or do not verify the information obtained during the interview. However,

component errors may also occur when the tenant supplies incorrect information, either intentionally

or unintentionally. The discussion explains methods used to identify the various types of errors,

error rates, and related estimated variances and the dollar costs of the various types of errors.

Consistency and Transcription Errors. The two most common administrative errors are

transcription and consistency errors. The HUD Public and Indian Housing Information Center

(PIC) and Tenant Rental Assistance Certification System (TRACS) data systems check the rent

calculations on Form HUD-50058/50059. For tenants for whom data are submitted (and corrected

if required), these systems virtually eliminate rent determination calculation errors for the items

included on the forms. However, not all cases are reported, and some cases that are returned to

program administrators for correction may either be ignored or changed in HUD systems without

the changes actually being implemented.

Overdue Recertifications. In general, HUD requires that every household be recertified annually.

About 2 percent of households had overdue recertifications in FY 2015, which was the same rate

found in FY 2014.

Verification Errors. Recognizing the issues associated with verifying tenant information, HUD

program staff have taken steps to clarify and—to some extent—simplify verification guidelines.

PIH Notice 2010-19, dated May 2010, and Housing Notice H 2010-10, dated July 2010, delineated

guidelines for new procedures for obtaining and using verification. The new HUD verification

guidelines were implemented at the end of FY 2010, and FY 2011 was the first fiscal year in which

they were applied. For the HUDQC Study, methodology was changed to reflect these new HUD

guidelines. In FY 2012, the acceptable verification date range for documents used by PHA/project

staff  was  extended  by  approximately  2  months  so  that  more  documents  in  the  tenant  file  met

the HUDQC Study requirements. In FY 2013, the study’s verification date range was revised to

provide a more accurate time frame for acceptable verification of documents. The changes included

new criteria for selecting the Quality Control Month and narrowing the time frame for acceptable

verification documents found in the tenant file to exclude any that were dated after the effective

date of the transaction being reviewed.

Obtaining income verification is often difficult. Even when repeated requests are made, employers

sometimes do not respond to requests for verification, or they require payment for the information.

Some program sponsors do a much better job than others of achieving third-party compliance

with written verification. The HUDQC Study shows that it is reasonable to expect all program

administrators to have as high a success rate as the current high performers. The study also shows

that there is significant room for improvement in using the verification data obtained.

Component Error Findings.   Incorrect income and allowance amounts were by far the most

significant sources of error when determining rents; only about 4 percent of households with rent

errors did not have an income or expense component error. Earned income (33 percent), medical

HUDQC Final Report for FY 2015

ES-ix

allowances (16 percent), pensions (15 percent), and other income (13 percent) continued to have

the highest percentage of households in error. Exhibit ES-5 shows the frequency of the most serious

component errors and the average dollar amount for each type. The percentage of households

represents households with any rent component error in which the specified rent component was

responsible for the largest error. The average dollar amount reported represents the average dollar

amount for the specified rent component for households in which the specified component was

responsible for the largest error. For comparison purposes, findings from FY 2014 are provided

in parentheses. While the percentage of households with specific rent component errors remained

relatively consistent from FY 2014 to FY 2015, there are often large differences in the annual

average dollar error from year to year.

Exhibit ES-5

Rent Components Responsible for the Largest Dollar Error for Households With Rent Error

Rent Component

Percent of Households

Annual Average Dollar Amount

Earned Income

33% (33%)

$3,860 ($4,528)

Medical Allowance

16% (17%)

$905 ($1,813)

Pensions

15% (15%)

$2,572 ($1,945)

Other Income

13% (13%)

$2,453 ($3,122)

Elderly/Disabled Allowance

6% (4%)

$400 ($422)

Dependent Allowance

5% (4%)

$522 ($566)

Public Assistance

4% (3%)

$2,133 ($1,519)

Asset Income

2% (3%)

$596 ($808)

Child Care Allowance

2% (2%)

$2,148 ($1,256)

No Rent Component Error

4% (7%)

$0

Total

100%

$2,326 ($2,625)*

* Refers to the sum of the dollars associated with the largest component in error divided by the number of households with that error.

Note: FY 2014 findings are provided in parentheses. Rounding may result in totals not equal to 100 percent. The cell size for elderly/

disabled allowance is small; therefore, estimates may not be reliable.

Exhibit ES-6 displays the impact of changes in the error threshold on the case error rate and gross

dollar error. As noted above, a monthly error of less than $5 is currently ignored due to rounding.

An increase in the error threshold of $5 to $10 would result in an increase in proper payments by

about 6 percent, as well as a decrease in the estimate for gross dollar error by about $28.8 million.

Based on the distribution of household error, most rent errors are within $100 per month, or

$1,200 per year. At the individual household level, the gross error may seem insignificant; however,

these errors can result in a substantial amount of gross dollar error for the assisted housing programs

in aggregate. Although an increase in the error threshold to $100 per month would result in

97 percent of cases being proper payments, the increased error threshold would not capture most

errors associated with improper payments.

ES-x

August 31, 2016

Executive Summary

Exhibit ES-6

Impact of Changes in the Error Threshold on Frequency and Estimates of Error (in $1,000s)

Percentage of Households

Dollar Error Amount

Rent

Rent

Rent

Rent

Monthly

Under-

Proper

Over-

Under-

Over-

Net

Error

payment Payment payment

payment

payment

Gross Error

Error

Exact Match

19.0%

61.4%

19.7%

$506.972

$256,933

$763,905

$250,039

Within $5

13.5%

73.6%

12.8%

$499,338

$248,484

$747,822

$250,854

Within $10

11.2%

79.8%

9.0%

$488,896

$230,173

$719,069

$258,723

Within $15

9.6%

83.5%

6.9%

$477,702

$215,132

$692,834

$262,570

Within $25

7.3%

87.8%

4.9%

$451,810

$191,948

$643,758

$259,863

Within $50

3.8%

94.5%

1.7%

$379,778

$129,017

$508,795

$250,761

Within $100

2.1%

97.3%

0.7%

$314,614

$90,526

$405,140

$224,088

*Row totals may not add up 100 percent to due to rounding.

D. Additional Findings

Eligibility  of  Newly  Certified  Households.  A  separate  analysis  of  newly  certified  households

(13 percent) was conducted to determine whether the households were eligible for HUD housing

assistance. Ninety-six percent of these households met all of the eligibility criteria—the same rate

found in new certifications in FY 2014. All certified households in the sample were income-eligible

on the basis of the QC income determination.

One percent of newly certified households failed to document Social Security numbers for one

or more household members, and 3 percent lacked the signed consent forms needed to authorize

verification of income and assets (for each member of the household who is at least 18 years old).

All households had the signed declaration forms or evidence accepted as proof of citizenship. These

findings estimate the percentage of newly certified tenants who were incorrectly determined eligible

for program admission.

Occupancy Standards. The study also determined the extent to which households are under- or

over-housed, relative to HUD’s occupancy standards. Seventeen percent of all households occupied

a unit with too many or too few bedrooms in FY 2015, according to the guidelines used for this

study. Historically, the percentage of households in units with the correct number of bedrooms

according to study guidelines has fluctuated between 83 percent and 88 percent since FY 2004.

Rent Reasonableness. The extent to which PHA-administered Section 8 Voucher rent comparability

(reasonableness) determinations are found in the tenant file was also investigated, along with the

method used to support the determinations. Ninety-six percent of new admission files contained rent

reasonableness documents, as did 89 percent of the files for households for which data were collected

for an annual recertification. However, the absence of documentation does not necessarily indicate

that a determination was not completed, only that it was not properly documented. Information was

also collected at the PHA level to understand the method used to determine rent reasonableness. To

determine whether the rent was reasonable, all PHAs in the study used unit-to-unit rent comparison,

unit-to-market rent comparison, or a point system.

Utility Allowances. For PHA-administered Section 8 Voucher households, the utility allowances

found on Form HUD-50058 were compared to the utility allowance worksheets found in the tenant

HUDQC Final Report for FY 2015

ES-xi

file and to the utility allowance values calculated using the utility allowance schedules provided

by the PHAs. For the first comparison, 83 percent of the utility allowance values matched. For the

second comparison, 87 percent of the values matched between the QC utility allowance amount

and that on the Form HUD-50058. However, nonmatching values may not necessarily mean the

utility allowance found on Form HUD-50058 was incorrect.

Payment Standards. A special analysis was conducted to determine whether the correct payment

standards were used for PHA-administered Section 8 Voucher households. The payment standard

found on Form HUD-50058 was compared to the payment standard schedules provided by the PHA

and to the Fair Market Rent (FMR) for the appropriate geographical area. For the first comparison,

82 percent of the payment standards matched. For the second comparison, 89 percent of the payment

standards found on Form HUD-50058 fell within the 90 to 110 percent FMR band. As with the

utility allowance analysis, the information needed to conduct the analysis was not always available.

Therefore, the fact that the payment standards did not match does not necessarily mean the incorrect

payment standard was used when calculating the amount of the tenant rent.

Form HUD-50058/50059 Rent Calculation Error Compared to QC Rent Error. The tenant

rent was calculated using only the data on Forms HUD-50058/50059 to determine the relationship

between errors detected using Forms HUD-50058/50059 and total rent errors found in the study.

When using only Form HUD-50058/50059 data to calculate rent, errors were found in 9 percent of

households. This is clearly different from the QC error calculation, in which calculation errors were

found in 26 percent of households. Calculation error was found in both Forms HUD-50058/50059

and the QC calculation in only 4 percent of households.

Automated Rent Calculation Systems. The study examined whether error rates in projects that use

an automated rent calculation system differ from errors in those using other calculation methods.

We did not find a difference between PHAs/projects that use automated rent calculation systems

and those that do not. This is not surprising because nearly all PHAs/projects use an automated rent

calculation system of some kind.

Tenant Characteristics and Project Characteristics and Practices. The conceptual approach

used for the multivariate modeling was updated to be a single combined model, but the analytical

approach used was the same used in FY 2014. The analysis identified patterns in which rent errors

related to project and household variables, particularly involving project-caused errors such as

transcription error and overdue recertification error and their association with Gross Rent Error.

These  findings  were  essentially  similar  to  those  reported  in  prior  years’  analyses—differences

among program types were not found to be statistically significant with regard to Gross Rent Error,

or subsidy overpayment, subsidy underpayment, net of other project and household effects. (See

Appendix F for more information on the Multivariate Analysis.)

Project-caused errors accounted for a large proportion of Gross Rent Error, controlling for

other effects. Of the project-caused errors, administrative errors, transcription errors, overdue

recertification errors, and the rate of items without third-party written verification predicted a higher

gross error, which has been consistently found in prior analyses. Transcription error was a driver

of overpayment, while the percentage of items without verification was a driver of underpayment.

Household background variables were strong predictors of Gross Rent Error, subsidy overpayment,

and subsidy underpayment. Variables indicative of complex financial conditions and income strongly

ES-xii

August 31, 2016

Executive Summary

predicted higher rent errors. The relationship between household financial or sociodemographic

variables and rent error is highly consistent across models and years, a finding which indicates that

PHA/project staff should emphasize quality control of these cases.

E. HUD Initiatives: 2000–2015

In response to the findings and recommendations of the 2000 Assisted Housing Quality Control

Study, HUD initiated a series of aggressive actions to address the causes of erroneous assistance

payments, including extensive onsite monitoring. Actions taken by HUD included the following:

1.   A Rental Housing Integrity Improvement Program committee, headed by the Office of the

Chief Financial Officer with representatives from other affected offices, was formed to

coordinate and monitor corrective actions. The committee meets to review progress and to

identify and resolve impediments to progress in reducing errors.

2.   The Office of Housing and PIH developed and issued new handbooks and instructional

materials that detailed all current HUD program requirements and standardized them to

the extent possible without regulatory or statutory change. The handbooks cover nearly

all aspects of occupancy policy, from the point of tenant application for admission and

rent calculations through ongoing occupancy to lease termination. For Public Housing, the

issuance of a Public Housing Occupancy Guidebook represented the first such effort in more

than 20 years and provided a defined methodology for calculating a number of complex

requirements (e.g., the Earned Income Disallowance).

3.   The Office of Housing and PIH substantially increased training efforts and held a number

of national and regional training sessions. This contrasts with a less educational approach

in the 1980s and 1990s.

4.   The Office of Housing and PIH initiated comprehensive, large-scale, and onsite occupancy

and management reviews, which also represented a major procedural change from the

previous two decades for most HUD offices:

● The Office of Housing primarily used new agreements with contract administrators,

which are usually State agencies, to perform this function. Contract administrators

provide technical support in adhering to HUD program requirements and routinely

perform detailed monitoring of agency compliance.

● PIH initiated a system of Rental Integrity Monitoring (RIM) reviews to detect and

reduce errors in income and rent calculations at targeted PHAs, reduce rent under- and

overpayments by residents, and ensure that HUD’s limited housing resources were

being used to serve eligible families in a fair and equitable manner, as intended by

Congress.

5.   HUD initiated a legislative change that granted it access to the U.S. Department of Health

and Human Services National Directory of New Hires (NDNH) income and wage database

for income matching purposes. HUD uses these data to compare tenant-reported income

with State wage data to better ensure that the right subsidy payments are made to the

right households in accordance with program statutory and regulatory requirements. This

legislation was passed in late 2003 and required the implementation of agreements and data

systems. HUD also negotiated agreements with some States to obtain access to the same

HUDQC Final Report for FY 2015

ES-xiii

information. Access to the NDNH database is available through the Enterprise Income

Verification (EIV) system.

6.  The Office of Housing and PIH initiated a computer matching program with the Social

Security Administration (SSA) that provides SSA data for tenants receiving assisted housing.

SSA electronically provides HUD with benefit information on all active household members

who have disclosed a valid Social Security number. HUD makes this information available

to administrators of the Public Housing and Section 8 programs through the EIV system.

This information allows PHAs to validate Social Security numbers and SSA benefits quickly

and efficiently.

7.   In 2010, HUD issued the  Implementation of Refinement of Income and Rent Rule, which

mandated the use of the EIV system (discussed in the previous two bullets) as a third-party

source to verify tenant employment and income information during mandatory recertification

of household composition and income. The use of EIV minimizes the need for traditional

third-party verification forms. To make the EIV system as effective as possible, the rule was

also revised to require all applicants and participants to disclose a Social Security number,

no longer exempting children younger than age 6.

HUD’s performance goals, which were developed in consultation with the Office of Management

and Budget, called for reducing the 2000 benchmark assisted-housing error levels by 50 percent by

the end of 2005. The study of program administrator error for FY 2005 showed that HUD exceeded

this goal, and HUD has further decreased error since. It should be noted, however, that the reduction

of errors and improper payments is unlikely to have an equivalent effect on budget outlays. HUD’s

experience has been that program integrity improvement efforts are likely to result in some higher

income tenants leaving assisted housing and being replaced with lower income tenants, requiring

increased outlays. Nevertheless, HUD’s goal remains to ensure that the right benefits go to the

right people.

F. Recommendations

HUD’s progress in decreasing improper payment since 2000 is impressive. However, findings from

the study suggest general actions that should be continued or policies that should be considered

to maintain or improve PHA/project performance in rent determination. We present the following

recommendations that may improve administrative error rates in HUD programs, based on insights

we have gathered during this and other studies:

1.  Continue Requiring the Use of EIV Reports. HUD should continue requiring the use

of EIV information in the process of rent determination. Data systematically collected

from  the  NDNH  and  SSA  provide  a  strong  method  of  identifying  specific  sources  of

income information. The study shows that the majority of subsidy errors are associated

with earned income. HUD may also want to consider forming relationships with State

programs, organizations, and companies to collect other data not currently captured by the

EIV system. Although EIV provides a uniform and efficient method of verifying income

sources that lessens the burden on program administrators, caution must be exercised when

using information from the system. The data are extremely helpful in identifying unreported

sources of income, but they are not current and sometimes contain errors (including

ES-xiv

August 31, 2016

Executive Summary

instances  of  identity  theft  and  incorrect  identification  of  disability  status).  HUD’s  EIV

requirement should be coupled with policies aimed at addressing the challenges of using

EIV for verification.

2.  Perform Onsite Review of Rent Calculation. HUD should continue onsite monitoring of

program administration, and PHA/Owners should be held accountable for implementing HUD

regulations and calculating rent accurately. Onsite monitoring that includes reviews at both the

local and Federal levels is essential to improving accountability. PHA/Owners with excessive

errors should be required to develop corrective action plans and show improvement within

specified time periods. Improved HUD monitoring was likely a key factor in reducing subsidy

error between the 2000 study and the current study.

We recommend that HUD require PHA/Owners to perform their own QC reviews on income

determinations and rent calculations. Agencies that have aggressively sought to improve

the performance of their programs have demonstrated success in this area, and one of

the most frequently used error-reduction strategies includes the establishment of internal

QC review procedures. Based on the Project Staff Questionnaire (PSQ) survey, it appears

that programs that conduct QC on all their transactions have a significantly lower rent

determination error rate than programs that do not perform QC on all their files. Of course,

a comprehensive approach may not be feasible, given limited staffing resources, but even a

review of a small percentage of transactions may be beneficial in supporting the reduction

of rent determination error.

In addition to internal agency reviews, HUD regional offices can support field offices by

conducting a secondary review of transactions. This review would provide HUD Federal

staff with more on-the-ground insights into the issues and challenges faced by local program

administrators. In addition, this approach would demonstrate HUD’s concern regarding

program integrity and improper payments, thereby focusing PHA/Owner attention on

accurately determining tenant income and rent.

3.  Continue to Streamline the Program Requirements. Continue to simplify Federal laws,

regulations, and HUD requirements to the extent possible. The new regulations outlined in

Notice PIH 2016-05 provide much needed clarification on previous regulations, such as the

time period associated with the Earned Income Disregard calculations. The implementation

of triennial certification for fixed income households will save time for PHA staff, allowing

them to spend more time conducting required reexaminations, following up on suspected

cases of fraud, and conducting more internal reviews of tenant files. Office of Housing

should implement similar changes.

4.  Create an Online Community to Share Best Practices and Tools. HUD should provide

PHA/Owners with an online venue to support the sharing of best practices for its assisted

housing programs. A Web-based resource could facilitate communication between HUD

and program administrators regarding identifying ways to improve and address challenges

related to proposed policies. Comprehensive supporting documents, including forms for

interviewing tenants, obtaining verification, and determining rent, could be posted to the site

for downloading. Manuals and training materials describing how to implement requirements

HUDQC Final Report for FY 2015

ES-xv

and accurately calculate rent could also be available electronically, with online webcasts

providing an additional training resource for local program offices.

HUD experts and local housing staff should be given the opportunity to work together to

develop tools and systems that reduce rent error. Many local PHA/Owners have already

developed forms, training materials, manuals, automated systems, and monitoring processes

that enable them to provide accurate, efficient service to their tenants. HUD should create

a platform for organizations to learn from each other.

5.  Develop a technical assistance platform for rent calculation staff. HUD should develop a

technical assistance hotline for project/PHA staff to call about particularly difficult policies

related to completing (re)certifications. The assistance could be a Web-based support system,

where the PHA/project staff would be given a time estimate of when they will receive an

answer after submitting a question, or a live telephone system staffed by experienced HUD

policy advisors. This open dialogue would help avoid common errors that contribute to rent

error. Additionally, HUD should keep detailed records for each call to determine if there

were commonly asked questions, and provide written guidance on these issues to all PHA/

projects for future use.

6.  Develop a calculation worksheet for use in annualizing income. This electronic worksheet

would be programmed to calculate annual income to be used for rent calculation. It would

also serve as documentation of the methodology used by the PHA/project staff to annualize

income. A physical copy could be kept in the tenant file after it is dated and signed by the

certification staff. This would help to avoid calculation errors and thus reduce rent error.

In addition to providing general program recommendations to decrease error rates, we seek to

improve the HUDQC Study that provides the estimates of the error rates. The current methodology

used by ICF to conduct the study is based on established study objectives and builds on insights from

previous studies. The following recommendations serve to expand the utility of the data collected,

support HUD’s research goals, and improve the overall efficiency of ongoing QC studies.

7.  Continue to Measure Improper Payment Associated With Billing Error. HUD should

conduct billing error studies to obtain a more accurate assessment of improper payments.  In

the  FY 2014 HUD Agency Financial Report, billing error estimates are based on FY 2004

data for the Public Housing program and FY 2009 data for Owner-administered programs. In

FY 2015, ICF conducted billing studies for both the Office of Public and Indian Housing and

the Office of Housing. This work should continue to be conducted. Current error estimates

could be obtained by conducting primary data collection or by using statistical modeling

to update the existing data. However, an updated study will provide HUD with a better

assessment of billing error associated with rental assistance programs in order to understand

one of the main contributors to improper payments. The information from these billing

studies could strengthen financial management controls so that HUD can better detect and

prevent improper payments.

8.  Incorporate Additional Objectives in the HUDQC Study. Data collected through the

HUDQC Study provide details that are not available through other HUD sources (e.g., PIC/

TRACS) that could be used to track trends, such as the extent to which income and expense

items are verified or the number of sources of employment income for a particular household

ES-xvi

August 31, 2016

Executive Summary

or household member. Furthermore, because a statistically valid nationwide sample

of projects and households is created for the study, other HUD-related topics could be

investigated using the HUDQC Study’s research mechanisms and data collection processes.

The rental integrity monitoring (RIM) review validation, identified in the July 2013 issuance

of the HUD Research Roadmap for FY 2014–FY 2018, is a task that could be incorporated

into the HUDQC Study’s data collection process. Additional topics could include a review

of the changing demographics of HUD tenants, participant satisfaction surveys, and a more

in-depth review and evaluation of MTW programs.

9.  Continue to Conduct a Utility Allowance Comparison Study. In response to tightening

budgets and overall concerns with energy efficiency, HUD should undertake a study to

better understand utility costs and consumption in subsidized housing. ICF conducted this

work in FY 2015, and this work should continue. HUD should consider conducting an

in-depth quality control study of how utility allowance values are calculated and used in rent

calculation. This study could involve collecting data from utility companies regarding utility

use for a given fiscal year and comparing actual consumption with the utility allowance

subsidy calculated by program administrators.

The FY 2015 Utility Allowance Comparison Study found that current HUD Utility Schedule

Model (HUSM) users produce incorrect allowances via the tool. HUD should consider

making improvements to the HUSM tool to increase accuracy of data entry and calculated

utility allowances. Additionally, performing project-level reviews and approvals of utility

allowance levels would improve accountability in updating allowances in a timely manner

and would likely reduce subsidy error in utility allowances.

10. Learn More About PHA/Project Policies and Practices.  Each PHA establishes its own

policies, procedures, and forms for collecting information that is ultimately used to calculate

tenant rent. The differences in these practices should have some (possibly major) impact

on rent error, yet the analysis of the project practices and characteristics collected by the

PSQ does not demonstrate the expected impact. We recommend conducting focus groups,

interviews, and discussions with program administrators to identify additional PHA/

project-level factors that may impact error. This information could be used to revise the

PSQ to include questions focused on the specific practices expected to influence errors. The

analysis of more detailed, project-level data would assist in this process.

11. Continue to Perform the HUDQC Study. The HUDQC Study provides a consistent

ongoing method of monitoring, managing, and improving HUD rent determination processes.

The ongoing evaluation of HUD rental housing assistance programs is essential to program

management and improvement, and rigorous research is important for understanding how

well HUD programs are reaching their goals for the communities served. The primary

objective of the HUDQC Study is to measure rent calculation and improper payment error;

however, the study also gives HUD the opportunity to learn more about methods to reduce

rent calculation errors and better manage current and changing conditions at PHAs/projects.

Annual evaluations facilitate more accurate, cross-year comparisons of rent errors. They

also allow data collection and data analysis staff to develop specific expertise in HUD policy

areas, supporting the development of tailored solutions for improving data quality.

HUDQC Final Report for FY 2015

ES-xvii

I. Introduction

The U.S. Department of Housing and Urban Development (HUD) provides housing subsidies

to Multifamily project owners and Public Housing Authorities (PHAs) to administer housing

assistance primarily to low-income households. The Office of Public and Indian Housing (PIH) and

the Office of Multifamily Housing provide funding for rental subsidy through Public Housing, the

PHA-administered Section 8 Voucher program, and the Owner-administered Section 8 project-based

programs. Collectively, these programs are referred to as HUD’s Rental Housing Assistance

Programs (RHAP). They are administered by more than 4,000 intermediary agencies and provide

affordable housing for approximately 4.97 million households (i.e., 1.1 million though Public

Housing, 2.2 million through the PHA-administered Section 8 program, and 1.6 million through

project-based programs).5

Under the Improper Payments Elimination and Recovery Act (IPERA), signed into law in 2010, and

the guidance of the Office of Management and Budget (OMB), agencies assess all programs they

administer and identify those that may be susceptible to improper payments. An improper payment

is any payment that should not have been made or that was made in an incorrect amount. In fiscal

year (FY) 2014, $32 billion of HUD’s total payments were attributed to HUD’s rental assistance

programs. These programs constitute a significant amount of HUD’s total payments and continue

to be assessed as being at high risk of significant improper payments.6

During this challenging economic period, it is more important than ever to evaluate program

administration and internal controls to maintain sustainable, quality programs that meet the needs

of communities. The reduction of improper payments directly increases the efficacy of HUD’s

housing programs and ensures that Federal dollars are being allocated fairly across the nation. This

section outlines the purpose of the Quality Control for Rental Assistance Subsidy Determination

Study (HUDQC Study), provides background information on the study, and explains how the report

is organized.

A. Purpose of the Improper Payment for Quality Control for Rental

Subsidy Determinations Study for FY 2015

ICF International was contracted to perform the HUDQC Study to support HUD’s continued

dedication to reducing the amount of annual improper payments in its programs and to comply with

the reporting and administrative requirements under IPERA. The HUDQC Study provides national

estimates of the level of improper payments and rent calculation error in tenant subsidies for Public

Housing; PHA-administered Section 8 Vouchers and Moderate Rehabilitation programs; and the

Owner-administered Section 8, Section 202, and Section 811 Project Rental Assistance Contracts

(PRAC) and Section 202/162 Project Assistance Contracts (PAC) programs. For the purpose of

this study, error is defined as any rent calculation or eligibility decision that is determined based on

methods discrepant from HUD’s income certification and rent calculation requirements. The study

5  U.S. Department of Housing and Urban Development.  Annual Report: FY 2015 Agency Financial

Report.  Washington, DC: U.S. Department of Housing and Urban Development, 2015. pg. 12

6   U.S. Department of Housing and Urban Development.  Annual Report: FY 2015 Agency Financial

Report.  Washington, DC: U.S. Department of Housing and Urban Development, 2015. pg. 213

HUDQC Final Report for FY 2015

I-1

I. Introduction

examines the sources, associated costs, and frequency of subsidy errors in tenant certification and

annual recertification processes for recertification transactions conducted during Federal FY 2015.7

This report examines a total of 16 objectives and outlines them in more detail in Section III. The main

focus of this work involved collecting and analyzing information that pertained to eligibility and

rent determination processes to identify possible causes of error in rent calculation. Throughout this

report, information is reported for the three major housing programs separately and in combination.

As a separate analysis, key error estimates are also provided for the 20 largest PHAs included in the

study sample. In addition, some special analyses were conducted regarding PHA utility allowances,

payment standards, and rent reasonableness practices. As part of our review, we also compared unit

size to household size to identify any errors in the determination of unit size.

B. Study Background

HUD defines potential rental assistance improper payment based on three major error types. These

error types include the following:

1.   Program administrator error,  which is the program administrator’s failure to correctly

determine eligibility and income and to apply all income exclusions and deductions when

conducting the recertification.

2.   Tenant income reporting error,  which   is a consequence of the tenant’s failure to disclose all employment income and unemployment compensation sources.

3.   Billing error,  which occurs when there is incorrect billing and payment of subsidies between

HUD and third-party program administrators and/or housing providers.

The FY 2015 HUDQC Study is the 14th in a series of studies designed to:

●  Identify potential metrics for improper payments error, including HUD eligibility

determination, income calculation, and rent calculation.

●  Translate regulations for HUD programs (i.e., Public Housing, PHA-administered Section

8, and Owner-administered projects) into data collection and survey instruments.

●  Develop  an  error  detection  system  for  flagging  inconsistencies  in  household  data  and

establishing an internal quality control process for data collectors.

●  Provide nationally representative estimates of rent subsidy errors.

Activities for the FY 2015 HUDQC Study commenced in December 2015, starting the review of

recertification transactions effective November 1, 2014, to October 31, 2015. Tasks completed

prior to data collection that have not been listed above included designing the research and survey

methodology and automating the data collection process. Data were collected from a nationally

representative sample of HUD-assisted housing projects, and participant household data were

collected from tenant files, household interviews, and third-party verification when necessary.

7   PHAs and owners of HUD-assisted housing are required to make an initial determination of eligibility

and, thereafter, an annual recertification of each household’s rent. In this report, the term recertification

refers to the initial certification and annual recertification. This study does not include interim

recertification transactions.

I-2

August 31, 2016

I. Introduction

C. Organization of This Report

This report is organized into the following sections:

● Section I: Introduction

● Section II: Methodology

● Section III: Study Objectives and Analytic Methods

● Section IV: Findings

● Section V: Recommendations

● Appendices

○ Appendix A: Rent Calculations

○ Appendix B: Weighting Procedure

○ Appendix C: Source Tables

○ Appendix D: Consistency and Calculation Errors

○ Appendix E: Project Staff Questionnaire Descriptive Analysis

○ Appendix F: Multivariate Analysis

D. Definitions of Key Terms

The HUDQC Study uses key terms for the study of RHAP rent calculation error and improper

payments. These key terms are used throughout the report and can be referenced here:

● Abstract Month—The month in which the data collection process for any given household

was initiated

● Actual Rent—The tenant rent listed on the Form HUD-50058/50059

● Administration Type—PHA or owner

● Calculation Errors—Arithmetic errors within subsections of the Form HUD-50058/50059

● Case Type—Certification, recertification, and overdue recertification

● Component Errors—The income components (i.e., employment income, Social Security

and pensions, public assistance, other income, and asset income) and expense components

(i.e., elderly/disabled allowance, dependent allowance, medical expenses, child care

expenses, and disability expenses) responsible for an error in rent calculation

● Consistency Errors—Errors in logical conformity between elements within the Form

HUD-50058/50059

● Dollar Rent Error—The difference between the household’s QC Rent (see definition below)

and the Actual Rent

● Error Rate—The sum of the dollar amount of Rent Error divided by the sum of the dollar

amount of the QC Rent

● Gross Rent Error—The sum of the absolute values of under- and overpayments

HUDQC Final Report for FY 2015

I-3

● Largest Component Dollar Error—The annual dollar amount of error in the component

with the largest error

● Net Rent Error—The arithmetic sum of underpayments and overpayments

● (Rent) Overpayment—Results when the household paid more than it should have paid,

making HUD’s contribution less than it should have been

● Payment Type—Underpayment, proper payment, or overpayment

● Program Type—Public Housing, Section 8 Housing Choice Voucher, Section 8 Moderate

Rehabilitation, Section 8 project-based, Section 202 PRAC, Section 811 PRAC, or Section

202/162 PAC

● Quality Control Date—The day the tenant rent was calculated by the project staff; this date

is used to determine whether verification is acceptable

● Quality Control Month—The month in which the PHA/owner completed the rent calculation;

used during the household interview to obtain data for the correct time period

● Quality Control (QC) Rent—Calculated by ICF using the tenant file, household interview,

and verification data

● Rent  Component—One  of  the  five  sources  of  income  (i.e.,  earned,  pensions,  public

assistance, other income, and assets) or the five types of deductions (i.e., medical, child

care, disability assistance expenses, dependent allowance, and elderly/disabled allowance)

● Rent Error—The difference between the monthly Actual Rent and the monthly QC Rent

● Total Component Dollars in Error—The absolute sum (i.e., the sum of the positive and

negative amounts, ignoring the plus or minus signs) of all individual income and expense

component errors, combined to provide an overall Total Dollars in Error and presented as

an annual amount

● Transcription Errors—Errors in transferring information from documentation in the tenant

file to the Form HUD-50058/50059

● (Rent) Underpayment—Results when the household paid less than it should have paid,

making HUD’s contribution higher than it should have been

I-4

August 31, 2016

II. Methodology

A. HUD Requirements and Study Standards

ICF used the  Code of Federal Regulations and official HUD handbooks and notices to consolidate

all HUD rules relevant to the determination of rent into a set of HUD requirements. We used these

requirements to create a uniform set of rules that could identify errors in eligibility determination,

rent calculation, and unit assignment for the housing programs in the study. In general, this

uniform set of rules—known as the standards—follows the official HUD requirements. However,

for some complex requirements, standardized procedures were developed to ensure a uniform

manner of data collection. A complete list of the standards used in this study can be found in the

Final FY 2015 Improper Payment for Quality Control for Rental Subsidy Determinations Data

Collection Standards.8

B. The Sample

The initial sampling design called for a nationally representative sample of 600 projects with

4 households randomly selected from each project, equaling 2,400 households. We selected projects

with probabilities proportional to size, but 8, 12, or more households were selected from larger

projects whose size exceeded the sampling interval; these were counted as more than one project

for the purpose of determining the sample size. The sampling design required approximately equal

allocations for the three assisted program types: Public Housing, PHA-administered Section 8

(Vouchers and Moderate Rehabilitation), and Owner-administered (Section 8, Section 202 PRAC,

Section 202/162 PAC, and Section 811 PRAC).

Project Sampling. Certain projects were excluded from the study because of their different

eligibility and rent calculation rules, such as Owner-administered Rental Assistance Payment

or Rental Supplement Program (RAP/SUP) projects. Based on a discussion with HUD, Owner-

administered projects that were in the process of converting via the Rental Assistance Demonstration

(RAD) program were excluded from the project frame files in FY 2015. This decision was due to

the difficulty of determining rental subsidy error in units undergoing the transition. Universe files

requested from HUD either excluded out-of-scope projects or those projects were identified for easy

removal. Given that some large projects were selected multiple times, the study sample included

583 distinct projects in 57 geographic areas in the United States and Puerto Rico. We sampled 200

projects from each major program type9 and collected data for a multiple of 4 households from each

project. An additional project was added to the sample to ensure that the sample would include a

minimum of 2,400 households, even if unexpected circumstances were encountered. The final data

set includes responses from 2,400 households in 583 projects.

Household Sampling. For the second year in a row, ICF selected households using HUD-provided

PIH  Information  Center/Tenant  Rental  Assistance  Certification  System  (PIC/TRACS)  data.

Previously, field interviewers selected the sample of households while onsite at PHAs/projects

8   ICF International unpublished report to HUD dated September 18, 2015.

9   For the purpose of this study, a “project” for the Section 8 Voucher program is defined as the

administration of the program in one county/township. Therefore, if a PHA administers vouchers in more

than one county/township, the PHA could be represented in this study by more than one “project.”

HUDQC Final Report for FY 2015

II-1

II. Methodology

using PHA/project-supplied tenant rosters. ICF automated this process using PIC/TRACS data

that included all active non-Moving to Work (MTW) tenants who had been certified or recertified

in FY 2015. Due to nonstandard recertification cycles permitted by various MTW PHAs, ICF’s

PIC/TRACS data request did not include MTW households. Certain MTW PHAs selected for the

FY 2015 study allow tenants to have biennial or triennial recertifications, so those tenants would

not be included in a list of all active and assisted tenants who were either certified or recertified

in FY 2015. In order to not exclude eligible tenants from the study sample, ICF collected tenant

rosters for MTW projects from the individual PHAs/projects and selected those samples using the

previous method used for random sampling.

A random sample of 4 households was selected from most projects, with some larger projects having

a larger random selection of 8, 12, or more households. For example, 16 PHA-administered Section

8 Voucher projects, including those in New York City (NYC) and Los Angeles, had household

sample sizes of 12 or more. An equal number of “replacement” households were identified at each

selected project as potential substitutes in the event that a selected household did not meet the study

requirements or was unavailable to be interviewed.

Sampling for the 20 Largest PHAs. Once  the  sample  for  the  HUDQC  Study  was  identified,

additional projects and households were selected for the 20 largest PHAs in the study sample.

This additional sample allowed us to provide supplemental findings for these large PHAs. At least

32 cases were sampled per PHA. If a PHA’s HUDQC Study sample size was sufficiently large,

we did not supplement it; however, if only a few households were sampled from the PHA, we

added substantially to the sample. As in the HUDQC Study, we allowed vouchers to be selected

more than once. Since we selected households in groups of four, we aimed for eight projects per

PHA, with possible multiple selections for the PHA-administered Section 8 Voucher and Moderate

Rehabilitation projects. The resulting sample yielded 71 new projects that were not selected for the

HUDQC Study and 268 new households. For additional information on the sampling procedures,

see the  Final FY 2015 Improper Payment for Quality Control for Rental Subsidy Determinations

Sampling Plan.10

Weighting. In studies from FY 2004 to FY 2011, Owner-administered RAP/SUP projects and MTW

projects in Public Housing and Voucher programs were excluded from the population totals because

of the differences in their eligibility and rent calculation rules. Beginning in FY 2012, however,

MTW projects were included in the study, at HUD’s request.

In studies from FY 2005 to FY 2010, the population totals from the June 13, 2005, request for

proposal were used as the basis for the estimate of occupied units in each of the programs. In

FY 2011, a comparison of the previous population totals to the frame population totals showed

a change sufficient enough to warrant updating the population counts. In FY 2012, the inclusion

of MTW projects led to an increase in the population. These population totals were used until

FY 2014. For FY 2015, these population totals were updated per HUD’s request. Exhibit II-1

displays the changes to frame population totals over time.

10  ICF unpublished report to HUD dated September 18, 2015.

II-2

August 31, 2016

II. Methodology

Exhibit II-1

Change in Frame Population Totals Used to Pull the Study Sample Over Time

FY 2005–

FY 2010

FY 2012–

Population

FY 2011

FY 2014

Program Type

Totals*

Study Sample* Study Sample**

FY 2015**

Public Housing Total

955,000

1,052,503

1,154,796

1,061,690

Public Housing (non-MTW)

955,000

1,052,503

1,040,708

959,766

Public Housing (MTW)

0

0

114,088

101,924

PHA-administered Section 8 Total

1,858,000

1,912,467

2,198,722

2,209,296

PHA-administered Section 8 (non-MTW)

1,858,000

1,912,467

1,935,597

1,916,735

PHA-administered Section 8 (MTW)

0

0

263,125

292,561

Owner-administered

1,320,000

1,382,670

1,378,158

1,382,453

Total

4,133,000

4,347,640

4,731,676

4,653,439

* Excluding RAP/SUP and MTW populations

** Excluding RAP/SUP; including MTW

The use of the same population counts increases the comparability of data, so any change from year

to year would not be due to a change in the number of households in the program but to an actual

change in the average gross dollar error or percentage of households that are in error. However,

maintaining constant population counts over time despite changes in the population itself may

result in estimates for total dollar amounts and the proportion of the population represented by

each program type not being representative of the current population. Based on these reasons and

given the inclusion of MTW projects in the FY 2012 sample and with HUD’s agreement, ICF

updated the population counts for the FY 2012 study and continued to use those counts for the

FY 2013 and FY 2014 studies.11 In order to ensure accuracy with the current rental assistance

population, these population counts were updated for FY 2015 based on the FY 2015 HUD sampling

frame.

C. Data Collection

This study used a multistage data collection process to obtain all required information. Web surveys

provided project-level information from PHA/project staff. Tenant-level information was obtained

by field interviewers who abstracted data from the household file, interviewed households, and

requested tenant consent to verify income, expense, and household composition items from third

parties.12 Tenant income, expense, allowance, and third-party verification information was collected

using HUD-sanctioned data collection procedures. Field data collection began in December 2015

and ended in May 2016.

Most data collection activities were categorized as either project level data collection or household

level data collection, but select processes applied to both.

11  For a more detailed discussion of population total updates, please reference  Appendix B.

12  Verification is a process of obtaining information about income or expenses from a third party that can

attest to the accuracy of the information provided by the household. HUD requires that most information

provided by the household be verified by a third party or substantiated using documents (e.g., printouts

from the Enterprise Income Verification [EIV] system).

HUDQC Final Report for FY 2015

II-3

Creating the Data Collection Instruments. For this study, more than 30 data collection instruments were used to collect data at both the project and tenant levels. These instruments were similar to

those used for the previous data collection efforts, although instruments were modified to improve

the data collection process. Project-level instruments were used to gather information to facilitate

data collection, collect the data elements necessary to calculate QC rent, and gather information

about certification and recertification practices. The tenant-level data collection instruments were

created to collect data and determine whether:

1.  There were errors in the eligibility determination

2.   The household rent was calculated correctly

3.   Units were correctly assigned according to the study standards

Each instrument was created by a survey research specialist and reviewed by a HUD policy expert.

The Office of Management and Budget approved all data collection instruments.

Automating the Data Collection Process. This study used an enhanced version of the data collection

system than was used in previous studies. Project-level data were collected through the PSI and

PSQ Web surveys that were developed using Select Survey Software. Data from household files

were entered directly into laptop computers, and a computer-assisted personal interviewing (CAPI)

system was used to interview tenants. This system, referred to as the HUD Data Collection Software

(HDCS) system, was developed by a special team of ICF survey specialists and computer systems

experts.13

Project Level Data Collection

Project Level Web Surveys. The initial collection of project-level data began in October 2015

with the Web-based Project Specific Information (PSI) questionnaire. Using contact information

obtained from HUD, this survey requested background information essential to the data collection

process as well as specific data used for the calculation of QC Rent. The rent calculation information

requested varied by program, but included questions relating to items such as passbook rate, utility

allowance schedules, payment standards, minimum rent, and flat rent. PHA/project staff verified the

project type and size and the location of project offices and files. Projects were also asked to indicate

whether the selected project had been designated a “special demonstration project” by HUD. If a

project answered “yes” to this question, and this status was confirmed, the project was replaced in

the study. In addition, PHA-administered Section 8 Voucher projects and Public Housing projects

were asked to provide their Administrative Plan and their Admissions and Continued Occupancy

Policy (ACOP) documents. Public Housing projects were asked to identify the location of any

information on permissible deductions. Administrative Plans and ACOPs were thoroughly reviewed

for local discretionary policies that would impact QC rent determination.

The data requested from the PHA/project were essential to the calculation of the QC Rent and to

preparing interviewers to begin the process of data collection. For these reasons, a 100 percent

13  The base of HDCS is the Census and Survey Processing System (CSPro) software package, which is

used by the U.S. Agency for International Development to collect demographic and health information in

many countries.

II-4

August 31, 2016

response rate to our request for information was targeted. In FY 2015, 99 percent of the projects

selected responded to the survey and supplied the necessary documents. Rigorous strategies were

employed to ensure compliance and the completeness of requested information prior to field data

collection.

Another Web-based survey, the Project Staff Questionnaire (PSQ), was sent to projects in February

2016. This survey was designed to obtain information from PHA/projects about the processes that

they use to calculate rent during certifications.

Because PHAs/projects have varying practices, ICF designed data collection instruments

and guidelines for data collection that were flexible enough to obtain data from the variety of

circumstances found in PHAs/projects. The major tasks accomplished during data collection and

the instruments used to accomplish those tasks are discussed as follows.

Contacting the PHA/Project. PHA/project contact names were obtained from HUD headquarters

staff. Emails were sent to PHA/project staff advising them of the study and requesting their

participation.  When  field  data  collection  commenced,  PHA/projects  were  sent  communication

introducing the field interviewer and requesting cooperation during their site visit.

Household-Level Data Collection

Hiring and Training Field Interviewers. Seventy-three field interviewers were hired to complete

the field data collection, and each interviewer was assigned a group of projects. Field interviewers

typically lived in the same general area as the projects selected for the study. Fifty field interviewers

who had not worked on the previous year’s study (for FY 2014) attended a 7-day training;

twenty-three experienced interviewers who completed the FY 2014 study attended a

3-day training. The 7-day training covered the following topics:

● Project background

● HUD programs and requirements

● Survey procedures

● Automated data collection

● Administrative procedures

The 3-day training covered a review of the project background and data collection procedures and

focused on changes implemented for the FY 2015 study.

Abstracting From Household Files. At  certification  and  recertification,  PHAs/projects  must

complete either Form HUD-50058 (for each household in Public Housing and PHA-administered

Section 8 programs), Form HUD-50058 MTW (for each household in MTW projects), or Form

HUD-50059 (for all other programs in the study). Data from Form HUD-50058/50059 were entered

directly into HDCS on each field interviewer’s laptop computer. As the data were entered, the system

identified potential data entry errors, such as incorrect codes or numbers, on the basis of internal

calculations and consistency checks. These electronic checking procedures enabled field interviewers

to make immediate corrections and updates.

HUDQC Final Report for FY 2015

II-5

This structured, automated process greatly reduced the need to edit, code, and clean the data after the close of data collection. HDCS data were securely transferred to study headquarters electronically

on a daily basis. The incoming data were reviewed in an ongoing QC process. This continual data

review during the collection process ensured data accuracy and permitted study headquarters staff

to resolve issues or request other clarifying documents while interviewers were still in the field.

HDCS was designed to collect data in the same format as the official Form HUD-50058 and Form

HUD-50059 published by HUD. The NYC Housing Authority uses a Form HUD-50058 format that

differs slightly from the standard. However, because of the large number of NYC Public Housing

units and PHA-administered Section 8 Voucher cases in the study, copies of the corresponding PIH

Information Center (PIC) 50058 data for these cases were requested and used for data collection

when available. In previous study years, we encountered projects where Form HUD-50058 differed

from the official HUD format. In those cases, ICF developed crosswalks by examining the data

elements on the atypical forms and developing a plan that illustrated which fields corresponded

to the standard Form HUD-50058. In the FY 2015 study, three nonstandard documents required

crosswalks, compared to four in FY 2014. These three documents were used by three projects, two

of which were MTW PHAs.

In addition to the data collected from Form HUD-50058/50059, field interviewers collected data

from the household files to document the determination of tenant eligibility and the calculation

of rent. A series of documents that supported the certification action were copied by the field

interviewer to verify income, assets, household composition, expenses, and other items needed

for accurate rent calculations. The documents may have been supplied to the PHA/Project by the

tenant or by a third-party agency. When these documents met various study verification criteria,

they were used in the QC rent determination.

Interviewing Tenants. For this study, an adult household member (preferably the head of household)

was interviewed in person using CAPI. Interview questions focused on family composition as

well as sources and amounts of income, assets, and applicable expenses. Data were collected for

the same point in time that the recertification was conducted. HDCS compared data from Form

HUD-50058/50059 with those entered during the interview to alert the interviewer to possible errors.

While interviewing tenants, field interviewers also requested specific verification documents, which

they then scanned and securely sent to study headquarters. Those documents were reviewed if the

supporting documents from the tenant file did not meet study verification criteria.

Requesting Verification From Third-Party Sources. When there was no evidence in the household

file that the PHA/Owner verified the information used for calculating rent, or the existing verification

information did not meet the requirements for this study,14  ICF  requested  verification  from

appropriate third-party sources. Verification was also requested from third parties when household

interviews resulted in the identification of sources of income that were not found in the household

files. Tenants signed release forms during the household interview so that third-party verification

14  For the purposes of this study, verification was acceptable if it was in writing, was received from a third

party, and was dated with the Quality Control Month (QCM) date or within 119 days prior to that date.

Acceptable verification could include documentation from a third party brought in by the tenant if the

documents met specific date criteria.

II-6

August 31, 2016

of income and expenses could be obtained. In addition, release form cover letters were also signed

by all adult members of the household to ensure that third parties contacted for the verification of

information would be satisfied with the validity of the request. Third-party entities completed the

forms and returned them to study headquarters, where data were compared to other file information.

Matching Social Security Data. Sample household members were matched with Social Security

Administration (SSA) files by HUD. The output from this match identified the Social Security

and Supplemental Security Income (SSI) benefit, as well as the Medicare premium data, for all

household members. These data were considered third-party verification during the final QC Rent

determination.

D. Field Data Collection Time Periods

Data were collected in the field between December 2015 and May 2016 for the certification or

annual recertification that occurred during FY 2015 (November 2014 through October 2015). Field

interviewers collected data related to actions that may have occurred up to 18 months prior to the

file abstraction and household interview. In collecting data to document actions taken in the past,

a major challenge was to develop methodologies to ensure that collected data reflect the situation

that existed at the selected point in time. For the respondent in the household interview, it may be

difficult to recall details of life situations at a past point in time. Some respondents in this population

may have unstable situations resulting from inconsistent income or changes to household size,

further complicating the collection of data from the past. In light of these challenges, ICF developed

study constructs to ensure the consistent and accurate collection of data across program types,

projects, and households in the study. The information below describes the two primary strategies

developed for this purpose: the quality control month and third-party verification rules.

Quality Control Study Time Period. The month for which data were collected is referred to as

the Quality Control Month (QCM) and was used during the household interview to obtain data

for the correct time period. This time period refers to when an eligible action occurred during

FY 2015. The Quality Control Date (QCD) refers to the day the tenant rent was calculated by the

project staff. For most households in the Owner-administered program, the QCD is the date on

which the project manager (or other authorized housing project staff member) or the tenant signed

Form HUD-50059, certifying that the information on the form was correct. For most households

in the PHA-administered programs, the QCD is the date on which the rent calculation worksheet

was signed. If these pieces of information were not available, the field interviewer used other

documentation in the household file to determine when the action was taken.

After the QCD was established, the data from Form HUD-50058/50059 corresponding to the

selected action was entered into HDCS. The data from the documents used by project staff to

verify information on Form HUD-50058/50059 on the QCD were also entered in a separate HDCS

module. The household interview included frequent reminders to the respondent that the questions

being asked pertained to the QCM.

Note: If the recertification was overdue by more than 12 months, 12-month intervals were added

to the QCM so that the QCD fell within the FY 2015 review period. In this situation, during the

household interview, the respondent was questioned about circumstances for the month in which the

recertification would have been completed had housing project staff completed it on time. In rare

HUDQC Final Report for FY 2015

II-7

situations when the rent was calculated after the effective date of the action (because of retroactive adjustments), the QCM is the date of that action.

Third-Party  Verification  Rules.  Occasionally  the  verification  documents  found  in  the  file

for household composition, income, assets, and expense items were different from those

required by HUD.

To ensure that data from the correct documents (i.e., those that were gathered to verify the

information on the certification or recertification under review) were used to calculate QC Rent,

and to apply rules fairly and consistently across all households in the study, ICF developed a

set of guidelines defining acceptable verification. For the purpose of this study, verification was

considered acceptable if it was either given verbally or in writing, was from the relevant third party,

and was dated within the 4 months leading up to the time period of interest. These documents were

considered acceptable whether they were received directly from the third party, provided by tenants

during the recertification process, or submitted during the household interview.

ICF study headquarters staff classified these documents and determined whether each document met

the acceptability criteria. For items that did not meet the requirements, verification was requested

from the appropriate third-party entity.

E. Construction of the Analysis Files

The  initial  data  files  consisted  of  information  obtained  from  the  tenant  file  at  the  project  site,

information from the household interview, and third-party verification data. Data items were collected

at both the member and household levels. ICF constructed an analysis file that annualized all income

and expense data at the household level. For some items, such as stable income from Social Security,

this calculation was relatively easy. For other items, such as sporadic employment or medical

expenses, annualizing income or deductions was more complicated. A unique linking variable was

created to compare information abstracted from file documentation with information obtained in the

household interview and received from third-party verification. This variable specifically identified

the income, asset, or expense and the household member to which the item belonged.

For the calculation of rent error, the final analysis files contained income and expense/allowance data

aggregated at the household level in annual amounts. Rent data were in monthly amounts. Separate

files were created for the analysis of issues such as verification, internal Form HUD50058/50059

errors, and occupancy standards.

F. Rent Formula

HUD uses a specific set of rules for determining tenant rents for each of its programs. The algorithm

for determining the Total Tenant Payment (TTP) is the same for all programs except Sections 202

PRAC, 811 PRAC, and 202/162 PAC and MTW. The TTP is the greater of the following:

● Thirty percent of a household’s adjusted monthly income, defined as one-twelfth of the

total of all household members’ earned and unearned income—other than those amounts

specifically  excluded  by  HUD  or  PHA  policy—less  allowances  for  elderly/disabled

households and household dependents and deductions for disability, medical, and childcare

expenses

II-8

August 31, 2016

● Ten percent of a household’s gross monthly income, with no allowances or expense

deductions

● The welfare rent in as-paid states (New York was the only as-paid state in this study)

● The minimum rent ($25 for Owner-administered projects or an amount established by the

PHA, not to exceed $50)

The formula for determining the TTP for the Sections 202 PRAC, 811 PRAC, and 202/16

PAC programs includes the first three items above, but there is no minimum rent requirement

for those programs.

MTW programs have the flexibility of modifying their TTP calculation process from the standard

formulas if the modification was established in their ACOP or Administrative Plan. In order to

ensure that the MTW projects were not found in error if modifications to rent calculation processes

had been approved, ICF reviewed the ACOPs and Administrative Plans for all MTW projects.

Based on the review, modifications to the standard TTP calculations were implemented for specific

projects. Some common modifications used by MTW projects were:

● Using 28 percent, or some other set percentage, of a household’s adjusted monthly income

to calculate TTP, instead of 30 percent

● Not deducting dependent or elderly/disability allowances from total annual income

● Using rent schedules for households within certain income bands

● Not counting income from assets if total assets were less than $50,000, or allowing for

self-certification of assets when assets totaled less than $50,000

● Using a tiered schedule to determine the amount of childcare, medical, or disability expense

deductions

● Using non-standard or alternate disability, medical, or dependent allowances

Five different rent calculations were used to calculate the actual amount of the household’s rent,

depending on program type:

● Public Housing (MTW and non-MTW)

● Section 8 Project-based (including Moderate Rehabilitation) and Sections 202 PRAC,

811 PRAC, and 202/162 PAC

● Section 8 Vouchers (MTW and non-MTW)

● Section 8 Enhanced Vouchers (there were 18 Enhanced Voucher households in the study)

● Manufactured Home Space Rental for Section 8 Vouchers (there were no households in the

FY 2015 study sample that met this criterion)

The household rent was calculated after data from all sources were collected. When calculating rent,

a cap was placed on the maximum amount of rent the tenant was required to pay. For all Section 8

programs, this is the lesser of the gross rent or the payment standard; in the Public Housing program,

this is the flat rent. If the flat rent was not available, the Ceiling Rent was used to cap the rent. The

rent was not capped for the Sections 202 PRAC or 811 PRAC programs.

HUDQC Final Report for FY 2015

II-9

Additional rent calculations were necessary for households with ineligible noncitizens. Determining

the correct rent for these households was a multistep process that first determined whether the

household is entitled to continued assistance or a temporary deferral of termination of assistance,

and then prorated the rent, if appropriate. Two proration formulas were used, one for Public Housing

and one for all Section 8 programs.

The algorithms for the rent calculation formulas can be found in  Appendix A.

G. Calculation of Rent Error

The monthly rent algorithms used by ICF to calculate the national estimates of error are:

● Actual Rent—The Actual (AC) Rent is the monthly rent indicated on Form

HUD50058/50059. If this item was missing on Form HUD-50058/50059, the AC Rent

was taken from another official document in the file.15

● Quality Control Rent—The Quality Control (QC) Rent is the monthly rent calculated by

ICF using all verified household information.16

Rent error was calculated by subtracting the QC Rent from the AC Rent. A discrepancy of $5 or

less between the monthly AC and QC rents was not considered an error. The $5 increment was

used to allow for minor calculation and rounding errors and to focus the data analysis on major

sources of error.

H. Quality Control Rent

ICF calculated QC Rents using the best available information. Every effort was made to use data

that would have been available to the PHA/project when determining which data to use in the QC

Rent calculation. Each income and expense item was processed individually. For each item, ICF

first used available verification from the household files. If acceptable verification was not available

from the household file, verification was requested during the household interview. If verification

was not available during the household interview, verification was requested from an appropriate

third party (see Section II-D for a discussion of acceptable verification). If verification was not

returned by a third-party entity, data from certain documents in the household file were used, even

if those documents did not meet the verification criteria. The only documents used when acceptable

verification was not available were verification documents from third-party entities whose date fell

outside the acceptable date range (when documents were present with other verification documents

in the file for a particular transaction) and tenant self-certification documentation collected during

15   Rent Roll data were not used as a substitute for AC Rent because a previous study found that the Rent

Roll sometimes included amounts to make up for previously unpaid rent, fines, or damages.

16  Attempts were made to verify items that were not verified by PHA/Owner staff; however, verification

was not always obtained. If verification was not available, other information from the household file or

documentation obtained during the household interview that met study requirements was used to calculate

the QC Rent.

II-10

August 31, 2016

the  household’s  recertification  process.  The  following  special  procedures  were  followed  when calculating the QC Rent, as appropriate:

● Income that started after the QCM was not counted when calculating the QC Rent.

● Income that ended after the QCM was counted for the full year unless it was clear that the

PHA/Owner knew that this income was going to end.

● Earned income bonuses were not counted unless it was clear that the bonus was paid on a

regular basis.

● Temporary Assistance for Needy Families (TANF) and other welfare income were treated as

the same source of income so that income listed as TANF on one form (e.g., the household

questionnaire) and “Other Welfare” on another form (e.g., the documentation forms) would

not be counted twice.

● Welfare income (TANF and other welfare), child support income, and childcare expenses

were treated at the household level instead of the household member level so that the same

source of income assigned to various household members would not be counted twice. For

example, if one household member (e.g., the head of household) was assigned a source of

income on one document and the same income was assigned to another household member

(e.g., a child) on another form, the income would not be counted twice because it was

assigned at the household level.

● Disability status was assigned to a household member based on Enterprise Income

Verification (EIV) documentation if two items were evident on the EIV printout: (1) receipt

of Social Security or SSI benefits and (2) a disability status of “yes.”

● Passbook rates (for determining the imputed income from assets) for PHA-administered

programs were taken from the project-level data provided by PHA/Owner staff. The

passbook rate for Owner-administered programs was 2 percent prior to February 2015, and

0.06 percent for those certifications and recertifications starting in February 2015.

● For new certifications, the low and very low income limits were obtained from HUD’s

website.

● When determining the prorated rent for Public Housing households with ineligible

noncitizens, if the maximum rent was not present on Form HUD-50058, the Fair Market

Rent was used instead of the 95th percentile of gross rent because the 95th percentile of

gross rent was not available.

● The values from Form HUD-50058 were used for minimum rent, gross rent, payment

standard, and flat rent unless the value was missing, in which case the missing value was

taken from the PHA/project-level data collection information provided by PHA staff.

● The values from Form HUD-50059 were used for gross rent and contract rent unless the

value was missing, in which case the missing value was taken from the project-level data

collection information provided by owner staff.

● Welfare rent for the State of New York was taken from the project-level data collection

information provided by PHA staff.

HUDQC Final Report for FY 2015

II-11

● A separate verification code was used to identify verification obtained from the EIV system.

When Social Security, SSI, or Black Lung benefits were verified by EIV, the information

was  considered  third-party  in-writing  verification.  If  EIV  information  was  in  the  file

for  earned  income  or  unemployment  benefits,  the  dates  associated  with  the  form  were

examined to determine whether PHA/project staff had access to the EIV information at

the time of recertification. Copies of EIV reports (as well as other types of verification of

earned income found in the household file) were sent to study headquarters and reviewed

by data quality specialists to prevent mistakes in calculating the QC earned income value.

Note: EIV was not considered an acceptable verification source for earned income.

● When working with Social Security and SSI benefit information obtained through the SSA

data match, discrepancies were sometimes found between those data and EIV printouts

contained  in  the  household  file.  If  the  two  sources  of  information  were  contradictory,

the  information  found  on  the  EIV  printout  (from  the  household  file)  was  used  in  the

QC calculation.

I. HUD Requirements Complicating the Analysis

Several HUD requirements affected the data collection methodology and subsequent analysis. As

noted in Section II-A, relevant HUD requirements were incorporated into the study standards used

to determine error. All data collection procedures and analyses were developed on the basis of

these standards. Although most standards were easily implemented, several were more problematic,

complicating the data collection process or the analysis. These complications are discussed below.

Anticipated Income. The amount of rent a household will pay is determined based on anticipated

household income and deductions for the 12 months following recertification. For households with

a stable income source, such as Social Security or steady employment, annual income estimates for

the next 12 months are relatively accurate. However, many assisted households have members with

sporadic employment or members who move in and out of the household. Also, certain expenses

(e.g., medical expenses for elderly/disabled households, child care costs) are difficult to anticipate.

Determining whether such income and expense amounts were calculated correctly at the time of

recertification is very difficult when data are collected after the changes occurred. Every effort was

made to treat questionable income or expenses in the manner they were treated by PHA/project

staff. Several of the special procedures described in Section II-H were created for this purpose.

Third-Party Verification Requests. HUD regulations require the information supplied by residents

at recertification to be verified by third parties (e.g., employers, SSA, banks, medical personnel).

Field interviewers obtained release forms from the household when evidence of verification was not

present in the tenant’s file; the release forms were used to request verification from the appropriate

third parties. However, some third parties did not respond, returned information for incorrect time

periods, required payment for the information requested, or presented other challenges that prevented

ICF staff from obtaining the correct verification. Follow-up requests for missing verifications were

not made in all cases because of time constraints.

ICF and HUD established a set of verification rules to determine whether an item was verified.

Section II-C identifies the rules used to determine whether verification was acceptable for each

matched item used in the rent calculation. Tables 1a to 1h (in  Appendix C) and Exhibit IV-19 in

Section IV-D present the verification rates for different rent components.

II-12

August 31, 2016

Earned Income Disregard (EID). The regulations governing the Public Housing and PHA-

administered Section 8 Voucher programs require PHAs to exclude a portion of earned income for

households meeting certain criteria. Only participants in these programs—not applicants entering

the programs—are eligible for this income exclusion.

To identify households eligible for EID, tenants were asked during the household interview about

training and self-sufficiency programs. Sixty household members were identified as possibly entitled

to an EID.

For these household members, we examined information on Form HUD-50058 and other household

file documentation. We compared the QC-calculated earned income exclusion (using the household

questionnaire information) with the earned income used by the PHA when calculating the total

annual income.

From the original 60 cases identified from household interview data, QC calculations determined

that 20 cases were possibly entitled to an EID. After investigating further for additional factors that

affect EID, 16 of the 20 cases were eligible and 4 were not eligible for EID. In 4 of the 16 eligible

cases, our QC calculation confirmed the PHA’s EID determination. In the remaining 12 cases, our

QC review determined that an EID was appropriate, but the PHA did not provide the household

with the income exclusion.

Training Programs. The regulations governing all housing programs included in this study require

the PHA/Owners to exclude all amounts received under training programs funded by HUD as well

as the incremental earnings and benefits resulting to any household member from participation in

qualifying State or local employment training programs.

To identify households eligible for the training program exclusion, field interviewers documented

training  program  information  found  in  the  household  file  and  provided  during  the  household

interview. This process identified members of 13 households with indications of involvement in

training programs, and 4 of those 13 households were found to be eligible for this income exclusion.

Permissible Deductions. Public Housing programs may adopt other deductions from annual income

in addition to HUD’s required deductions. To ensure that the appropriate additional permissible

deductions were taken into consideration when determining the adjusted annual income, two sources

of information were examined. First, ICF reviewed items 8b through 8e on Form HUD-50058,

which records the type and amount of permissible deductions. Second, a copy of local discretionary

policies from all PHAs was requested to identify additional exclusions adopted in their Public

Housing program. In the review of these documents, few unique permissible deductions were found

across Public Housing programs. The few special deductions that we found are attributed to unique

employment circumstances (i.e., union payments or individuals not utilizing the Mandatory Earned

Income Deduction), and household members making childcare and alimony expenses. In FY 2015,

this review found 5 households with permissible deductions using the local discretionary policies

at the 2 PHAs.

Moving to Work Exceptions. As mentioned in Section II-F, MTW programs have the flexibility

of  modifying  their  TTP  calculation  process  from  the  standard  formulas  if  the  modification

was  established  in  their  ACOP  or  Administrative  Plan.  To  ensure  that  all  modifications  were

incorporated into the QC Rent calculation, policies on the various exceptions were extrapolated

HUDQC Final Report for FY 2015

II-13

from each project’s ACOP or Administrative Plan, and these policies were included in the

QC Rent calculation.

Flat Rent.  Households that elected to pay a flat rent rather than an income-based rent were included

in the study. For these households, there is no rent error; the QC Rent is the same as the flat rent

used by the PHA. In FY 2015, there were 79 flat rent cases in the study sample. Determining

whether a household is paying a flat rent is not always easy because of contradictory data within

Form HUD-50058. In most cases, items 2a (i.e., Flat Rent Annual Update) and 10u (i.e., Type of

Rent Selected) could be used to determine whether the household is paying a flat rent instead of an

income-based rent. However, if these two items contradicted one another, information from other

documents in the file was used to determine the type of rent a selected tenant paid.

Ineligible Noncitizens. HUD regulations require that rents be prorated for households with ineligible

noncitizens unless the household meets certain criteria that allow the continuation of full assistance.

ICF reviewed all households with ineligible noncitizens to ensure that the rent was calculated

correctly. Twelve households (less than 1 percent of households in this study) included an ineligible

noncitizen and had a prorated rent amount per HUD regulations.

Reduced  or  Terminated  TANF  Benefits.  The regulations governing Public Housing and

PHA-administered Section 8 programs included in the study require using the amount of the TANF

benefit before reduction or termination when such changes to TANF benefits resulted from fraud or

failure to cooperate with the welfare family self-sufficiency program. To identify households with

reduced or terminated TANF benefits, tenants were asked during the household interview about

previous receipt of TANF and whether their TANF benefits were reduced.

If the TANF benefits were reduced or terminated because of fraud or failure to comply with the

welfare family self-sufficiency requirements, the value of the TANF benefit before the reduction or

termination was used in the QC Rent calculation.17 TANF benefits in 18 households were reviewed,

and we identified no households for which TANF amounts should have been imputed.

Students. The regulations governing the PHA-administered Section 8 and Owner-administered

programs included in the study require that students ages 17–24 meet certain criteria. If these criteria

are not met, the parent’s income must be included when determining whether the student meets

the program’s financial requirements. For households with students, field interviewers documented

student enrollment and member characteristics found in the household file or provided during the

household interview. These households were then reviewed to determine whether the student met

the special student criteria as defined by HUD regulations. Nine cases were reviewed, and all cases

were determined to be correctly receiving housing assistance.

17  The value of the reduced or terminated TANF benefit is offset by the amount of additional income that

the family received starting after the sanction was imposed.

II-14

August 31, 2016

III. Study Objectives and Analytic Methods

This section identifies the 16 HUDQC Study objectives and a brief description of the methodology

that was used to fulfill these objectives.18 A summary of these objectives and the location of this

information in the report is presented at the end of this section, in Exhibit III-2.

Objective 1:  Identify the various types of rent errors and rent error rates and related

estimation variances.

The FY 2015 study identified the error types and error rates that were previously analyzed and

presented in the FY 2000 through FY 2014 studies. Using verified information as determined in

the HUDQC Study standards, the tenant rent (QC Rent) was recalculated and subtracted from the

tenant rent indicated on the Form HUD-50058/50059 (Actual Rent, or AC Rent) to determine error.

Rent error was categorized and described in a number of ways, including providing the percentage

of households paying correct and incorrect rent, total dollar error amounts, and dollar error rates.

Variance estimates (standard errors) were provided for selected error rates. Errors were determined

by recalculating the tenant rent on the basis of verified QC information and subtracting this amount

from the tenant rent indicated on the Form HUD-50058/50059 (AC Rent). The following three types

of dollar rent error estimates were calculated:

● Dollar Rent Error—The Dollar Rent Error is the difference between the monthly Actual

Rent (AC Rent) and the monthly QC Rent (i.e., AC Rent minus QC Rent). A household

rent was found to be in error if the difference between the AC Rent and QC Rent was

greater than $5, while proper rent payments reflected differences of $5 or less. Rates of

exactly matching AC and QC rents (within $1) were also presented. Simple percentages

of the number of households paying the proper and exact rents were reported, as well as

the percentage of households in error per program, the average gross dollars in error, and

the percentage of rent dollars in error. For households that were ineligible when initially

certified, the QC Rent was the flat rent for Public Housing households, or the Housing

Assistance Payment (HAP) for Section 8 programs. The dollar error in these cases was also

defined as the QC Rent amount minus the AC Rent.

● Total Component Dollars in Error—The Total Component Dollars in Error is the absolute

sum (i.e., the sum of the positive and negative amounts, ignoring the plus or minus signs) of

all individual income and expense component errors. These errors were combined to provide

an overall Total Dollars in Error and were presented as annual amounts.19 Each component

of identified error contributed to a dollar amount of rent overpayment or underpayment;

however, some of these errors overlapped or were offset. For example, earned income

may have been underreported because of a calculation error or the Supplemental Security

18   For a more detailed description of the methodology, see the  Analysis Plan for the FY 2015 HUDQC

Study,  an unpublished ICF report to HUD dated October 2, 2015.

19   Because dollar component errors (CE) are reported on an annual basis while dollar rent errors (RE)

are reported on a monthly basis, and rents are generally set at 30 percent of adjusted income, component

errors are usually 40 times the corresponding rent error: 0.30 * CE = 12 * RE, or CE = (12/0.30) * RE =

(120/3) x RE = 40 * RE.

HUDQC Final Report for FY 2015

III-1

III. Study Objectives and Analytic Methods

Income (SSI) may have been overstated. Thus, the net difference could be zero or a positive

or negative amount.

● Largest Component Dollar Error—The Largest Component Dollar Error is the annual

dollar amount of error for the income or expense components with the largest error in a case.

Income and expense components included the five sources of income (i.e., earned income,

pension,  public  assistance,  other  income,  and  assets)  and  the  five  types  of  deductions

(i.e., medical, childcare, and disability assistance expenses; and dependent allowance and

elderly/disabled allowance). If, for example, the component with the largest error was earned

income, the largest component dollar error was the difference between the earned income

identified by the PHA/project and the earned income identified in the QC Rent calculation.

The dollar error rate was used for other error calculations, including the national Rent Error Rate and

Net and Gross Error Rates. The latter error calculations link errors in the rent determination process

to dollar error rates. This information is used to support policies that promote better oversight and

administrative practices of HUD rental subsidies.

Objective 2:  Identify the dollar costs of the various types of errors.

Five types of administrative errors are linked to rent errors. Data obtained directly from the Form

HUD-50058/50059 as well as PHA/project and tenant information from the tenant file were used

to identify and measure each of the following error types:

●  Calculation errors

●  Consistency errors

●  Transcription errors

●  Incorrect determination of allowances and income sources

●  Overdue recertifications

Calculation errors were detected by recalculating section subtotals and the final rent based on the

exact information on Form HUD-50058/50059. The tenant rent was calculated using the detailed

information on Form HUD-50058/50059, and the amount was compared to the actual tenant rent

on Form HUD50058/50059. A difference in these two rents was identified as a calculation error.

Consistency   errors  identified  a  lack  of  logical  conformity  between  elements  within  Form

HUD-50058/50059. For example, the Effective Date of Action had to be on or after the Date of

Admission; elderly status information had to be consistent with information about the age of the

head of household or spouse.

Transcription  errors are errors that were identified by comparing Form HUD-50058/50059 data

with information in the tenant file. If the amount of a specific income or expense item on the Form

HUD-50058/50059 did not match the tenant file data, a transcription error existed.

Incorrect  determinations  of allowances and income sources were also detected using tenant file information and comparing it to the Form HUD-50058/50059 data. Allowance errors were detected

by calculating the allowances based on the tenant file information and comparing this QC allowance

amount to the actual allowance on Form HUD-50058/50059. Similarly, income was calculated

III-2

August 31, 2016

III. Study Objectives and Analytic Methods

based on the types and amounts of income identified in the tenant file. The improper application of

allowances and the incorrect calculation of income were a subset of transcription errors.

Overdue   recertifications often produced rent errors because rents were based on out-of-date

information. For households with overdue recertifications, the QC information was based on the

month the recertification should have been completed rather than the month that the recertification

was completed.

Objective 3:  Estimate national-level net costs for total errors and major error types.

This analysis included determining the national Rent Error Rate, the numbers and proportions of

households found to be in error, the dollar amount of rent error, and the proportion of total dollars

found to be in error. Sample data were weighted to provide national estimates.

Objective 4:  Determine the relationship between errors detectable using Form HUD-

50058/50059 and total errors found in the study.

As discussed under Objective 2, calculation and consistency errors identified mistakes made by

the housing PHA/project staff. Under Objective 4, households with calculation and consistency

errors were compared to households with QC errors to determine whether errors found within Form

HUD-50058/50059 could be used to predict QC errors.

Objective 5:  Determine whether  error rates  and  error costs  have  statistically significant

differences from program to program.

This analysis presented differences in error rates by program type. Data were provided for three

program groups: Public Housing, PHA-administered Section 8 (Section 8 Vouchers and Moderate

Rehabilitation programs), and Owner-administered programs (Section 8, Section 202 PRAC, Section

811 PRAC, and Section 202/162 PAC). The Gross and Net Error Rates were provided for each

of these program types. The Gross Error Rate was the sum dollar amount of gross error divided

by the sum dollar amount of QC Rent, and the Net Error Rate was the sum dollar amount of net

error divided again by the sum dollar amount of QC Rent. Multivariate analyses were performed

to determine whether differences from program to program are statistically significant.

Objective 6:  Determine the apparent cause of significant rent errors, either on a sample or

a comprehensive basis, to provide HUD with information on whether the error

was caused primarily by the tenant or by program sponsor staff.

As in the previous studies, ICF provided descriptive information on the sources of discrepancies

between  housing  file  information  and  verified  information,  and  described  the  incidence  of

administrative errors and their impacts. We also examined whether failure to verify sources of

income and expenses contributed to QC error. Multivariate analyses using administrative errors and

income components as independent variables were performed to identify how these errors affected

the QC Dollar Rent Error.

HUDQC Final Report for FY 2015

III-3

Objective 7:  Determine the extent to which households are over-housed relative to HUD’s

occupancy standards.

This objective addressed whether households resided in units with the correct number of bedrooms.

Exhibit III-1 shows HUD’s generally accepted guidelines specifying the appropriate size unit for

assisted households.20

For most programs, unit-size rules are not based solely on household size but allow for discretion

on the part of PHA/project staff. All programs allow exceptions to these rules. This study replicated

the analyses in the previous studies that identified bedroom size and program, and the proportion of

households in compliance with and in violation of occupancy standards according to the guidelines

outlined in Exhibit III-1.

Exhibit III-1

PHA-administered Section 8 Unit Size Standards

Number of Persons in Household

Number of Bedrooms

Minimum

Maximum

0

1

1

1

1

2

2

2

4

3

3

6

4

5

8

5

5

10

Objective 8:  Provide information on the extent to which errors are concentrated in projects

and programs.

ICF conducts further descriptive analyses to examine whether errors are concentrated within or

randomly distributed across PHAs/projects. Multivariate analyses are conducted with the tenant as

the unit of analysis. Tenant and PHA/project characteristics are analyzed as independent variables

predicting error rates. This analysis identified how each of these variables contributes to rent error.

The results will help guide HUD’s management of error rates and help HUD evaluate relationships

between management practices and project/tenant characteristics that affect error rates.

Objective 9:  Identify  the  percentage  of  newly  certified  tenants  who  were  incorrectly

determined eligible for program admission.

Incorrect initial eligibility determinations create long-term problems for assisted-housing programs.

Newly certified households were reviewed to determine whether they met the eligibility requirements

for assisted housing.

Five  eligibility  requirements  reviewed  at  initial/move-in  certification  are  not  a  part  of  the

recertification process (and thus are not confirmed on an ongoing basis): the definition of family,

citizenship status, the verification of Social Security numbers, signed tenant consent forms, and

20   Housing projects have discretion in determining unit size and may determine unit size differently than

shown.

III-4

August 31, 2016

low to very low income limits. This study did not investigate the definition of family because it

is determined by the PHA or owner. Therefore, findings were provided on four of the five initial

certification criteria. This study also did not include suitability factors that PHA/owners may use

in selecting tenants—factors such as tenant histories and history of drug use or criminal activity.

Objective 10: Determine the extent to which Section 8 Voucher rent comparability

determinations are found in the tenant file and indicate the method used to

support the determination. Determine whether voucher payment standards

are within 90 to 110 percent of fair market rents, and determine whether the

correct utility allowances are being used in Section 8 Voucher households.

To comply with the rent reasonableness requirement, housing authorities must determine that Section

8 Voucher rents are reasonable in comparison with rents for similar housing in the private, unassisted

market. Using information collected from tenant files, we estimated the proportion of Section 8

Voucher recipients with rent comparability documentation. For those with documentation, we

classified the type of evidence cited in the tenant file documentation (e.g., no evidence, cited market

estimates for comparable units, or the rents of one or more units considered to be comparable). We

presented weighted proportions of voucher recipients with rent comparability data.

Additionally, payment standard data from Form HUD-50058 were compared with Fair Market Rent

(FMR) data to identify the households whose payment standards fall outside the 90 to 110 percent

FMR band. Utility allowance schedules were likewise matched to tenant files to evaluate the issues

associated with independently evaluating utility allowances as a potential component of rent error.

Objective 11: Estimate total positive and negative errors in terms of HUD subsidies.

Proper payments are those in which the AC Rent equals the QC Rent. Errors can be either tenant

overpayments (i.e., AC Rent is greater than QC Rent) or tenant underpayments (i.e., AC Rent

is less than QC Rent). Overpayment error rates were calculated by dividing the total amount of

overpayment by the total QC Rent; underpayment error rates were calculated similarly by dividing

the total amount of underpayments by the total QC Rent.

Objective 12: Determine the extent to which error rates in projects that use an automated

rent calculation system differ from errors in those that do not.

We investigated the relationship between using an automated rent calculation system and

project-level gross error rate by using an analysis of variance. We also examined whether Gross

Rent Error differed significantly by computer use between programs.

Objective 13: Determine whether other tenant or project characteristics on which data are

available are correlated with higher or lower error rates.

To respond to this objective, we used multivariate analysis to conduct more detailed analyses

of differences among PHAs/projects and to provide HUD with more information for identifying

projects and tenants likely to exhibit high error rates.

HUDQC Final Report for FY 2015

III-5

Objective 14: Determine whether cases for which Form HUD-50058/50059 data had been

submitted to HUD were more or less likely to have errors than those for which

data had not been submitted.

The QC sample was matched to the Public and Indian Housing (PIH) Information Center/Tenant

Rental Assistance Certification System (PIC/TRACS) data. Analysis was conducted to compare the

average dollars in error for households included in PIC/TRACS with those that were not included.

Objective 15: Determine the extent of program administrator rent and income determination

errors.

This objective is essentially a summary of Objectives 1–3. The percentage of households in error

and the dollars associated with those households will be determined analytically and reported

accordingly.

Objective 16:  Determine the error rate for the 20 Largest PHAs.

For the 20 Largest PHAs sample, 32 households were selected per PHA. Since the desired error rate

can use the households sampled in the HUDQC Study, we interpreted HUD’s needs in terms of the

total sample needed from these PHAs. Thus, if a PHA already had 32 households selected for the

HUDQC Study, no further cases would be sampled. However, if fewer than 32 households were

selected in the HUDQC Study, additional tenant files would be selected to achieve 32 households

per PHA. Analysis for the 20 Largest PHAs will include tables regarding administrative error and

payment error.

ICF addressed administrative error and provided the percentage of households with overdue

recertification and transcription errors and the percentage of income and expense items verified

by  PHA  staff,  both  with  written  third-party  verification  only  and  with  verbal  or  written  third-

party  verification  or  documentation.  Totals  for  the  20  Largest  PHAs  were  compared  to  the

PHA-administered projects supplied for the HUDQC Study and the HUDQC Study as a whole.

Additionally, payment error information was provided. This includes proper payments, under- and

overpayments of tenant rents, and the mean gross rent errors by PHA. As with administrative error,

totals for the 20 Largest PHAs were compared to the PHA-administered projects supplied for the

HUDQC Study and the HUDQC Study as a whole.

Exhibit III-2 presents all study objectives and where they are addressed.

III-6

August 31, 2016

Exhibit III-2

Summary of Study Objectives

#

Objective

Where Objective Is

Addressed in Section IV

Identify the various types of rent errors and rent error rates and calculate their

variance estimates. These include:

Exhibit 1;

1

•  Dollar Rent Error

Exhibit 2;

•  Total Component Dollars in Error

Exhibits 3a & 3b

•  Largest Component Dollar Error

Identify the dollar costs of the various types of errors, including:

•  Calculation errors

Exhibit 6;

•  Consistency errors

Exhibit 8;

2

•  Transcription errors

Exhibit 16;

•  Incorrect determination of allowances and income sources

Exhibits 22a & 22b

•  Overdue recertifications

3

Estimate the national-level costs for total error and major error types.

Exhibit 4

Determine the relationship between errors detectable using the Form

Exhibit 15;

4

HUD-50058/50059 and total errors found in the study.

Exhibit 17

5

Determine whether error rates and error costs have statistically significant

differences from program to program.

Exhibits 4-5

Determine the apparent cause of significant rent errors to provide HUD with

6

information on whether the error was caused primarily by the tenant or by

Exhibits 11 – 14

program administrator staff.

7

Determine the extent to which households are over-housed relative to HUD’s

occupancy standards.

Exhibits 24a – 24c

8

Provide information on the extent to which errors are concentrated in projects

and programs.

Exhibits 18 – 21

9

Identify the percentage of newly certified tenants who were incorrectly

determined eligible for program admission.

Exhibits 7a – 7b

For Section 8 Voucher households, determine:

•  The extent to which rent comparability determinations are found in the

Exhibits 25 – 27;

10

tenant file, and indicate the method used to support the determination

Exhibits 29a – 29d;

•  Whether payment standards are within 90%–110% of fair market rents

Exhibits 28a – 28d

•  Whether the correct utility allowances are being used.

11 Estimate total positive and negative errors in terms of HUD subsidies.

Exhibits 9a – 10

Determine the extent to which error rates in projects that use an automated

12 rent calculation system differ from errors in those that do not use an automated

Appendix F

rent calculation system.

13 Determine whether other tenant or project characteristics on which data are

available are correlated with higher or lower error rates.

Appendix F

Determine whether cases for which Form HUD-50058/50059 data were

14 submitted to HUD were more or less likely to have errors than those for which

Exhibits 30a – 30e

data were not submitted.

15 Determine the extent of program administrator rent and income determination

errors.

Appendix E

16 Determine the error rate for the 20 Largest PHAs.

Exhibits 31a–31b

HUDQC Final Report for FY 2015

III-7

IV. Findings

A. Overview

Analyses were conducted using nationally weighted sample data for 2,400 households.21 Data are

presented by the three program types that were the basis for the sampling design: Public Housing;

Public Housing Authority (PHA)-administered Section 8 Housing Choice Voucher and Moderate

Rehabilitation programs (PHA-administered Section 8); and Office of Housing-administered Section

8, Section 202 Project Rental Assistance Contracts (PRAC), Section 811 PRAC, and Section

202/162 PAC programs (Owner-administered). The major study findings, the reasons for the errors,

and other background information concerning these errors are discussed as follows. In many of

the exhibits in this report, the data collected during the current study (referred to as the FY 2015

data) are compared with the data collected in the previous study (referred to as the FY 2014 data).

Our discussion is divided into 11 parts:

1.  The errors in the rent amount based on the QC data (rent errors)

2.  The errors in sources of income and expenses (component errors)

3.  The errors found using only project-file data (administrative errors)

4.   Occupancy standards

5.  Findings related to rent reasonableness determinations

6.  Utility allowance analysis

7.  Payment standard analysis

8.  Comparisons with PIH Information Center/Tenant Rental Assistance Certification System

(PIC/TRACS) data

9.  Analysis of the responses received from PHA/project staff regarding PHA/project practices

(based on the Project Staff Questionnaire)

10.  Multivariate analysis

11.  Errors in the 20 largest PHAs

The first three parts present different types of errors, described as follows.

Rent  errors are errors that result in an actual dollar error. A dollar error means that the household paid too much rent (an overpayment) or the household paid less rent than it should have paid (an

underpayment).

Component  errors are errors in the income and expense components used to calculate rent. The

income components are employment income, Social Security and pensions, public assistance, other

income, and asset income. The expense/allowance components are elderly/disabled allowance,

dependent allowance, medical expenses, child care expenses, and disability expenses.

21   Appendix B presents the procedure used in weighting the data.

HUDQC Final Report for FY 2015

IV-1

IV. Findings

Administrative  errors are errors that result from administrative mistakes. They include the following:

● Consistency errors, which are errors in logical conformity between elements within Form

HUD-50058/50059

● Calculation errors, which are arithmetic errors within subsections of Form HUD-50058/

50059

● Transcription errors, which are errors in transferring information from documentation in

the tenant file to Form HUD-50058/50059

● Failure to conduct a recertification in a timely manner

● Failure to verify information

Component and administrative errors may or may not result in rent errors. Administrative errors tell

us at what point during the rent determination process an error occurred, while component errors

tell us which income or expense caused the error. Data supporting this discussion are presented in

the source tables found in  Appendix C.

B. Rent Error

Overview. Rent errors were identified by subtracting the QC Rent from the Actual (AC) Rent.22 The

QC Rent was calculated using data obtained from one of several sources, including the tenant file,

the household interview, and third-party verification. If acceptable verification was present in the

tenant file, that information was used. If acceptable verification was not present in the tenant file,

other sources were used. The Actual Rent is the Tenant Rent from Form HUD-50058/50059. As

noted previously, a household was considered to be correct (proper payment) if the QC Rent and

the Actual Rent matched within $5. All exhibits included in this report (except Exhibit IV-1) and

all tables in  Appendix C (except the supplemental tables) define households in which AC Rents and

QC Rents matched within $5. As noted, this is not the case for the supplemental tables in  Appendix

C (designated by the letter “S”), which are based on exact matches between these two rents.

Definitions of Rent Errors. Dollar error can be determined by comparing the rent the household

should have paid with what it was paying or by identifying the percentage of the Federal subsidy that

was paid in error. In this study, error was determined by the first method. The rent errors presented

throughout this report were calculated in the following manner:

●  Dollar Rent Error was calculated at the household level by subtracting the household’s QC

Rent from the Actual Rent. Note that these are monthly rents. A negative number indicates

an underpayment, meaning that the household paid less than it should have paid and that

HUD’s contribution was higher than it should have been. A positive number indicates a

household overpayment, meaning that the household paid more than it should have paid

and that HUD’s contribution was less than it should have been.

22   Rent error is determined on the basis of Tenant Rent, not Total Tenant Payment (TTP). Error based on

TTP may differ from error based on Tenant Rent because of the program-specific rent formulas applied

when calculating Tenant Rent. These rent formulas are listed in Section II-F and presented in detail in

Appendix A.

IV-2

August 31, 2016

IV. Findings

●  Gross Rent Error is the absolute value (i.e., the sum of the absolute value of positive and

negative Rent Error) of the Dollar Rent Error for the sample as a whole or a specified group

of households. The Gross Rent Error functions simply as a measure of the magnitude of

the errors. The dollar amounts presented in the tables are Gross Rent Error values, unless

otherwise indicated.

●  Net Rent Error is the arithmetic value (i.e., the sum of the negative and positive values of

underpayments and overpayments) of the Rent Error.

●  Error Rate  is calculated by dividing the sum of the Rent Error (gross or net) by the sum of

the QC Rent for the entire sample or a specified group of households.

Financial Impact of Identifying Rent Error. Reduction in the rent error associated with the

programs included in this study does not mean there will be an overall savings in the costs associated

with administering these programs. Given that there are large numbers of eligible households

on waiting lists, if a household leaves the program because it is no longer eligible for a subsidy,

another household will take its place, and the replacement household may be entitled to a smaller

or a larger subsidy than the household that left the program. Therefore, the most direct benefit

of identifying households with rent error is ensuring that the households that are eligible for the

program are receiving the correct subsidy, rather than reducing the funds needed to administer the

programs. The most appropriate use of this study is as a tool for strengthening HUD’s procedures

for ensuring administrative compliance with regulations. The recommendations presented in this

report may require greater rather than fewer resources to provide HUD, PHAs, and Owners with

the written policy guidelines and training, standardized forms, and ongoing monitoring needed to

ensure that the programs are administered correctly.

Proper Payments.  Exhibit IV-1 shows the percentage of households with proper payments by

program, for households in which the AC Rent and QC Rent matched within $5 and in which the

AC Rent and QC Rent matched exactly.

● At certification/recertification, the rent was calculated correctly (within $5) in 73 percent of

households, a slight drop from the 75 percent of households for which rent was calculated

correctly in FY 2014.

● There was an exact match of rent payment in 61 percent of households in FY 2015, which

was a decrease from the 65 percent of the last two study years.

Exhibit IV-1

Percentage of Households With Proper Payments

Percentage of Households  Standard  Percentage of Households  Standard

Within $5

Error

That Matched Exactly

Error

FY

FY

FY

FY

FY

FY

FY

FY

Administration Type

2013

2014

2015

2015

2013

2014

2015

2015

Public Housing

77%

73%

75%

2.4%

67%

64%

63%

2.3%

PHA-administered Section 8

77%

74%

71%

1.6%

62%

63%

58%

1.8%

Total PHA-administered

77%

74%

72%

1.5%

64%

63%

60%

1.5%

Owner-administered

80%

80%

77%

1.7%

68%

69%

66%

2.8%

Total

78%

75%

73%

1.3%

65%

65%

61%

1.5%

Source: Tables 2 and 2(S), Appendix C

HUDQC Final Report for FY 2015

IV-3

Households With QC Rent Error.  Exhibit IV-2 shows the percentage of households in error,

average dollar amount in error, and error rate by program.

● Twenty-six percent of households had a rent error greater than $5, which is higher than the

25 percent recorded in FY 2014.

The average gross dollars in error is calculated by dividing the sum of the dollar amount of gross

error (i.e., the sum of the absolute values of underpayments and overpayments) by the total number

of households.

● The average gross dollars in error was $13 in FY 2015, lower than the $14 average gross

dollar error in FY 2014.

The total gross dollar error rate was calculated by dividing the sum of the dollar amount of Gross

Rent Error by the sum of the dollar amount of the QC Rent.

● The total gross dollar error rate decreased by 1 percent, from 6 percent in FY 2014 to

5 percent in FY 2015.

Exhibit IV-2

Percentage of Households With Error, Average Dollars in Error,

and Dollar Error Rate for All Households With Error

Percentage of

Average Gross Dollars

Households With Error

in Error

Gross Dollar Error Rate

Administration Type

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Public Housing

27%

25%

$18

$14

7%

5%

PHA-administered Section 8

26%

29%

$15

$15

7%

6%

Total PHA-administered

26%

28%

$16

$15

7%

5%

Owner-administered

20%

23%

$8

$10

4%

4%

Total

25%

26%

$14

$13

6%

5%

Source: Tables 2 and 5, Appendix C

Underpayment and Overpayment Households. Exhibits IV-3a and IV-3b show the percentage

of households and average dollar amount of error for all households when errors of $5 or less are

excluded from the calculations; these exhibits present the error for underpayment and overpayment

households, respectively.

● Fourteen percent of all households paid in excess of $5  less than they should have in

FY 2015, which was the same percentage found in FY 2014.

● For FY 2014 households, the average monthly underpayment error was $66, which is an

increase from the average of $59 in FY 2014 and $52 in FY 2013.

IV-4

August 31, 2016

Exhibit IV-3a

Underpayment Households: Percentage of Households

and Average Monthly Dollar Amount of Error

Average Dollar Amount of Error

For Underpayment

Percentage of

Households

Households in Error

(with errors > $5)

For All Households

FY

FY

FY

FY

FY

FY

FY

FY

FY

Administration Type

2013

2014

2015

2013

2014

2015

2013

2014

2015

Public Housing

11%

14%

13%

$70

$63

$59

$7

$9

$8

PHA-administered Section 8

13%

14%

15%

$53

$63

$73

$7

$9

$11

Total PHA-administered

12%

14%

14%

$58

$63

$69

$7

$9

$10

Owner-administered

10%

12%

12%

$36

$46

$57

$4

$6

$7

Total

12%

14%

14%

$52

$59

$66

$6

$8

$9

Source: Tables 2 and 4, Appendix C

● Thirteen percent of all households paid in excess of $5 more than they should have in

FY 2015, which is greater than the 11 percent found in FY 2014 and the 10 percent found

in FY 2013.

● The average monthly overpayment for households with overpayment error was $35 in

FY 2015, a marked decrease from the past 2 years. This value was $51 in FY 2014 and $44

in FY 2013.

Exhibit IV-3b

Overpayment Households: Percentage of Households

and Average Monthly Dollar Amount of Error

Average Dollar Amount of Error

For Overpayment

Percentage of

Households

Households in Error

(with errors > $5)

For All Households

FY

FY

FY

FY

FY

FY

FY

FY

FY

Administration Type

2013

2014

2015

2013

2014

2015

2013

2014

2015

Public Housing

13%

13%

12%

$42

$70

$57

$5

$9

$7

PHA-administered Section 8

10%

11%

15%

$53

$51

$29

$5

$6

$4

Total PHA-administered

11%

12%

14%

$49

$58

$37

$5

$7

$5

Owner-administered

10%

8%

11%

$29

$27

$28

$3

$2

$3

Total

10%

11%

13%

$44

$51

$35

$5

$5

$4

Source: Tables 2, 3, and 4, Appendix C

Figure IV-1 shows the percentage of underpayments, proper payments, and overpayments by

program type. Programs were grouped into three categories: Public Housing, PHA-administered

Section 8, and Owner-administered. For all program types, the majority of cases fall into the

proper-payment category.

HUDQC Final Report for FY 2015

IV-5

Figure IV-1

Payment by Program Type

Gross and Net Dollars in Error. Exhibit IV-4 presents the gross and net average dollars in error

and their associated standard error. To obtain the Gross Rent Error and the Net Rent Error, the

dollar amount of overpayments was added to the dollar amount of underpayments, first using the

absolute values for gross error and then the arithmetic values for the net error.

● Gross average dollar error in FY 2015 decreased from FY 2014 in Public Housing ($18 to

$14, respectively), remained steady in PHA-administered Section 8 at $15, and increased

in the Owner-administered program from $8 in FY 2014 to $10 in FY 2015.

● In FY 2015, there were no statistically significant results between either Gross Rent Error

or Net Rent Error for any program types across the current and previous year’s results.

● The net error measures the dollar cost of the errors and was -$5 (indicating a tenant

underpayment) for FY 2015; the average gross dollar error was $13 for FY 2015 and

represents the dollars associated with the errors (the magnitude of the errors).

Exhibit IV-4

Gross and Net Dollar Rent Error (Monthly) for All Households

Gross Rent Error

Net Rent Error

Average Dollars

Average Dollars

in Error

Standard Error

in Error

Standard Error

FY

Administration Type

2014

FY 2015 FY 2014 FY 2015 FY 2014 FY 2015 FY 2014 FY 2015

Public Housing

$18

$14

$2.09

$2.70

-$0.33

-$1

$2.44

$2.69

PHA-administered Section 8

$15

$15

$2.24

$1.95

-$3

-$7

$1.35

$1.91

Total PHA-administered

$16

$15

$1.77

$1.88

-$2

-$5

$1.22

$1.65

Owner-administered

$8

$10

$1.12

$1.33

$4

-$4

$1.26

$1.39

Total

$14

$13

$1.42

$1.29

-$3

-$4

$0.97

$1.17

Source: Table 5, Appendix C

IV-6

August 31, 2016

Error Rates by Program. Differences in error rates by program type were investigated, and the results are summarized in Exhibit IV-5.

Differences include the Gross Error Rate (i.e., the sum dollar amount of gross error divided by the

sum dollar amount of QC Rent) and the Net Error Rate (i.e., the sum dollar amount of net error

divided again by the sum dollar amount of QC Rent).

● The Gross Error Rate of 5.6 percent for PHA-administered Section 8 programs remained

higher than the rate for either Public Housing or Owner-administered programs. In total,

PHA-administered programs showed a slight decrease of 1.2 percent in their Gross Error

Rate in FY 2015 compared to FY 2014.

● The Net Error Rates for all programs increased slightly, from -1.1 percent in FY 2014 to

-1.7 percent in FY 2015.

Exhibit IV-5

Gross and Net Dollar Error Rates (Monthly) for All Households

Error Rates

Gross Error Rate

Net Error Rate

Administration Type

FY 2014

FY 2015

FY 2014

FY 2015

Public Housing

6.6%

5.2%

-0.1%

-0.4%

PHA-administered Section 8

6.6%

5.6%

-1.4%

-2.4%

Total PHA-administered

6.6%

5.4%

-0.9%

-1.8%

Owner-administered

3.6%

3.9%

-1.7%

-1.5%

Total

5.8%

5.0%

-1.1%

-1.7%

Source: Table 5, Appendix C

Certifications/Recertifications. The sampled households included both certifications (i.e., newly

admitted  households)  and  recertifications.  Certifications  were  analyzed  to  determine  whether

the households were eligible for HUD housing assistance, and recertifications were analyzed to

determine whether they were overdue.

Figure IV-2 presents the breakdown of cases by case type: timely certifications, recertifications,

and overdue recertifications.

HUDQC Final Report for FY 2015

IV-7

Figure IV-2

Percentage of Cases by Case Type

Source: Table 6, Appendix C

Exhibit  IV-6  shows  the  percentage  of  certifications,  timely  recertifications,  and  overdue

recertifications, by program type.

● Eight-five percent of households had timely recertifications in FY 2015, a decrease for the

third year in a row. In FY 2014, 88 percent of households received timely recertifications

and in FY 2013, the rate was 90 percent.

● Two percent of households had overdue recertifications for the second year in a row.

● There  was  a  slight  increase  in  the  total  percentage  of  certifications,  from  11  percent  in

FY 2014 to 13 percent in FY 2015.

Exhibit IV-6

Certifications and Recertifications by Administration Type*

Overdue

Certifications

Timely Recertifications

Recertifications

Administration Type

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Public Housing

11%

13%

87%

84%

2%

3%

PHA-administered Section 8

9%

12%

89%

86%

2%

2%

Total PHA-administered

10%

12%

88%

86%

2%

2%

Owner-administered

14%

14%

85%

86%

< 1%

< 1%

Total

11%

13%

88%

85%

2%

2%

Source: Table 6, Appendix C

*Totals may not add up 100 percent to due to rounding.

IV-8

August 31, 2016

Certifications.  Exhibit IV-7a presents a summary of the findings related to eligibility criteria, and Exhibit IV-7b shows the percentage of newly certified households meeting the certification criteria

by program type.

The reviewed criteria included qualifying as low-income or very-low-income households,

citizenship, social security number, and signing the appropriate consent form.

● However, only those households that do not meet the appropriate low-income or

very-low-income limit are ineligible for assistance. All households (according to the QC

Rent calculation) fell within the low-income limit for total gross income.

A household met the citizenship criteria if there was evidence in the tenant file that citizenship had

been verified.

● A citizenship code (indicating whether each household member was a citizen, eligible

noncitizen,  or  ineligible  noncitizen)  was  available  from  either  the  tenant  file  or  the

household interview for each household member.

● According to the citizenship codes, no households in FY 2015 had a household member for

whom there was no verification of citizenship. This is unchanged since FY 2010.

To meet the citizenship verification requirement, the file must have contained one of the following

for each household member: a signed declaration of U.S. citizenship or eligible immigration

status, proof of age documentation, a U.S. Citizenship and Immigration Services (USCIS) card or

USCIS system verification of citizenship status, or documentation that the household member was

undergoing verification or an Immigration and Naturalization Service hearing.

To  meet  the  social  security  number  verification  requirement,  the  file  must  have  contained  for

each household member a copy of the social security card or statement from the Social Security

Administration (SSA) verifying the social security number.

● One percent of households had at least one member for whom there was no verification of

his or her social security number.

● In 97 percent of households, there was a signed consent form dated within 15 months of the

Quality Control Month (the date for which data were collected) for all members age 18 or

older, the same percentage found in FY 2014.

● A social security number was available for 99 percent of all Public Housing households, 100

percent of PHA-administered Section 8 households, and 97 percent of Owner-administered

households.

Note: Not meeting the citizenship, social security number, or consent form criteria may not mean

that the household was not eligible for assistance; rather, it may mean that project staff did not

follow HUD requirements in documenting the information.

HUDQC Final Report for FY 2015

IV-9

Exhibit IV-7a

Percentage of Newly Certified Households Meeting Certification Criteria

Met Criteria

Certification Criteria

FY 2014

FY 2015

Citizenship

100%

100%

Social Security Number

99%

99%

Consent Form

97%

97%

Low and Very Low Income

100%

100%

Meets All Eligibility Criteria

96%

96%

Source: Table 7, Appendix C

Exhibit IV-7b

Percentage of Newly Certified Households Meeting Certification Criteria by Program Type

Percentage of Households Meeting the Criteria

PHA-administered

Owner-

Certification Criteria

Public Housing

Section 8

administered

Citizenship

100%

100%

100%

Social Security Number

99%

100%

97%

Consent Form

93%

97%

99%

Low and Very Low Income

100%

100%

100%

Meets All Eligibility Criteria

93%

97%

97%

Source: Table 7b, Appendix C

Underpayments and Overpayments for Certifications, Timely Recertifications, and Overdue

Recertifications.  Exhibit IV-8 presents a summary of the households with overpayments and

underpayments by the type of case: certification, timely recertification, and overdue recertification.

The average dollar amounts are based on the sum of the dollar amounts for payment errors (either

underpayment or overpayment) for the type of household (i.e., certification, timely recertification, or

overdue recertification) divided by the number of households with that payment type.

● For  example,  the  sum  of  monthly  underpayment  dollar  amounts  for  new  certifications

($5.1 million) was divided by the total number of certifications (0.59 million). The result is

an underpayment average dollar amount of $9.

● The amount of underpayment and overpayment average dollar error in new certifications

and timely recertifications in FY 2015 ranged from $4 to $9 each month.

● As might be expected, there is a large difference in the underpayment dollar error for overdue

recertifications ($27) as well as the overpayment dollar error for overdue recertifications

($50).

It is important to note that the estimates for overdue recertifications can vary widely from year to

year because of the small number of cases (in FY 2015, 2 percent of cases were overdue).

IV-10

August 31, 2016

Exhibit IV-8

Average Monthly Underpayment and Overpayment:

Dollar Amounts Averaged Across All Households

Underpayment

Overpayment

Average Dollar Amount

Average Dollar Amount

Household Type

FY 2014

FY 2015

FY 2014

FY 2015

Certifications

$6

$9

$3

$4

Timely Recertifications

$8

$9

$5

$4

Overdue Recertifications

$29

$27

$21

$50

Total

$8

$9

$5

$4

Source: Table 8, Appendix C

Subsidies. The actual cost of errors to HUD is expressed in terms of subsidy payments. For the

purpose of this study, HUD subsidies for the PHA-administered Section 8 Voucher program equal

the lesser of the gross rent or the applicable payment standard minus the tenant share. For Public

Housing, the subsidy is the applicable payment standard minus the TTP. For Owner-administered

programs, the subsidy equals the gross rent minus the TTP. The subsidy is correct if the Actual Rent

equals the QC Rent (within $5). A negative subsidy error occurs when the tenant paid too much

rent (QC Rent < Actual Rent). A positive subsidy error occurs when the tenant paid too little rent

(QC Rent > Actual Rent).

These subsidy errors by program type are summarized in Exhibits IV-9a and IV-9b. The subsidy

errors by certification status are summarized in Exhibit IV-10.

● As shown in Exhibit IV-9a, the percentage of households with a positive subsidy error

remained steady for all program types between FY 2014 and FY 2015, at 14 percent.

● The average dollar amount of error increased for all households between FY 2014 and

FY 2015, from $8 to $9.

Exhibit IV-9a

Positive Subsidy Households (Tenant Underpayment)

Percentage of Households and Average Monthly Dollar Amount of Error

Average Dollar Amount of Error

Percentage of

For Positive Subsidy

Households

Households

in Error

(with errors > $5)

For All Households

Administration Type

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Public Housing

14%

13%

$63

$59

$9

$8

PHA-administered Section 8

14%

15%

$63

$73

$9

$11

Total PHA-administered

14%

14%

$63

$69

$9

$10

Owner-administered

12%

12%

$46

$57

$6

$7

Total

14%

14%

$59

$66

$8

$9

Source: Tables 2 and 4, Appendix C

Note: Table results replicate Exhibit IV-3a for the convenience of the reader.

HUDQC Final Report for FY 2015

IV-11

● As seen in Exhibit IV-9b, the percentage of households in error due to a negative subsidy

decreased for Public Housing households, from 13 percent to 12 percent. PHA-administered

Section 8 households in error increased by 4 percent, and Owner-administered households

increased from 8 percent to 11 percent.

Exhibit IV-9b

Negative Subsidy Households (Tenant Overpayment)

Percentage of Households and Average Monthly Dollar Amount of Error

Average Dollar Amount of Error

Percentage of

For Negative

Households

Subsidy Households

in Error

(with errors > $5)

For All Households

Administration Type

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Public Housing

13%

12%

$70

$57

$9

$7

PHA-administered Section 8

11%

15%

$51

$29

$6

$4

Total PHA-administered

12%

14%

$58

$37

$7

$5

Owner-administered

8%

11%

$27

$28

$2

$3

Total

11%

13%

$51

$35

$5

$4

Source: Tables 2 and 4. Appendix C

Note: Table results replicate Exhibit IV-3b for the convenience of the reader.

●  As seen in Exhibit IV-10, the average dollar amount of error for overpayment across

program types decreased from $5 in FY 2014 to $4 in FY 2015. That amount increased for

underpayment errors, from $8 in FY 2014 to $9 in FY 2015.

●  Overdue recertifications remained the source of the highest average dollar amount for both

underpayment and overpayment.

Exhibit IV-10

Average Monthly Underpayment and Overpayment:

Dollar Amount Averaged Across All Households

Underpayment

Overpayment

Average Dollar Amount

Average Dollar Amount

Household Type

FY 2014

FY 2015

FY 2014

FY 2015

Certifications

$6

$9

$3

$4

Timely Recertifications

$8

$9

$5

$4

Overdue Recertifications

$29

$27

$21

$50

Total

$8

$9

$5

$4

Source: Table 8, Appendix C

Note: Table results replicate Exhibit IV-8 for the convenience of the reader.

IV-12

August 31, 2016

C. Sources of Error

Additional analyses examined which income and expense components contributed the most to

rent error. It should be noted that the component dollar amounts are annual income and expense

dollars, rather than the monthly figures used to present rent error data, and that rents are generally

computed at 30 percent of adjusted income. Therefore, every $100 of annual income or expense

error generally translates into $2.50 of rent error.

In addition, the sum of the component errors is greater than Net Rent Errors because of offsetting

errors. For example, the household presented in Figure IV-3 has earned income and child care costs

with errors in both components. The total component error is $1,000 ($800 + $200); however, the

adjusted net income error (the amount used to determine the household’s rent) is only $600.

Figure IV-3

Example of the Impact of Component Errors

Component

File Data

QC Data

Dollar Error

Earned Income

$2,200

$3,000

$800

Child Care Expense

$400

$600

$200

Adjusted Income

$1,800

$2,400

$600

Exhibit IV-11 presents each income and expense component included in the rent calculation and

the percentage of households in error23 when each specific component contributed the most to the

gross error.

● The exhibit indicates that the largest average dollar error continues to be in earned income,

with an average error of $3,860. Thirty-three percent of households were in error when

earned income was the largest component error.

● Pensions, etc. were the next largest component error, with an average dollar error of $2,572

found in 15 percent of households in error.

● Other income was a component of error in 13 percent of households, with an average

associated dollar error of $2,453.

● The rent component with the greatest average dollar error increase was child care allowance,

which experienced an increase from $1,256 in FY 2014 to $2,148 in FY 2015.

● For most rent components, the percentage of households in error remained relatively stable

between years.

● The rate of households in error without a component error decreased from 7 percent in

FY 2014 to 4 percent in FY 2015.

Note: For some households, the rent error was not caused by any 1 of the 10 components listed.

Rather, it was caused by other arithmetic errors or by the use of the wrong rent calculation formula.

23   The denominator in the percentage is the number of households with any component error, which was

the percentage of total households in FY 2015.

HUDQC Final Report for FY 2015

IV-13

Exhibit IV-11

Rent Components Responsible for the Average Annual

Largest Dollar Error for Households With Rent Error

Percentage of Households in Error

Average Dollar Amount

Rent Component

FY 2014

FY 2015

FY 2014

FY 2015

Earned Income

33%

33%

$4,528

$3,860

Medical Allowance

17%

16%

$1,813

$905

Pensions, Etc.

15%

15%

$1,945

$2,572

Other Income

13%

13%

$3,122

$2,453

Elderly Household Allowance

4%

6%

$422

$400

Dependent Allowance

4%

5%

$566

$522

Public Assistance

3%

4%

$1,519

$2,133

Asset Income

3%

2%

$808

$596

Child Care Allowance

2%

2%

$1,256

$2,148

No Rent Component Error

7%

4%

$0

$0

Total

100%

100%

$2,625

$2,326

Source: Table 9, Appendix C

Total and Largest Component Dollar Error by Program Type.  Exhibit IV-12 shows the dollar

amounts associated with the total dollars in error (the sum of the absolute value of errors in all rent

components) and the largest dollars in error (the largest error attributable to a specific source for

each household), by program type.

● There were decreases in the Average Total Dollars in Error for Public Housing and Owner-

administered households in FY 2015, with the largest decrease evident in Public Housing

programs. PHA-administered Section 8 households had a modest increase in FY 2015,

with an increase of $108.

● All program types decreased slightly in Average Largest Dollars in Error, with the largest

decrease seen in Public Housing.

Exhibit IV-12

Total and Annual Component Dollars in Error for Households With Rent Error

Average Total Dollars in Error

Average Largest Dollars in Error

Administration Type

FY 2014

FY 2015

FY 2014

FY 2015

Public Housing

$4,222

$3,401

$3,523

$2,833

PHA-administered Section 8

$2,472

$2,580

$2,308

$2,270

Total PHA-administered

$3,093

$2,817

$2,739

$2,432

Owner-administered

$2,665

$2,428

$2,266

$2,019

Total

$2,989

$2,717

$2,625

$2,326

Source: Table 10, Appendix C

IV-14

August 31, 2016

QC Rent Components by Payment Type and Administration Type.  Exhibit IV-13 shows the

percentage of the total number of households with and without component error by component

type and payment type. The exhibit also provides these data by PHA- and Owner-administered

households.

● For example, 4 percent of total households with overpayment rent error had errors in

earned income, 10 percent of households with proper payment had errors in pensions, and

3 percent of households with underpayment rent error had errors in medical allowances.

● The exhibit indicates that when considering both underpayment and overpayment, pension

income, medical allowance, other income, and earned income remain the rent components

with the highest percentage of error leading to improper payment.

Exhibit IV-13 also reflects component errors in proper payment households when the component

dollar error resulted in a tenant payment error of $5 or less.

● Considering all component errors, not just those that resulted in tenant payment error, the

pensions (16 percent), earned income (16 percent), and medical allowance (12 percent)

components had the highest rates of error.

Exhibit IV-13

Percentage of Households With Rent Component Error by Payment Type

Underpayment

Proper Payment

Overpayment

Rent Component

PHA

Owner

Total

PHA

Owner

Total

PHA

Owner

Total

Earned Income

7%

4%

6%

7%

3%

6%

5%

2%

4%

Pensions

4%

3%

3%

9%

12%

10%

3%

3%

3%

Public Assistance

1%

—

1%

2%

2%

2%

1%

1%

1%

Other Income

3%

2%

3%

5%

3%

5%

3%

2%

3%

Asset Income

1%

2%

1%

3%

4%

3%

1%

1%

1%

Dependent Allowance

1%

< 1%

1%

1%

< 1%

1%

1%

1%

1%

Elderly Household Allowance

1%

< 1%

< 1%

1%

< 1%

< 1%

1%

3%

2%

Child Care Allowance

1%

< 1%

1%

< 1%

—

< 1%

1%

—

1%

Disability Allowance

—

—

—

—

—

—

—

—

—

Medical Allowance

2%

5%

3%

5%

10%

6%

3%

5%

3 %

No Rent Component Error

1%

1%

1%

49%

53%

50%

1%

—

1%

Source: Table 11, Appendix C

Allowances. Elderly/disabled and dependent allowances were examined to determine whether these

allowances were applied correctly.24 The findings are summarized in Exhibit IV-14, which shows

the percentage of elderly/disabled and non-elderly/disabled households for which allowances were

correctly or incorrectly applied.

24   Households with an elderly or disabled head or spouse are entitled to one $400 allowance (i.e.,

deduction from gross annual income) in calculating rent. Households are entitled to a $480 allowance for

each dependent (defined as children under age 18, full-time students, and disabled household members

other than the head of household or spouse).

HUDQC Final Report for FY 2015

IV-15

● Elderly/disabled allowances were incorrectly used in 3 percent of all households in FY 2015.

● Four percent of elderly/disabled households received an incorrect allowance.

● One percent of non-elderly/disabled households received an allowance erroneously.

The exhibit also shows the percentage of households with and without dependents for which a

dependent allowance was correctly or incorrectly applied.

● The dependent allowance was incorrectly applied in 6 percent of all households with

dependents. These households were given the wrong amount.

● In less than 1 percent of households, a dependent allowance was given to a household that

did not have dependents.

● A total of 3 percent of all households had an incorrect dependent allowance in FY 2015.

Exhibit IV-14

Percentage of Households With Elderly/Disabled Allowances and Dependent Allowances*

Elderly/Disabled Allowance

Dependent Allowance

Non-Elderly/

Elderly/

Households  Households

Disabled

Disabled

All

Without

With

All

Allowance

Households Households Households Dependents Dependents Households

No Allowance

99%

—

42%

100%

—

60%

Incorrect Allowance

1%

4%

3%

< 1%

6%

3%

Correct Allowance

—

96%

56%

—

94%

38%

Total

100%

100%

100%

100%

100%

100%

Source: Tables 12a and 12b, Appendix C

*Totals may not add up 100 percent to due to rounding.

D. Errors Detected Using Information Obtained From Project Files

To respond to HUD’s interest in understanding the cause of errors, tenant rent was recalculated

using only income and expense items documented in the tenant file. The source of information used

for this analysis included only items that were documented clearly in the tenant file in a location

other than Form HUD-50058/50059. If an item was recorded on Form HUD-50058/50059 but not

documented elsewhere in the tenant file, it was not included when the tenant rent was calculated

for this analysis. Therefore, it is possible that some of the discrepancies identified between Form

HUD-50058/50059 tenant rents and tenant rents calculated solely on the basis of file data were not,

in fact, due to incorrect determinations, but rather to program sponsor failure to maintain information

supporting income or expense items.

Therefore, relying solely on information in tenant files may result in misstating the basis for the

program sponsor income and rent determination and could lead to a determination that an error

existed when the determination was actually correct. The fact remains that, even if a program

sponsor made the correct income determination, failure to document the determination is a serious

administrative problem. Also, in practice it appears that these types of discrepancies are often

suggestive of subsidy determination errors, even if they cannot be assumed to prove the existence

of such errors.

IV-16

August 31, 2016

The findings from this analysis were compared with the QC findings in which tenant rent was

calculated based on all the information collected during the study (including household interview

data and verification obtained by ICF through third-party sources).

Errors Detected in the Tenant File Compared to QC Sources. Exhibit IV-15 shows the percentage

of households in error and the average dollar error based on the tenant file, but without income and

expense items identified during the household interview and verified by ICF through third-party

sources.

The percentage of households with overpayment and underpayment errors are nearly identical if

only the tenant file information or all sources of QC data were used. Findings varied considerably,

however, for the average annual dollar error.

● There was a difference of $68 in subsidy overpayment average dollar error between what

was discovered in the tenant file and what the QC review determined. When calculated as

a monthly average, this value is $6.

● For subsidy underpayment, the difference in error between the QC determination and what

was found in the file was $175 in FY 2015, or approximately $15 as a monthly average

value.

The difference found in average dollar error using information other than the tenant file implies that

there were income and expense items not listed in the file at the PHA/project.

Exhibit IV-15

Findings With and Without Information Obtained From Sources Other Than the Tenant File

Percentage of

Average Annual

Households in Error

Dollar Error

Subsidy

Subsidy

Subsidy

Subsidy

Error Source

Overpayment

Underpayment

Overpayment

Underpayment

Error Based on All Income and Expense

Items Identified During

14%

13%

$793

$416

the Study

Error Based on Tenant File Without

Income and Expense Items Identified

During the Household Interview and

13%

13%

$725

$591

Verification Obtained by ICF Through

Third-Party Sources

Source: QC Tables 2 and 4 and Tenant File Tables 2 and 4, Appendix C

Calculation and Consistency Errors. Analysis of the errors on Form HUD-50058/50059 was

completed to determine whether the errors identified using Form HUD-50058/50059 as the sole

source of information are representative of the total errors in the program. These analyses focused

on the following calculation and consistency errors.

Calculation errors were identified from income, expenses, and allowances used to calculate the

rent amount and recorded on Form HUD-50058/50059. This calculation did not take into account

whether dollar amounts were verified or whether the recertification was conducted on time. This

analysis identified errors resulting from arithmetic mistakes, the incorrect use of a formula, and

HUDQC Final Report for FY 2015

IV-17

items that were not completed but should have been. This analysis did not identify households in

which items were recorded in the wrong place on Form HUD-50058/50059, although improper

use of a field on Form HUD-50058/50059 can result in a calculation error. Table 13 in  Appendix

C presents the number of households with a Form HUD-50058/50059 that contained calculation

errors by the rent component contributing to the error. The items considered when determining

calculation error are listed in  Appendix D.

Consistency errors  were based on the logical conformity of elements in Form HUD-50058/50059.

For example, the effective date of action must be on or after the date of admission, elderly status

information should be consistent with household head and spouse ages, and number of dependents

should not exceed the number of household members. Table 14 in Appendix C shows the number of

households with consistency errors on Form HUD-50058/50059, summarized by form subsections.

Appendix D lists the data items by subsection that were included in this analysis.

Exhibit IV-16 shows the percentage of households with calculation and consistency errors by

Form HUD-50058/50059 subsections. It is important to emphasize that Form HUD-50058 is

formatted differently from and contains more line items of information than Form HUD-50059.

Consequently, the number and types of calculation and consistency errors on the forms differ, and

findings from the two forms are not directly comparable. The Office of Housing has periodically

implemented a new version of Form HUD-50059. A minor revision was made in FY 2011, and a

new version was released during FY 2014. This further complicates the comparison between the

Form HUD-50058/50059 elements.

● The large number of calculation errors (particularly in the allowances and adjusted income

from Form HUD-50058) may be a contributing factor to QC errors.

A calculation or consistency error does not necessarily lead to a rent error. The PHA/Owner may

make an error when completing one section of the form yet still calculate the rent correctly.

Exhibit IV-16

Percentage of Households With Calculation and Consistency Errors

Percentage of Households

Calculation Errors

Consistency Errors

Form

Form

Form

Form

HUD-

HUD-

HUD-

HUD-

Form HUD-50058/50059 Item

50058

50059

Total

50058

50059

Total

General Information

n/a

n/a

n/a

1%

12%

4%

Household Composition

5%

—

4%

4%

23%

10%

Net Family Assets and Income

10%

5%

8%

2%

—

1%

Allowances and Adjusted Income

42%

—

30%

9%

1%

7%

Family Rent and Subsidy Information

9%

—

7%

2%

< 1%

2%

Source: Tables 13 and 14, Appendix C

Comparison of Form HUD-50058/50059 Errors to QC Error.  A comparison was made between

the rent calculation errors on Form HUD-50058/50059 and errors identified through the QC Rent

calculation process. The purpose of this comparison was to determine whether errors identified

IV-18

August 31, 2016

using only Form HUD-50058/50059 data could predict the rent errors found in a QC review. Exhibit

IV-17 summarizes these results for FY 2014 and FY 2015.

● When using only Form HUD-50058/50059 data to calculate the Actual Rent, errors were

found in 9 percent of the households in FY 2015. This was a slight increase when compared

with the findings in FY 2014, which had a rate of 8 percent.

● The QC error calculation found errors in 26 percent of households in FY 2015, up from

25 percent in FY 2014. The results are quite different from the individual and joint

comparison methods (which were 3 percent in FY 2014 and 4 percent in FY 2015).

This comparison emphasizes that data from Form HUD-50058/50059 alone cannot accurately

identify rent error.

Exhibit IV-17

Form HUD-50058/50059 Rent Calculation Error Compared With QC Rent Error

Percentage of

Percentage of

Households With

Households With

Correctly

Incorrectly

Calculated Rent

Calculated Rent

Rent Calculation

FY 2014

FY 2015

FY 2014

FY 2015

Using Information on Form HUD-50058/50059

92%

91%

8%

9%

According to the QC Rent Calculation

75%

74%

25%

26%

Both Form HUD-50058/50059 Calculation and QC

Rent Calculation

70%

69%

3%

4%

Source: QC Table 2 and Tenant File Table 2, Appendix C

Verification Used in Determining the QC Rent. Verification errors   were identified by whether an

item was verified by the project and, if it was, whether the correct information was transferred to

Form HUD-50058/50059. An error occurs when the verified amount obtained by the project was

not used or the incorrectly calculated amount was recorded on the Form HUD-50058/50059 (and,

presumably, not used correctly in the rent calculation). When determining whether a verified income

or expense item matched the amount used on Form HUD-50058/50059, we assumed a variance of

$100 to accommodate potential rounding errors when annualizing data.

In 2010, HUD issued the  Implementation of Refinement of Income and Rent Rule, which mandated

the use of Enterprise Income Verification (EIV) as a third-party source to verify tenant employment

and income information during mandatory recertification of family composition and income. The

use of EIV minimizes the need for traditional third-party verification forms. FY 2011, the first fiscal

year affected by this rule, displayed significant verification rate decreases across the board when

compared to FY 2010, as verification was required in fewer instances. In FY 2015, this trend was

repeated, with modest decreases in items verified for the seven rent components.

The table series 15a through 15n in  Appendix C shows the number of households with and without

verification by type of verification (i.e., third-party in writing, third-party verbal, EIV, Upfront

Income Verification (UIV), and documentation). These tables provide this information for each of

the rent components and also by program type.

HUDQC Final Report for FY 2015

IV-19

As indicated above, a set of rules was established for the use of third-party verification (see Section II-C). If an income or expense component was used for a rent calculation and was not verified by

the PHA/Owner, ICF staff sought third-party verification. However, ICF verification could not be

obtained for all PHA/Owner unverified items, despite considerable effort.25 As of the FY 2011 study,

ICF has accepted third-party documentation submitted by the tenant, based on new HUD guidelines.

Exhibit IV-18 shows the percentage of each rent component that was verified by either the PHA/

Owner or ICF. Findings from FY 2015 are compared to findings from FY 2014.

The first two columns present the percentage of rent components that were verified using third-party

in writing, third-party verbal, documentation, EIV, or Upfront Income Verification (UIV), which

was counted as part of third-party in writing in studies prior to FY 2012.

● Verification of most rent component categories remained relatively stable between FY 2014

and FY 2015, with the exception of child care expenses, which saw a marked increase

(from 79 percent in FY 2014 to 100 percent in FY 2015.

As of FY 2011, the category of third-party in writing only included written third-party verification

forms, which are sent directly to the third party and completed by the third party.

● Verification using third-party in writing increased slightly in FY 2015 compared to FY 2014

for several rent components, with the most notable change seen in child care expense

verification. This increased from 42 percent in FY 2014 to 57 percent in FY 2015.

Exhibit IV-18

Percentage of Households Fully Verified by Either the PHA/Owner or ICF

Third-Party Verbal or in

Writing, Documentation,

EIV, or UIV

Third-Party in Writing

Documentation

Rent Component

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Earned Income

88%

90%

34%

31%

41%

47%

Pensions

97%

97%

9%

11%

24%

23%

Public Assistance

97%

96%

21%

22%

49%

28%

Other Income

80%

77%

25%

19%

37%

36%

Asset Income

93%

91%

41%

40%

24%

24%

Child Care Expense

79%

100%

42%

57%

34%

42%

Medical Expense

89%

96%

31%

42%

22%

19%

Source: Tables 1a, 1b, and 1e, Appendix C

25   If third-party verification was not available, documentation from the tenant file was used to calculate

the QC Rent. If neither third-party verification nor file documentation was available, documentation

collected during the household interview that met study-specific date requirements was used to calculate

the QC Rent. Information collected during the household interview that did not meet study-specific date

requirements was not used.

IV-20

August 31, 2016

Documentation of Verification. In FY 2011, HUD issued new guidelines regarding verification.

These revised guidelines indicated that documentation from a third party submitted by the tenant

is acceptable, requiring fewer instances of direct third-party requests. Such documentation would,

for example, include paystubs or letters from benefits agencies. Since FY 2011, there was a general

trend showing increased use of documentation for verification. Between FY 2014 and FY 2015,

this trend generally stabilized, with the exception of public assistance verification.

● The use of documentation decreased for public assistance documentation, from 49 percent

in FY 2014 to 28 percent in FY 2015.

Tables 1c, 1d, 1e, 1f, and 1g in  Appendix C provide additional verification information by rent

component, including the number of households for which the income or expense component

was not verified (i.e., no component items verified), partially verified (i.e., some component items

verified), or fully verified (i.e., all component items verified) by different types of verification.

●  Table 1c includes items verified by a third party in writing or by EIV/UIV.

●  Table 1d provides data for items verified by verbal third-party information.

●  Table 1e provides data for items verified via tenant file documentation.

●  Table 1f includes items verified by EIV.

●  Table 1g includes items verified by UIV.

Exhibit IV-19 summarizes the findings in Table 15a in  Appendix C.

● In FY 2015, the number of households where verification was not obtained by the PHA/

Owner increased for three of the seven rent components.

● Public  assistance  and  other  income  showed  an  increase  in  lack  of  verification,  at

7 percent and 5 percent, respectively.

● There was improvement in child care expense, which increased by 25 percent over FY 2014.

The remainder of the rent components had a relatively stable rate of project verification.

● Child care expenses were the most commonly verified rent component item in FY 2015,

with a rate of 98 percent, up from 73 percent in FY 2014.

● Percentage of verifications found to match Form HUD-50058/50059 entries within $100

decreased for three of the seven rent components in FY 2015, with the most marked

decrease being within other income. This value was 56 percent in FY 2014 and decreased

to 44 percent in FY 2015.

HUDQC Final Report for FY 2015

IV-21

Exhibit IV-19

Verification of Form HUD-50058/50059 Rent Components by PHA/Owners**

Percentage of

Percentage of

Percentage of

Households Where

Households With No

Households With Item

Verification Matched

Project Verification

Verified by Project

Form HUD50058/50059*

Rent Component

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Earned Income

16%

14%

84%

86%

58%

60%

Pensions

4%

5%

96%

95%

86%

82%

Public Assistance

15%

22%

86%

79%

69%

65%

Other Income

30%

35%

71%

65%

56%

44%

Asset Income

12%

10%

88%

90%

76%

79%

Child Care Expense

28%

2%

73%

98%

60%

73%

Medical Expense

19%

9%

81%

91%

60%

68%

Source: Table 15a, Appendix C

* Within $100

** Row totals may not add up 100 percent to due to rounding.

Verification Found Within Forms HUD-50058/50059. Exhibit IV-20 shows verification results

by form type, again presenting the verification rate for each rent component and the proportion that

matched within $100 of Form HUD-50058/50059 amounts.

Rates of verification found in the file for Form HUD-50058 remained relatively stable for six out

of the seven rent components between FY 2014 and FY 2015. The following changes are notable:

● Within Owner-administered programs using Form HUD-50059, public assistance

verification saw the largest decrease, falling from 78 percent in FY 2014 to 63 percent in

FY 2015.

● For all PHA-administered programs using Form HUD-50058, the most commonly verified

rent  components  were  pension  income  and  child  care  expenses. The  least  verified  rent

components were other income items, with a rate of 67 percent.

● Within Owner-administered programs using the Form HUD-50059, pension income and

child care expenses were most commonly verified, while other income items were the least

verified rent components.

● Other income had the lowest percentage of items that were matched within $100 based on

file documents across all program types.

IV-22

August 31, 2016

Exhibit IV-20

Verification of Form HUD-50058/50059 Rent Components by PHA/Owner Staff, by Form Type

Form HUD-50058

Form HUD-50059

Matched

Matched

Verified

Within $100

Verified

Within $100

Rent Component

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Earned Income

83%

85%

58%

59%

88%

93%

60%

66%

Pensions

96%

94%

85%

80%

97%

96%

89%

86%

Public Assistance

88%

83%

69%

67%

78%

63%

72%

55%

Other Income

71%

67%

56%

45%

68%

57%

58%

43%

Asset Income

79%

88%

64%

74%

97%

92%

86%

84%

Child Care Expense

66%

98%

55%

69%

93%

100%

75%

90%

Medical Expense

77%

89%

51%

61%

87%

93%

69%

76%

Source: Table 15h, Appendix C

Tenant File Verification Compared With QC Error. Errors identified through the QC process

were investigated to determine whether they were associated with sources of income and expenses.

Exhibit IV-21 presents the percentage of households with QC error for which verification was

missing in the tenant file. Each error is presented by rent component. The data indicate that missing

verification in both PHA- and Owner-administered programs continues to be strongly associated

with households that have QC error. This was observed for every rent component for both the

PHA- and Owner-administered programs.

●  Within PHA-administered programs, the percentage of households in error with missing

verification decreased the most for those with child care expense rent components, with

a decrease of 44 percent, from 85 percent in FY 2014 to 41 percent in FY 2015. Public

assistance components with missing verification increased from 47 percent in FY 2014 to

66 percent in FY 2015. Rates across years stayed relatively stable for the remaining five of

seven rent components.

●  Owner-administered  households  in  error  with  missing  verification  were  most  common

within those with public assistance income rent components, followed by other income

rent components. Households with child care expense rent components showed the

largest decrease in households in error with missing verification between FY 2014 and

FY 2015 (53 percent to 31 percent, respectively).

Across all program types, the percentage of households in error with missing verification was lowest

among households with pension income rent components, which was also the lowest percentage in

FY 2014. However, for some of these components, the number of households in error was relatively

small; therefore, the estimates may vary substantially from year to year and may not be reliable.

HUDQC Final Report for FY 2015

IV-23

Exhibit IV-21

QC Error Households With Missing Verification in the Tenant File

Form HUD-50058

Form HUD-50059

Percentage of

Percentage of

Percentage of

Households With

Percentage of

Households With

Households With

QC Errors and

Households With

QC Errors and

QC Error

Missing Verification

QC Error

Missing Verification

Rent Component

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

Earned Income

11%

12%

52%

48%

6%

6%

54%

38%

Pensions

6%

7%

33%

31%

5%

6%

36%

29%

Public Assistance

2%

2%

47%

66%

1%

1%

76%

100%

Other Income

5%

6%

52%

47%

3%

3%

73%

67%

Asset Income

2%

2%

62%

57%

3%

3%

39%

37%

Child Care Expense

1%

1%

85%

41%

1%

< 1%

53%

31%

Medical Expense

6%

6%

69%

79%

8%

8%

62%

64%

No Component Error

77%

75%

—

—

82%

80%

—

—

Source: Tables 16a and 16b, Appendix C

Summary of Form HUD-50058/50059 Errors.  Exhibits IV-22a and IV-22b provide a summary of

the errors identified from Form HUD-50058/50059. These included consistency errors, calculation

errors, and overdue recertifications.

Note: Exhibit IV-22a excludes Moving to Work (MTW) cases, as they do not have Form

HUD-50058/50059 recalculated rent error, and Exhibit IV-22b shows all cases with QC Rent error.

Both exhibits show the percentage of households in error, the average dollar error, and the standard

errors for both households with recalculated Form HUD-50058/50059 error (i.e., error determined

using only Form HUD-50058/50059) and households with QC Rent error. This information is

provided for households with error by each error type.

Beginning with the FY 2005 study, transcription error for any household was added to the source

table, and the data that were described as an unduplicated count of Form HUD-50058/50059 error

was revised to indicate an unduplicated count of any type of administrative error.

●  Exhibit IV-22b shows that several individual types of Form HUD-50058/50059 errors

were not closely associated with QC Rent error, such as allowance calculation, income

calculation, and other calculation errors.

●  Form HUD-50058/50059 with only transcription errors were associated with QC Rent

error in 49 percent of households.

●  Any type of administrative error, which includes transcription, consistency, calculation, or

overdue recertifications, was associated with QC Rent error in 57 percent of the households.

This increase over those households in error with only transcription errors is primarily due

to the small number of households with income calculation error, resulting in estimates

with variances that are rather large from year to year.

●  When compared to FY 2014, there is an increase in FY 2015 in the percentage of households

in error for both recalculated Form HUD-50058/50059 and for households with QC Rent

IV-24

August 31, 2016

error, while the average dollars in error have decreased for households with recalculated

Form HUD-50058/50059.

● The highest average dollar error decrease for recalculated Form HUD-50058/50059

pertains to income calculation error. This was $106 in FY 2014 and has decreased to $40

in FY 2015.

● In addition, the average dollar error for households with any recalculated Form

HUD-50058/50059 error was $13.

● In contrast, the average dollar error for households with QC Rent error was $51.

These values support the assertion that an administrative error on Form HUD-50058/50059 is not

necessarily associated with a QC Rent error.

To understand the reason for the change in the average dollar error for households with recalculated

Form HUD-50058/50059 error, it is important to review how this number is calculated. The number

is the average dollar rent error for all cases with error in the category (based on recalculated Form

HUD-50058/50059 rent error, not QC Rent error).

So, for example, although the average rent error dollars for households with allowance calculation

error is $70, because many of these cases have a large rent error (which may have nothing to do

with the allowances) and the number of cases with allowance calculation error is small (2 percent of

households in error), the average dollar error is large. The combination of a small number of cases

with allowance calculation errors and a large dollar value of rent error leads to a comparatively

large average dollar error.

Exhibit IV-22a

Form HUD-50058/50059 Administrative Error: Percentage of Households,

Average Dollars in Error (Non-MTW Households)

Non-MTW Households With Recalculated

Form HUD-50058/50059 Error

Percentage of

Standard

Standard

Error Type Based on Form HUD50058/50059

Households

Error of

Average

Error of

Recalculation

in Error

Percent

Dollar Error

Mean

Households With Transcription Error

49%

4.7%

$12

$3.27

Households With Consistency Error

16%

3.8%

$15

$6.06

Households With Allowance Calculation Error

2%

0.7%

$70

$43.72

Households With Income Calculation Error

3%

1.2%

$40

$28.45

Households With Other Calculation Error

7%

2.8%

$20

$14.31

Overdue Recertifications

2%

1.2%

$5

$5.14

Unduplicated Count, Any Type of Administrative Error

57%

5.0%

$11

$2.88

Total Households

100%

—

$13

$4.54

Note: Data exclude MTW households; MTW cases do not have Form HUD-50058/50059 recalculated rent error.

Source: Table 17a, Appendix C

HUDQC Final Report for FY 2015

IV-25

Exhibit IV-22b

Form HUD-50058/50059 Administrative Error: Percentage of Households,

Average Dollars in Error (Households With QC Rent Error)

Households With QC Rent Error

Percentage of

Standard

Standard

Error Type Based on Form HUD50058/50059

Households

Error of

Average

Error of

Recalculation

in Error

Percent

Dollar Error

Mean

Households With Transcription Error

82%

2.7%

$51

$4.55

Households With Consistency Error

22%

2.4%

$46

$7.47

Households With Allowance Calculation Error

2%

0.6%

$68

$36.80

Households With Income Calculation Error

3%

0.9%

$94

$29.81

Households With Other Calculation Error

6%

1.3%

$89

$24.22

Overdue Recertifications

4%

1.0%

$113

$35.28

Unduplicated Count, Any Type of

Administrative Error

86%

2.8%

$51

$4.26

Total Households

100%

—

$51

$4.09

Source: Table 17b, Appendix C

Summary of Administrative Errors. As outlined in the study objectives, calculation errors,

consistency errors, transcription errors, failure to recertify on time, and failure to apply allowances

appropriately produce administrative errors.

Exhibit IV-23 shows the Gross Rent Error and Net Rent Error for households with each type of

administrative error.

●  In FY 2015, the percentage of households in error was generally very similar to the

percentage in FY 2014 for all error types, though households with transcription error

decreased by 2 percent between FY 2014 and FY 2015.

●  Gross Rent Error calculated from specific types of error increased slightly in FY 2015 as

compared to FY 2014 values for all seven categories.

●  Net Rent Error calculated from the various types of error increased in FY 2015 for the

majority of error types as compared to FY 2014.

●  Overdue recertifications had large differences in error amounts due to the small number of

overdue cases.

IV-26

August 31, 2016

Exhibit IV-23

Administrative Error: Percentage of Households Average Dollars in Error for All Households

Gross Rent Error

Net Rent Error

Percentage of  Average  Standard  Average  Standard

Households in  Dollars in  Error of  Dollars in  Error of

Error Type

Error

Error

Mean

Error

Mean

Households With Transcription Error

42%

$26

$2.68

-$8

$2.69

Households With Consistency Error

21%

$13

$2.65

-$5

$1.64

Households With Allowance Calculation Error

1%

$32

$16.45

-$23

$16.60

Households With Income Calculation Error

2%

$34

$11.72

-$15

$12.51

Households With Other Calculation Error

4%

$39

$10.36

$9

$11.70

Overdue Recertifications

2%

$77

$25.58

$23

$25.78

Unduplicated Count, Any Type of

Administrative Error

53%

$22

$2.12

-$7

$2.15

Total Households

100%

$13

$1.29

-$4

$1.17

Source: Table 18, Appendix C

E. Occupancy Standards

Exhibit IV-24a presents a summary of the analysis that determined whether households are assigned

units with the correct number of bedrooms. It shows the percentage of households by actual number

of bedrooms and correct number of bedrooms according to the guidelines used in the study. Note

that the guidelines used in this study are generally acceptable HUD guidelines. All programs

allow exceptions to HUD’s rules. For example, the PHA-administered Section 8 Voucher program

sometimes allows households to rent units with fewer or more bedrooms than specified by the

guidelines.

● Sixteen percent of all households occupied a unit with too many or too few bedrooms in

FY 2015, compared to 17 percent in FY 2014.

● Sixteen percent of Public Housing households were over- or under-housed in

FY 2015.

● Twenty-two percent of PHA-administered Section 8 program households were under- or

over-housed in FY 2015, the same percentage found in FY 2014.

● Eight percent of Owner-administered households were under- or over-housed in FY 2015.

HUDQC Final Report for FY 2015

IV-27

Exhibit IV-24a

Percentage of Households in Units With the Correct

Number of Bedrooms (According to Study Guidelines)

PHA-administered

Number of

Public Housing

Section 8

Owner-administered

Total

Bedrooms

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

FY 2014

FY 2015

0

98%

99%

100%

97%

96%

98%

98%

98%

1

98%

99%

100%

100%

99%

99%

99%

100%

2

73%

73%

63%

68%

78%

75%

69%

71%

3

78%

78%

83%

72%

79%

81%

81%

74%

4

68%

59%

46%

65%

57%

56%

53%

62%

5+

24%

8%

47%

29%

—

—

53%

22%

All Units

83%

84%

78%

78%

91%

92%

83%

84%

Source: Table 19, Appendix C

Exhibits IV-24b and IV-24c show the percentage of households that met these guidelines for each

bedroom size for FY 2014 and FY 2015, respectively. The shaded cells indicate the percentage of

households that fell within study guidelines.

Exhibit IV-24b

Percentage of All Households in FY 2014 by Number

of Bedrooms and Number of Household Members*

Number of

FY 2014 Number of Household Members

Bedrooms

1

2

3

4

5

6

7

8+

0

98%

2%

1

91%

8%

1%

2

29%

40%

24%

5%

2%

1%

3

7%

12%

35%

26%

13%

6%

1%

4

2%

5%

14%

24%

25%

15%

4%

11%

5+

9%

26%

12%

6%

46%

Source: Table 19a, Appendix D of the  HUDQC Final Report for FY 2014, delivered on September 25, 2015.

*Row totals may not add up 100 percent to due to rounding.

IV-28

August 31, 2016

Exhibit IV-24c

Percentage of All Households in FY 2015 by Number

of Bedrooms and Number of Household Members*

Number of

FY 2015 Number of Household Members

Bedrooms

1

2

3

4

5

6

7

8+

0

98%

1%

1%

1

91%

9%

< 1%

< 1%

< 1%

2

27%

43%

20%

7%

2%

1%

< 1%

3

9%

13%

33%

24%

12%

6%

2%

1%

4

3%

5%

9%

22%

27%

18%

10%

9%

5+

3%

6%

9%

36%

18%

13%

14%

Source: Table 19a, Appendix C

*Row totals may not add up 100 percent to due to rounding.

F. Rent Reasonableness

The PHA-administered Section 8 program assists low-income families in obtaining housing in the

private market. A PHA responsible for administering the program must not enter into a HAP contract

until the housing authority has determined that the initial rent paid to the owner is a reasonable

amount. The PHA must also determine whether the rent to the owner is reasonable in comparison

to rent for other comparable unassisted units.

Rent reasonableness is an important factor in determining participant subsidies and is critical for

effective, PHA-administered Section 8 Voucher program operations. If PHAs approve rents that

are low compared to the private market, landlords may only participate with their lowest cost,

lowest quality units, or they may not rent out their units at all to program participants. HUD

regulations require PHAs to conduct a rent reasonableness determination before units are leased,

before rent increases are granted to owners, and when Fair Market Rents decrease by at least

5 percent. Our analysis examines whether PHAs fulfilled the requirement for documenting rent

reasonableness determinations, but does not investigate whether rents were in fact reasonable.

Methodology. We surveyed PHAs administering the Section 8 Voucher program in our study. This

year, 137 projects26 in our study fall into this category. The projects were asked about their standard

rent reasonableness processes and file documentation from the project’s household sample were

reviewed.

We instructed field interviewers to review tenant files for 787 Voucher households in order to

locate the documents supporting the rent reasonableness certification. For 113 new certifications,27

field interviewers reviewed the file for the initial rent reasonableness certification and recorded

the  date  it  was  conducted.  For  the  674  annual  recertifications  we  reviewed,  field  interviewers

were asked to ascertain when the current rent to the owner became effective and to locate the

26   For the purposes of this study, a project for the Section 8 Voucher program is defined as a PHA/county

combination. Therefore, if a PHA administers vouchers in more than one county, that PHA could be

represented in this study by more than one project.

27   Beginning in FY 2007, portability move-ins were classified as annual recertifications. In FY 2006, they

were categorized as new admissions.

HUDQC Final Report for FY 2015

IV-29

relevant supporting rent reasonableness documentation. If this documentation (relative to the

date that the rent to the owner became effective) was not found, field interviewers were asked to

search for any rent reasonableness certification in the file and enter the date of certification. The

owner’s rent certification on the Request for Tenancy Approval form was considered a certification

of rent reasonableness.

Findings Pertaining to Rent Reasonableness Methods Used by PHAs. The most common

method of determining rent reasonableness is the unit-to-unit comparison (see Exhibit IV-25).

Sixty-two percent of the housing authorities that responded reported using unit-to-unit comparison

as the predominant method for their rent reasonableness determination. The unit-to-unit method

is similar to the standard real estate appraisal technique of comparing a unit to similar private,

unassisted units in the same general location. Rent amounts are sometimes modified for differences

in unit characteristics (e.g., size, age, amenities, housing services, maintenance, or utilities).

Exhibit IV-25

PHAs by Predominant Rent Reasonableness Method*

FY 2013

FY 2014

FY 2015

Method

Number

Percent

Number

Percent

Number

Percent

Unit-to-unit Comparison

96

69%

91

64%

85

62%

Unit-to-market Comparison

21

15%

25

17%

25

18%

Point System

17

12%

17

12%

20

15%

Other or Rent Control

6

4%

6

4%

5

4%

No Single Predominant Method

0

—

2

1%

2

1%

No Information

0

—

2

1%

0

1%

Total

140

100%

143

100%

143

100%

*Column totals may not add to 100 percent because of rounding.

The unit-to-market comparison approach estimates the average and/or range of “market” rents for

units with similar characteristics in the private, unassisted market. Eighteen percent of housing

authorities reported primarily using this method. Valuation adjustments are based on typical

units in the private market. Fifteen percent of housing authorities indicated that their primary

method of  making rent reasonableness determinations was based on a point system. Using

this system, units are assigned points based on their condition and attributes, and comparisons

are made to unassisted units.

We asked PHA staff to identify only the primary method used to determine whether rents to owners

were comparable to the private market, rather than enter a percentage use of various methods. When

asked to identify a single predominant method, most PHAs selected only one. Two PHAs selected

“no single method predominates,” the same number of PHAs as in FY 2014 as opposed to none in

FY 2013. PHAs were also asked whether they used a software program and/or an outside contractor

to determine whether the rent to owner was reasonable. Eighty-eight of the 137 voucher projects

(64 percent) use rent reasonableness software. GoSection8.com    remained the most commonly

used software vendor, cited by 33 projects in FY 2015 and 37 projects in FY 2014, followed

by Nelrod EZ Reasonable Rent Determination, used by 9 projects in FY 2015 and 10 projects

IV-30

August 31, 2016

in FY 2014. Three PHAs reported using software and systems developed in-house in FY 2015,

compared with seven in FY 2014.

Findings Pertaining to Rent Reasonableness Documentation Found in Tenant Files for

New Admissions and Annual Recertifications.  In FY 2015, 96 percent of new admission files

contained rent reasonableness documents, up from 90 percent in FY 2014 and 91 percent in

FY 2013 (see Exhibit IV-26a). Annual recertifications require rent reasonableness documents only

when owners increase rental rates. We examined case files to determine when the current rent

to owner first became effective and reviewed the file for the rent reasonableness documentation

specific to that rent determination. If no rent reasonableness documentation was found within this

specific timeframe, we reviewed any rent reasonableness documentation in the file. In FY 2015,

89 percent of these case files had certified rent reasonableness documents, compared to 87 percent

in FY 2014 and 82 percent in FY 2013 (see Exhibit IV-26a).

Exhibit IV-26a

Rent Reasonableness Documentation for New Admissions and Annual Recertifications

FY 2013

FY 2014

FY 2015

Status

New Admissions

Recertifications

New Admissions

Recertifications

New Admissions

Recertifications

Determination Documented

91%

82%

90%

87%

96%

89%

No Determination Documented

9%

18%

10%

13%

4%

11%

Total

100%

100%

100%

100%

100%

100%

The absence of rent reasonableness documentation does not necessarily indicate that a determination

was not completed, only that it was not properly documented. Of new admission files that had

documentation, 53 percent contained a statement signed by the PHA staff member certifying that

the rent was reasonable. For recertifications with rent reasonableness documentation, 70 percent

contained a statement signed by the PHA staff member certifying that the rent was reasonable (see

Exhibit IV-26b). The increase in the use of Form HUD-52517 as a rent reasonableness source

document among new admissions may be a result of study procedure that directed the data collector

to locate the Request for Tenancy Approval as a utility document. Thirty-three percent of new

admission rent reasonableness forms found were documented on the Form HUD-52517 in FY 2015,

compared to 12 percent in FY 2014 and 14 percent in FY 2013.

HUDQC Final Report for FY 2015

IV-31

Exhibit IV-26b

Type of Rent Reasonableness Documentation for New Admissions and Annual Recertifications*

FY 2013

FY 2014

FY 2015

Type of Documentation

New Admissions

Recertifications

New Admissions

Recertifications

New Admissions

Recertifications

A signed statement certifying the rent is reasonable

50%

54%

61%

62%

53%

70%

Comparable units documented by the property

owner in Section 12a of Form HUD-52517

14%

10%

12%

11%

33%

16%

Comparable units documented on other documents

33%

30%

22%

21%

9%

11%

Any other reference to rent reasonableness

3%

5%

6%

6%

6%

3%

Total

100%

100%

100%

100%

100%

100%

*Totals may not add up to 100% due to rounding.

HUD requires that rent reasonableness determinations be conducted before signing the contract and

lease. The timeliness of the rent reasonableness determination was evaluated by comparing the lease

date (depending on the type of transaction, the lease date is the effective date of the current contract

rent or the lease start date) with the rent reasonable certification date in the case file. Since the PHA

is required to conduct a rent reasonableness assessment when the contract rent is increased by the

owner, the current contract rent is compared with the previous rent amount to determine when and

whether there was a change in the contract rent. This data is used to determine whether there was

a timely rent reasonableness assignment. Exhibit IV-27 provides a summary of how the date of the

rent reasonableness documentation relates to the initial lease date or contract rent change date for

those households where a reference to the rent reasonableness determination was found in the file.

Exhibit IV-27

Timing of Most Recent Rent Reasonableness Determination:

New Admissions and Annual Recertifications*

FY 2013

FY 2014

FY 2015

sions

tions

sions

tions

sions

tions

Determination: Certification Chronology

New Admis

Recertifica

New Admis

Recertifica

New Admis

Recertifica

More than 4 months before lease date

7%

40%

5%

28%

0%

22%

Up to 4 months before lease date

82%

50%

86%

47%

84%

60%

Up to 2 months after lease date

10%

3%

3%

3%

8%

4%

Greater than 2 months after lease date

0%

2%

1%

15%

1%

11%

Date missing

2%

5%

5%

7%

7%

3%

Total

100%

100%

100%

100%

100%

100%

*Totals may not add up to 100% due to rounding.

IV-32

August 31, 2016

If the effective date of the lease with the current contract rent occurred prior to the date of the rent reasonableness documentation, rent reasonableness may not have been considered as a factor in

approving the unit’s rent. In FY 2013, ICF changed the data collection methodology slightly, as

discussed in Section II, and we collected the date the rent was in effect. This explains the greater

number of cases with certifications occurring more than 4 months before the lease date in FY 2013,

a trend that continued in FY 2014 and FY 2015. The percentage of rent reasonable determinations

made after the rent had been established as part of the initial lease agreement increased for new

admissions from 4 percent in FY 2014 to 9 percent in FY 2015. For annual recertifications in

FY 2015, the percentage of rent reasonable documentation dated after the effective date of a lease

decreased from the previous study year: 15 percent in FY 2015 compared to 18 percent in FY 2014.

Conclusion.  PHAs are not fully documenting rent reasonableness determinations as required

by HUD regulations, and a large number of existing rent determinations may have been made

on the basis of less formal means of evaluating rents. Timely reviews decreased slightly in

FY 2015 compared to FY 2014 for new admissions and increased for recertification transactions.

The proportion of cases lacking any rent reasonableness decreased compared to FY 2014.

Four percent of new admissions and 12 percent of annual recertification transactions lacked a rent

reasonableness document in FY 2015, compared to 10 percent and 13 percent, respectively, in

FY 2014. These findings may be attributed in part to PIH 2003-12, issued May 16, 2003, which

supports a more simplified rent reasonableness determination process. PIH 2003-12 states that

a PHA need not consider all nine criteria cited in 24 C.F.R. 982.507(b) to fully comply with the

regulation. It justifies less formal methods of rent determination, stating that “each PHA should use

appropriate and practical procedures for determining rental values in the local market.”

G. Utility Allowance Analysis

As part of the FY 2015 HUDQC Study, ICF conducted two separate analyses on the utility

allowances provided to households through the PHA-administered Section 8 Voucher program.

For the first analysis, we focused on whether there was documentation in the tenant file indicating

how the utility allowance amount used in the rent determination was calculated, and whether those

documents were used correctly in calculating the utility allowance amounts. For the second analysis,

we focused on identifying discrepancies between the utility allowance on the Form HUD-50058 and

the appropriate utility allowance as listed on a PHA staff-provided utility allowance schedule. These

schedules often varied within a county by unit type, effective date of recertification, and location.

Documentation of Utilities and Utility Allowance Values. We asked PHAs to provide information

about the forms that were used to document and calculate the utility allowance and to provide the

utility allowance schedules that were used for actions effective in FY 2015. In addition, we asked

field interviewers to copy documents showing calculation of utility allowances found in tenant files

at the PHA office.

ICF staff selected 137 distinct PHA-administered Section 8 Voucher projects for the study sample.

These projects, administered by 120 housing authorities (several of which administered projects in

multiple counties), participated in the FY 2015 HUDQC Study. According to information provided at

the PHA level by 128 projects, 33 percent of the projects used Form HUD-52667 (Allowance Schedule)

as the official source for identifying the utilities for which the households were responsible. This

HUDQC Final Report for FY 2015

IV-33

was a change from FY 2014, when Form HUD-52517 (Request for Tenancy Approval) was the most

commonly used document.

In FY 2015, 24 percent of projects used Form HUD-52641 (HAP Contract) for identification purposes,

an increase from 14 percent in FY 2014. With respect to the document used to calculate utility al owance

value, the most common document was Form HUD-52667 (Allowance Schedule) in FY 2015. In

Exhibit IV-28a, we provide information on the type of documents used as the official source for

identifying utilities for which the households were responsible.

Exhibit IV-28a

Types of Documents Used by PHAs to Identify Utilities and Calculate the Utility Allowance Value*

Identifying Utilities

FY 2014

FY 2015

Document Used

Number

Percent

Number

Percent

Form HUD-52517 (Request for Tenancy Approval)

46

34%

36

28%

Form HUD-52641(HAP Contract)

19

14%

31

24%

Form HUD-52667 (Allowance Schedule)

43

32%

42

33%

Other (e.g., lease, reports, comparisons)

11

8%

14

11%

Various combinations of documents above

15

11%

5

4%

Total

134

100%

128

100%

*Totals may not add up to 100% due to rounding.

Comparison of Form HUD-50058 Utility Allowance Values to Worksheets Found in the

Household File. Seven hundred and ninety-nine (799) households from the PHA-administered

Section 8 Voucher program were selected for this study. Field interviewers were able to

locate worksheets or other documents indicating how the utility allowance was calculated for

753 households (94 percent).

For each household with utility allowance documentation available, we compared the utility

allowance amount from the Form HUD-50058 to the amount on the utility allowance worksheet

obtained  from  the  tenant  file.  For  627  households  (83  percent)  the  Form  HUD-50058  utility

allowance amount matched the worksheet amount. For 11 percent of the households, the worksheet

provided was for the incorrect period of time or was missing critical information. For these

79 households, we could not determine whether the utility allowance amount used in the rent

calculation was correct. In the remaining 6 percent of the households, there were discrepancies

between the amounts on the worksheet and on the Form HUD-50058.

In FY 2015, there were slightly more households with matching values than in FY 2014

(83 percent and 81 percent, respectively). There were fewer inadequate worksheets in FY 2015

(11 percent) as compared to FY 2014 (17 percent). Exhibit IV-28b presents a summary comparing the

utility allowance listed on the Form HUD-50058 and the amount on worksheets found in tenant files.

IV-34

August 31, 2016

Exhibit IV-28b

Comparison of Utility Allowance on the Form HUD-50058 to the Utility Allowance Worksheet

Outcome

Number

Percent

Form HUD-50058 (AC) amount matched with worksheet amount

627

83%

Worksheet in file for incorrect period of time or is missing critical information

79

11%

Discrepancy due to mathematical error or other clerical errors

25

3%

Discrepancy—Unable to determine reasons

22

3%

Total

753

100%

Comparison of Form HUD-50058 Utility Allowance Values to the Correct (QC) Utility Allowance

Value. The ICF team calculated the QC utility allowance amount in two steps. In the first step, the

utilities that the tenant was responsible for were identified by using documents—usually PHA utility

allowance worksheets—found in tenant files that indicated those specific utilities. In the second

step, we mapped the identified household’s specific utilities onto the utility allowance schedule and

summed the total to determine the QC allowance amount.

We matched the utility allowance amount on Form HUD-50058 to the QC utility allowance

amount. We were unable to calculate the QC utility allowance for 2 percent of the households

(15 households) because worksheets were not available; consequently, the specific utilities paid

by the household could not be identified. Furthermore, we were unable to calculate the QC utility

allowance in an additional 1 percent of households because the worksheets in the files did not

include specific utilities or other critical information needed for QC allowance calculation. Another

1 percent could not be calculated because the appropriate utility allowance schedule was not

available. Exhibit IV-28c differentiates between the households in which we were able to calculate

the QC allowance amount and lists the reasons and number of households where we were unable

to calculate the QC utility allowance amount.

Exhibit IV-28c

Availability of All Information to Enable QC Utility Allowance (UA) Calculation*

QC UA Amount

Outcome

Calculated

Number

Percent

Appropriate worksheet and schedule available

Yes

771

97%

UA worksheet or other comparable document not available

No

15

2%

Appropriate UA schedule not available

No

4

1%

Worksheet was missing critical information

No

9

1%

Total

799

100%

*Column totals may not add up 100 percent to due to rounding.

We calculated the QC utility allowance amounts for 771 households and then compared the QC util-

ity allowance to the Form HUD-50058 utility allowance amount. In 87 percent of these households,

Form HUD-50058 and the QC utility allowance values matched. We categorized the remaining

13 percent of households where the values did not match into two broad categories: administrative

error or unknown (i.e., ICF was unable to determine the reason for the discrepancy in utility al-

lowance amounts). The majority of these errors were categorized as administrative. We present the

findings from this analysis in Exhibit IV-28d.

HUDQC Final Report for FY 2015

IV-35

Exhibit IV-28d

QC Utility Allowance Compared to Form HUD-50058 Utility Allowance

Outcome

Number

Percent

QC UA matched amount on Form HUD-50058

671

87%

Discrepancy due to math error/transfer error

84

11%

Discrepancy—Unable to determine reasons

16

2%

Total

771

100%

Note: The QC rent that is calculated for this study uses the utility allowance amount from Form HUD-50058 and not the QC

allowance amount that was calculated for this comparison.

H. Payment Standard Analysis

As part of the FY 2015 HUDQC study, ICF conducted a special analysis to determine whether

PHAs are using the correct payment standard amount. This special analysis was independent of the

rent calculation error findings presented in another section in this report, and it did not affect rent

calculation determinations. The payment standard analysis consisted of three parts:

1.   The payment standard on Form HUD-50058 was compared to the payment standard

schedules provided by the PHA.

2.  The payment standard on Form HUD-50058 was compared to the Fair Market Rent (FMR)

for the appropriate geographical area.

3.  The payment standards were compared to the FMRs to ensure that they fell between

90 percent and 110 percent of FMR for each project.

The findings from these comparisons are presented below.

Background.  Payment standards are used in the PHA-administered Section 8 Voucher program to

determine the tenant’s portion of the rent to owner. Payment standards must be kept current and set

between 90 percent and 110 percent of the FMR. If a PHA does not ensure that its payment standards

are within this range or if program administrators fail to apply the current payment standards, this

will result in errors in tenant rent determinations.

PHAs can apply payment standards incorrectly in a variety of ways that will result in errors in tenant

rent. A PHA may have several payment standards for different geographic areas with complex

borders and/or based on groups of ZIP codes or municipalities that change every few years. This can

make it difficult to select the correct payment standard for any given address within the jurisdiction.

Additionally, a household can rent a unit whose number of bedrooms differs from the authorized

number of bedrooms on the voucher, which is based on family size. In such a case, program

administrators must remember to use the lower of the payment standard based on the voucher

bedroom size or the payment standard for the actual size of the unit leased. Annual changes in

payment standards can cause similar confusion; payment standard increases take effect immediately,

but administrators may forget that there is a 1-year delay before decreases affect the determination

of household rents or assume that the delay extends to increases as well.

Other potential areas for error include whether a PHA has been authorized to use FMRs based on

the 50th percentile of the rents in the area, whether the PHA has been authorized to use Success

Rate Payment Standards based on the 50th percentile of rents, and whether the PHA continues to be

IV-36

August 31, 2016

eligible for these higher subsidy standards. Moreover, PHAs are allowed to change a household’s

payment standard only at the time of the annual recertification or before the household moves to a

new address. Thus, even if a change in the family composition requires an interim recertification

with several family members moving in or out, the payment standard used to determine the rent

should not be changed at the interim recertification. Yet, despite the complexity of payment standard

guidelines, most of the errors found in this review were not due to this reason.

Comparison of the Payment Standard on Form HUD-50058 With the Payment Standard

Schedules Provided by the PHA. For the first analysis, we compared the payment standard on

Form HUD-50058 (the actual, or AC, payment standard) with the payment standard schedule (the

quality control, or QC, payment standard) provided by the PHA. For all Voucher households in

the study, we selected the appropriate QC payment standard when possible and compared it with

the AC payment standard. We selected the QC payment standard from the schedules provided by

the PHA on the basis of:

● The lower of either the number of bedrooms in the unit or the number of authorized

bedrooms for the household on the voucher.

● The Effective Date of Action.

● The determination and application of any special exception to payment standard guidelines

provided by the PHA staff.

In previous years, for every household where the AC and QC payment standard did not match,

we placed a call to the PHA staff for clarification and collected payment standard schedules when

appropriate. Historical discussions with projects on the determination of the QC payment standard

uncovered a host of other issues that required consideration when selecting the QC payment standard.

While time restraints prevented us from conducting calls to PHA staff for FY 2015, types of past

complications considered during the analysis included the following:

● Some PHAs use the previous (higher) payment standard amount for the first recertification

after a decrease in the payment standard amount. Exceptions for special circumstances,

such as living in a house with additional amenities or setting the payment standard to the

gross rent for Enhanced Vouchers, may be granted to some households.

● Exception Rent Areas may have higher payment standards.

● Some PHAs use payment standards from the initial housing authority for port-in households,

with the understanding that the rates would be adjusted at the next annual recertification.

● Some PHAs use software systems that identify the lesser of gross rent or the payment

standard to populate the payment standard field on Form HUD-50058.

The study included 791 PHA-administered households. Of these, ICF was able to determine a

QC payment standard for 661 households, or 84 percent. For the majority of these households

(82 percent), the AC payment standard matched the QC payment standard. There were

118 households with discrepant payment standards; 74 (63 percent) of the households with

discrepant payment standards were elderly or disabled households. Elderly and disabled

households are identified separately because they are often entitled to individual exemptions to

the payment standard rules. We attributed discrepancies to one of seven common reasons, as listed

HUDQC Final Report for FY 2015

IV-37

in Exhibit IV-29a. The most typical reason for a discrepancy between the AC and QC payment

standard was for reasons categorized as Other: overdue recertifications, a PHA using the FMR

rather than the payment standard, and/or enhanced vouchers, etc. (32 percent). The use of incorrect

payment standard schedules accounted for a cumulative 24 percent of the discrepancies found,

while approximately a fifth of discrepancies (22 percent) were attributable to the use of a project-

based voucher. Exhibit IV-29a summarizes the number and percentage of households with payment

standard discrepancies and indicates the reasons for these discrepancies.

Exhibit IV-29a

Number and Percentage of Households With Payment Standard Discrepancies

Number of

Number of

Percent of

Households

Households

Households

(Elderly/

(Non-Elderly/

With

Reason

Disabled)

Non-Disabled)

Discrepancies

Used incorrect number of bedrooms/household members

7

1

7%

Used incorrect payment standard schedule

15

13

24%

Used gross rent instead of the payment standard amount

4

2

5%

Project staff made a typographical error

4

7

9%

Section 12 of Form HUD-50058 was incomplete/missing

1

0

1%

Project-based voucher: No payment standard (Section 11

filled out)

19

7

22%

Other reasons (e.g., overdue recertification, enhanced

voucher)

24

14

32%

Total

44

74

100%

Comparison of the Payment Standard on Form HUD-50058 to the Fair Market Rent for the

Appropriate Geographic Area. In the second analysis, we compared the payment standard on

Form HUD-50058 to the FMRs for the appropriate geographic area. The payment standard for

552 households (70 percent) fell within the 90 percent to 110 percent FMR band; 39 of the

households (5 percent) that fell outside the 90 percent to 110 percent band used an amount that

exceeded 110 percent of the FMR; and 34 of the households (4 percent) that fell outside the band

used an amount that was less than 90 percent of the FMR. Exhibit IV-29b summarizes the number

and percentage of households by the relationship of the payment standard to the acceptable FMR. The

table is based on data for the 661 cases where we were able to determine correct payment standards.

IV-38

August 31, 2016

Exhibit IV-29b

Payment Standard Compared With the Fair Market Rent

Fair Market Rent

Percent of  Total

Cases Outside the

Characteristic

Under 90%

90%–110%

Over 110%

90% to 110% Band

Non-elderly or Non-Disabled

10

292

17

3%

Elderly or Disabled

24

260

22

6%

Total

34

552

39

9%

The analysis of cases that fell outside the 90 percent to 110 percent FMR band revealed that

12 percent of cases fell outside the FMR band for four general reasons: (1) the incorrect number of

bedrooms or household members was used; (2) the incorrect payment standard schedule was used;

(3) project staff used gross rent instead of the payment standard; or other reasons. Exhibit IV-29c

summarizes the number and percentage of households that fall outside the 90 percent to 110 percent

FMR band by category.

Exhibit IV-29c

Details of Cases Falling Outside 90%–110% of the Fair Market Rent*

Fair Market Rent

Percent of Cases

Outside the

Reason

Under 90%

Over 110%

90% to 110% Band

Used incorrect number of bedrooms or household members

6

0

8%

Used incorrect payment standard schedule

3

2

7%

Used gross rent instead of the payment standard amount

0

4

5%

Other reasons—overdue recertification, used 105% of FMR,

software limitations, original payment standard over 110%,

25

33

79%

unable to determine a reason for the discrepancy

Total

34

39

100%

*Totals may not add up 100% to due to rounding.

Comparison of the FY 2014 to the FY 2015 Payment Standard Analysis Results. ICF conducted

the same payment standard analysis for the FY 2014 study. Of the 791 PHA-administered Section

8 Voucher households in FY 2015, 661 had a QC payment standard determination. Of these, the

AC and the QC payment standard matched for 543 of the households (82 percent), compared to

75 percent in FY 2014. Also, 73 (11 percent) of the households had payment standards that did

not fall within the 90 percent to 110 percent FMR band. Therefore, a total of 11 percent of the

PHA-administered Section 8 Voucher households included in the FY 2015 study did not meet

HUD’s payment standard requirements, which indicates a 3 percent decrease from FY 2014.

Exhibit IV29d summarizes the results from the FY 2014 and FY 2015 payment standard analysis.

HUDQC Final Report for FY 2015

IV-39

Exhibit IV-29d

Comparison of the FY 2014 to FY 2015 Payment Standard Analysis

FY 2014

FY 2015

Percent of

Percent of

Characteristic

Number

Total Cases

Number

Total Cases

PHA-administered Section 8 Voucher sample

782

100%

791

100%

Households where QC payment standard was deter-

mined

670

86%

661

82%

Households where the AC and QC payment standard

did not match

171

26%

118

18%

Households where the AC payment standard did not

meet the 90% to 110% FMR threshold

91

14%

73

11%

Households that were not exempt from the

90% to 110% FMR threshold and did not meet HUD’s

91

14%

73

11%

payment standard requirements

I. PIC/TRACS Analysis

A special analysis was conducted to determine whether cases for which Form HUD-50058/50059

data had been submitted to HUD were more or less likely to have errors than those for which data

had not been submitted. These households were compared using two methods. The first was based

on whether household identifiers used by ICF were present in the PIC/TRACS data for the selected

households (i.e., social security number, name, and date of birth). The second method of comparison

examined whether the specific Form HUD-50058/50059 certification or recertification action used

by ICF for the selected households was present in the PIC/TRACS data.

In previous years, HUD provided PIC/TRACS head of household data for only those households in

the HUDQC Study. For the second study in a row, ICF requested PIC/TRACS data for all assisted

households with transaction data present within the FY 2015 study period in November 2015. These

data were used for both household sampling and the PIC/TRACS analysis. Due to nonstandard

recertification cycles permitted by various MTW PHAs, the PIC/TRACS data for MTW households

were not requested. Certain MTW PHAs selected for the FY 2015 study allow tenants to have

biennial or triennial recertifications, so those tenants would not have been included on the data file

if included in the request. For this reason, 2,211 of the 2,400 households (92 percent) were included

in the PIC/TRACS analysis (i.e., analytical sample).

Matching the HUDQC Study to the PIC/TRACS Data. Head of household data were provided

for all actions and updates from November 1, 2014, to October 31, 2015, resulting in multiple

observations per household. The households included in the HUDQC Study were matched

against these PIC/TRACS data using identifying information (a combination of the social security

number, name, and date of birth) for each head of household in the analytical sample. Using these

criteria, PIC records were found for just under 100 percent of households in PHA-administered

projects (99.9 percent), while TRACS records were found for exactly 100 percent of households

in Owner-administered projects. However, due to the use of PIC/TRACS data for household

sample selection, a near-100-percent match rate was anticipated. Figure IV-4 shows the change in

percentage of households in which PIC/TRACS was present over time, as identified in the HUDQC

IV-40

August 31, 2016

Study, beginning in FY 2005. PHA-administered percentages have increased since FY 2005, while

Owner-administered percentages have remained fairly steady over time.

In addition to the match described above, ICF further compared household data obtained during

the HUDQC study to PIC/TRACS data with the following certification information: program type,

type of action, and effective date. If duplicate observations for each head of household remained,

the transaction with the closest PIC/TRACS update date following the certification effective date

was selected. Using this more explicit matching technique and including the additional certification

elements, 2,124 of the 2,211 QC households in the PIC/TRACS analysis, or about 96 percent,

were fully represented by both head of household identifying information and certification data.

This matching rate was a slight increase from the previous rate of 95 percent in FY 2014 and a

decrease from the 98 percent found in FY 2013 and FY 2012. The current match rate of 96 percent

is still a substantial increase from rates in FY 2011 and FY 2010, in which 70 and 71 percent

of households, respectively, were fully represented by a match on both identifying information

and certification data.

In past years, most of the PIC/TRACS analysis was based on the broader match using identifying

information to maintain consistency with past years. However, with the new PIC/TRACS data request

process and nearly all households matching on identifying information, the PIC/TRACS analysis

presented in this report is based on the more explicit matching technique. Using these criteria, PIC

action records were found for 95 percent of the households in PHA-administered projects, while

TRACS action records were found for 97 percent of the households in Owner-administered projects.

Of the 2,211 households in this analysis, 2,124 households (or 96 percent) were successfully matched

with PIC/TRACS.

Figure IV-4

PIC/TRACS Data Present by Program Type for

All Households Over Time

HUDQC Final Report for FY 2015

IV-41

Analysis was conducted to compare the average dollars in Gross Rent Error for households that

had the QC selected action in PIC/TRACS (i.e., matching action type and effective date) with those

that did not. Exhibit IV-30a provides the percentage of households in each of the three program

types by whether or not the action data for the household were available in PIC/TRACS (present or

absent), and the average dollars in gross error based on all households in the study. Exhibit IV-30b

provides the same information, but only for those households that had rent error. Exhibits IV-30a

and IV-30b illustrate that the rate for which PIC/TRACS action data were present was reasonably

comparable between all households and for only those households with rent error.

Exhibit IV-30a

PIC/TRACS Data by Program Type and Average Gross Dollars in Error for All Households

PIC/TRACS Action Present

PIC/TRACS Action Absent

Percentage of

Average Dollars

Percentage of

Average Dollars

Administration Type

Households

in Error

Households

in Error

Public Housing

95.7%

$11.83

4.3%

$68.04

PHA-administered Section 8

95.5%

$13.47

4.5%

$31.15

Total PHA-administered

95.6%

$12.93

4.4%

$42.99

Total Owner-administered

97.3%

$9.21

2.7%

$7.89

Total

96.1%

$11.74

3.9%

$35.24

As presented in Exhibit IV-30b, the average dollars in Gross Rent Error for PHA-administered

projects were higher for households in error when PIC/TRACS action data were absent ($90)

than when PIC/TRACS action data were present ($47). More specifically, the largest difference in

average gross error dollars was found for Public Housing households, with and without PIC/TRACS

action data present ($51 and $192, respectively). However, because the number of households with

actions absent from PIC/TRACS is relatively low, these estimates are less reliable and more volatile

from year to year. In FY 2015, 96 percent of Owner-administered households in error had PIC/

TRACS data present. This rate is lower than in recent years, when the rate of Owner-administered

households in error with PIC/TRACS data present ranged from 97 percent to 100 percent between

FY 2010 and FY 2014.

Exhibit IV-30b

PIC/TRACS Data by Program Type and Average Gross Dollars in Error for Households in Error

PIC/TRACS Action Present

PIC/TRACS Action Absent

Percentage of

Average Dollars

Percentage of

Average Dollars

Administration Type

Households

in Error

Households

in Error

Public Housing

93.6%

$50.59

6.4%

$192.07

PHA-administered Section 8

93.1%

$48.89

6.9%

$71.35

Total PHA-administered

93.2%

$49.39

6.8%

$104.84

Total Owner-administered

96.0%

$41.61

4.0%

$23.89

Total

94.0%

$47.19

6.0%

$89.79

Exhibit IV-30c presents the percentage of households and average dollars in error for households

action matched/not action matched with PIC/TRACS by payment type. Although the percentages

of underpayment, overpayment, and proper payment are similar, both where PIC/TRACS action

IV-42

August 31, 2016

was present and where it was absent, there was a large difference in overpayment amounts

(e.g., $28 average overpayment error when PIC/TRACS action data could be matched, compared

to $118 overpayment error when PIC/TRACS action data were not matched). Interestingly, for

households with underpayments, average dollar in error was slightly less when PIC/TRACS action

was absent than when it was present (e.g., $66 compared to $63). However, because there are fewer

households in which PIC/TRACS data were absent, the average dollars in error amounts can vary

substantially from year to year.

Exhibit IV-30c

Average Gross Dollars in Error by Payment Type and PIC/TRACS Data

PIC/TRACS Action Present

PIC/TRACS Action Absent

Percentage of

Average Dollars

Percentage of

Average Dollars

Payment Type

Households

in Error*

Households

in Error*

Underpayment

94.0%

$65.54

6.0%

$63.26

Overpayment

96.8%

$27.67

3.2%

$118.13

Proper payment

94.0%

$0

6.0%

$0

Total

96.1%

$11.74

3.9%

$35.24

*Average dollar error per under- and overpayment subgroups.

Exhibit IV-30d examines net and gross errors by program type and whether there was a PIC/

TRACS action match. This exhibit illustrates the importance of reviewing net error and gross error

separately, as their average dollar errors are substantially different.

Exhibit IV-30d

Average Net and Gross Dollars in Error by Administration Type and

PIC/TRACS Data for All Households

Average Net Rent Error

Average Gross Rent Error

PIC/TRACS

PIC/TRACS

PIC/TRACS

PIC/TRACS

Administration Type

Action Present

Action Absent

Action Present

Action Absent

Public Housing

-$3.63

$47.89

$11.83

$68.04

PHA-administered Section 8

-$6.47

-$5.31

$13.47

$31.15

Total PHA-administered

-$5.54

$11.78

$12.93

$42.99

Total Owner-administered

-$4.06

$1.87

$9.21

$7.89

Total

-$5.07

$9.59

$11.74

$35.24

To further illuminate details of those households in which PIC/TRACS data matched on specific

study effective date and type of action (2,124 of 2,211 households), an additional analysis was

conducted to determine whether certain key variables matched. The key variables included gross

income, net income, total tenant payment, and tenant rent. Exhibit IV30e provides the percentage

of households in which the data gathered through the QC process matched those in PIC/TRACS.

HUDQC Final Report for FY 2015

IV-43

Exhibit IV-30e

Percentage of Matched and Non-Matched Dollar Amounts for Key Variables

Matching Variables From the Form HUD-50058/50059 and PIC/TRACS

Gross Income

Net Income

Total Tenant Payment

Tenant Rent

Match Status

PIC

TRACS

PIC

TRACS

PIC

TRACS

PIC

TRACS

No Match

1.7%

2.3%

2.0%

3.1%

2.6%

11.5%

10.9%

30.7%

Match

98.3%

97.7%

98.0%

96.9%

97.4%

88.5%

89.1%

69.3%

Total

100%

100%

100%

100%

100%

100%

100%

100%

J. Project Staff Questionnaire Analysis

The purpose of the Project Staff Questionnaire (PSQ), a self-administered, Web-based questionnaire,

was to obtain project-level information concerning the characteristics and processes that enable

PHA/project  staff  to  precisely  and  accurately  calculate  rent  during  certification  transactions,

including both initial/move-in certifications and annual recertifications. PHA/project staff identified

as the point of contact for the FY 2015 study were surveyed on topics related to PHA/project

characteristics, certification staff training and development, and performance management during

the study period of November 1, 2014, to October 31, 2015. The results were analyzed separately

for the three major program types: Public Housing, PHA-administered Section 8, and Owner-

administered programs.

A brief summary of the key findings from this analysis follows. A more detailed summary of the

PSQ information can be found in  Appendix E.

PHA/Project Characteristics. The PSQ surveyed respondents on PHA/project characteristics that

may help explain differences in error rates. Questions in this section included questions about

the number and types of staff, number of project units/tenants and certifications conducted, staff

work experience, certification staff work assignments and workload, and staff use of software and

computer technology. The findings of this section show the following:

● PHAs/projects that answered the survey based on project-specific information reported that

they employed an average of 11 staff members and served 1,238 households during the

study period. It is worth noting that each of the program types differed markedly in average

size. PHA-administered Section 8 projects averaged 26 employees and administered rental

assistance to 4,199 households, Public Housing projects had an average of 9 employees

and served 470 units on average, and Owner-administered projects averaged 4 employees

with 130 rental assistance units. Compared with FY 2014, the ratio of households to a

single staff member decreased slightly.

● PHAs/projects reported an average of 8 certification staff members on site, with an average

of 7 having more than 1 year of certification experience at the project. Thirty-one percent

of PHAs/projects in the study had at least 1 certification staff member stop working on

certification activities at the project during the study period. On average, PHAs/projects

that experienced turnover had 3 certification staff leave the PHA/project during the study

period.

IV-44

August 31, 2016

● The  most  frequently  used  methods  for  assigning  cases  to  certification  staff  were  by

transaction type (e.g., initial certifications, annual recertifications, transfers, and interims)

and by random assignment based on staff availability (19 percent and 16 percent,

respectively). Compared with FY 2014, more PHAs/projects reported that their staff had

a manageable workload, while fewer PHAs/projects reported a high workload. However,

answers varied by program type. PHA-administered Section 8 projects were most likely to

report a high workload (59 percent), while Owner-administered projects were most likely

to report a steady workload (66 percent). Less than 1 percent of respondents in all program

types reported that their certification staff had a low workload.

● Eighty-nine percent of all PHAs/projects responded that they did not outsource certification

activities to outside organizations during the study period. PHAs/projects were equally

likely to contract out certification activities to private companies and nonprofit organizations

(4 percent each), and somewhat less likely to choose a government organization (3 percent).

● Automated systems and computer software continued to play an increasingly integral

part in the daily tasks of PHAs/projects. PHAs/projects used computers most frequently

to print letters to tenants (98 percent), print Form HUD-50058/50059 (97 percent),

calculate income, expenses, and allowances (96 percent), and record tenant demographics

(92 percent). Overall, rates of computer usage remained stable since FY 2014. The largest

increase was found in keeping track of pending verification documents (from 59 percent to

62 percent), and the largest decrease was found in keeping other types of statistics (from

72 percent to 67 percent).

● During the study period, almost all PHAs/projects used computer software when performing

rent calculations (96 percent). Additionally, a majority of PHAs/projects used computer

software for other administrative tasks, including storing household data from previous

Form HUD-50058/50059 (96 percent), submitting data to the Public and Indian Housing

(PIH)  Information  Center/Tenant  Rental Assistance  Certification  System  (PIC/TRACS)

(92 percent), and annualizing income or expenses (91 percent).

Training and Development.  The PSQ collected information on an array of topics regarding training

and development provided to both new and experienced certification staff, including implementation

of rent calculation policies, certification staff work practices, and the nature and extent of rent

calculation training.

● Overall, 59 percent of PHAs/projects in FY 2015 had a training department or staff trainer

for certification staff, a decrease from FY 2014 (67 percent). By program type, 53 percent

of Public Housing sites reported having a training department or staff trainer, compared to

50 percent of PHA-administered Section 8 projects and 71 percent of Owner-administered

projects. On average, each new certification staff member received 49 hours of training

prior to performing rent calculations unassisted (the lowest in the last 6 years). In addition,

50 training hours were provided to new staff in FY 2014, 55 hours in FY 2013, 65 hours in

FY 2012, and an average of 89 hours of training in FY 2011.28

28  Training hour averages for FY 2012 and FY 2011 were calculated as the average for new (re)

certification staff and reassigned staff because of a change in the question for FY 2013.

HUDQC Final Report for FY 2015

IV-45

● Most PHAs/projects trained new certification staff members by providing them with policies

and procedural guides to read independently and by having them shadow or receive mentoring

from more experienced staff members (96 percent and 94 percent, respectively). During

the study period, PHAs/projects reported that a typical new certification employee spent

an average of 36 hours self-training with manuals and 127 hours shadowing experienced

staff. Additionally, the training hours dedicated to mentorship and classroom-style training

administered by an outside organization increased substantially since FY 2014.

● More than 90 percent of PHAs/projects trained new certification staff in calculating fixed

income sources, earned income sources, deductions, and/or assets, as well as in Enterprise

Income Verification (EIV) reports and EIV security. The average number of training hours

spent on each topic area per new staff member during the study period was about evenly

distributed among the topics listed in the PSQ, ranging from 24 to 36 hours for each topic

area. This was a substantial increase from FY 2014, when between 16 and 25 training hours

were spent on each topic area.

● The vast majority of PHAs/projects trained experienced staff by providing policies

and procedural guides to read (86 percent), while more than half of PHAs/projects had

experienced staff attend classroom training administered by an outside organization

(52 percent) and/or shadow more experienced staff (55 percent). A typical experienced

certification staff member in the Public Housing program spent an average of 15 hours

on training on the job with other experienced staff, compared to an average of 47 hours in

Owner-administered projects and 79 hours in PHA-administered Section 8 projects.

● PHAs/projects  most  frequently  endorsed  training  an  experienced  certification  staff

member on the topics of EIV reports and EIV security as well as calculating deductions

(81 percent and 66 percent, respectively). It was reported that the most training hours were

spent on entering Form HUD-50058/50059 (10 hours), while the fewest training hours

were spent on interviewing tenants (7 hours). On average, an experienced certification staff

member received approximately 25 fewer training hours for each topic than a new certification

staff member. This is a substantial increase from FY 2014, when experienced staff typically

received 13 fewer hours of training per topic area than new staff. Furthermore, in FY 2013,

experienced staff received only 7 fewer training hours than new certification staff.

● Twenty-seven percent of PHAs/projects typically implemented a new rent calculation

policy immediately following the issuance of a PIH Notice or Housing Notice, and

44 percent implemented a new rent calculation policy between 1 and 30 days after the

issuance of the notice. In January 2013, PIH Notice 2013-03 (HA)29 was issued, affecting

Public Housing, Moderate Rehabilitation, and Housing Choice Voucher programs. These

programs  reported  that  the  primary  methods  used  to  notify  certification  staff  about  the

policy change were discussing the policy with staff in a meeting and providing a paper

29   The PIH Notice 2013-03 (HA) allows PHAs to verify income using actual past income and allows

households with less than $5,000 in assets to self-certify their asset amount and asset income amount. It

also allows PHAs to conduct streamlined re-examinations for elderly/disabled families with fixed incomes

and to establish a payment standard of no more than 120 percent of Fair Market Rent as a reasonable

accommodation.

IV-46

August 31, 2016

or electronic copy of the notice (62 percent and 52 percent, respectively). Twenty-two

percent of Public Housing respondents stated that there was no discussion of this policy

with staff, and that staff did not receive a notice of the policy, compared to only 3 percent

of PHA-administered Section 8 projects.

● PHAs/projects overwhelmingly rated their certification staff as having average or above

average work behaviors during the study period. Ninety-eight percent of PHAs/projects

rated  their  certification  staff  as  either  organized  or  very  organized  when  working  on

certification activities in FY 2015. Similarly, 96 percent of PHAs/projects indicated that

their staff had either good or very good time management skills, and 98 percent reported

that  their  staff  paid  some  or  a  lot  of  attention  to  detail  while  working  on  certification

activities. Additionally, compared to FY 2014, PHAs/projects in FY 2015 were more likely

to indicate that their staff was very organized, had very good time management skills, and

paid a lot of attention to detail.

Performance Management. The PSQ also collected information regarding performance

management of certification activities and QC reviews. Questions addressed the timing of reviews,

methods used to select cases for review, type of information reviewed for QC, prevalence of various

types of rent calculation errors, and performance feedback methods and timing.

● Eighty-nine percent of PHAs/projects reviewed move-in and annual certifications as a QC

measure. On average, these PHAs/projects reviewed 47 percent of certifications. PHAs/

projects most frequently selected files randomly for QC review and conducted the reviews

prior to Form HUD-50058/50059 approval (79 percent and 27 percent, respectively).

● Sixty-eight percent of PHAs/projects had a dedicated QC staff to monitor tenant files and

certification activities, a decrease since FY 2014 (74 percent). Most often, PHAs/projects

had a team leader or supervisor conducting QC checks (83 percent), who conducted an

average of 266 file reviews during the study period.

● Twenty-six percent of PHAs/projects reported that they reviewed not only move-in

certifications and annual recertifications, but all tenant files during QC reviews. By program

type, 23 percent of PHA-administered Section 8 projects selected all cases for QC review,

compared to 27 percent for Public Housing and 29 percent of Owner-administered sites.

● Among PHAs/projects that conducted dedicated QC reviews, at least 88 percent had

reviewers check tenant files for correct income calculation, the presence of verification

documents, and/or proper core household documentation. Five percent or less of PHAs/

projects reported that verification, income calculation, expense calculation, or other errors

were made often or very often by certification staff, a decrease since FY 2014. Of all rent

calculation  errors,  late  annual  recertifications  were  reported  as  occurring  often  or  very

often at the highest rate (5 percent of PHAs/projects).

● In order to provide performance feedback to staff regarding errors found during the QC

process, PHAs/projects provided monthly rent calculation performance feedback to staff

(25  percent)  and  had  one-on-one  conversations  with  staff  to  discuss  QC  findings

(73 percent). Additionally, PHAs/projects overwhelmingly required that the certification

staff member who made the error be responsible for making file corrections (83 percent).

HUDQC Final Report for FY 2015

IV-47

● Twenty-eight percent of PHAs/projects reported that they do not track any of the certification

errors listed in the PSQ. PHAs/projects that did track errors most frequently monitored

late annual recertification transactions (78 percent), errors in earned income calculations

(57 percent), errors in calculating assets (56 percent), and errors in calculating medical

expenses (54 percent).

● The majority of PHAs/projects had error mitigation strategies in place: 69 percent had

a  formal  or  informal  goal-setting  process  related  to  rent  calculations  for  certification

staff, and 73 percent required that certification staff review a household’s previous Form

HUD-50058/50059 prior to beginning a new certification transaction.

K. Multivariate Analysis

The FY 2015 HUDQC multivariate modeling followed a similar conceptual approach to the one

used in previous years by analyzing variables representing project characteristics, project practices,

tenant characteristics, and project-caused errors. In previous years, we suggested a two-part

modeling approach consisting of examining the relationships of the variables affecting (1) gross

error, underpayment, and overpayment and (2) project-caused errors. However, in reality we were

more interested in an overall conceptual model that would examine the impact of variables through

the system in which we believe it interacts. As such, the conceptual model we currently propose is

presented in Figure IV-5.

Figure IV-5

Conceptual Model Used in Multivariate Analysis

Our approach to constructing the analytic model followed a pattern that was similar to the model

used in the past by initially examining the relationship between individual variables and the identified

outcomes (i.e., project-caused errors, gross rent error, underpayment error, and overpayment error).

The benefit in creating a single model is that we can examine the impacts of the project and household

characteristics in conjunction with both the probability and direction of rent error and the intensity

(amount) of gross rent error concurrently with the impact of the household and project characteristics

on project-caused errors. (See  Appendix F for more information on the multivariate analysis.)

Project-Caused Errors. Project-caused errors continue to represent the largest effects in measuring

rent error. Overall, two indicators were salient to the gross rent error (continuous), subsidy

underpayment (dichotomous) and subsidy overpayment (dichotomous). Applications that had a

transcription error led to a higher amount of gross rent error and were more likely to demonstrate

IV-48

August 31, 2016

subsidy overpayment error and subsidy underpayment error. Additionally, overdue recertification

errors showed an increase in both gross rent error and the likelihood of subsidy underpayment.

Given the high correlation of project-caused errors with improper rent payment, high priority should

be placed on reducing project-caused errors. We also found that transcription error was significantly

affected by both financial and membership household characteristics. Higher annual income, more

allowances, more income sources, and being an older head of household had an increased likelihood

of transcription error, while fewer expenses and household characteristics (e.g., dependents, minor

children, and adults over age 62) decreased the likelihood of a transcription error. Additionally, if the

project was Owner-administered there was a lesser likelihood of a transcription error. Alternatively,

overdue recertification errors increased slighted as the number of experienced, certified staff on a

project increased.

Household Characteristics. Household background variables were strong predictors of gross rent

error  and  subsidy  overpayment  and  underpayment.  Significant  relationships  between  a  few  of

the financial indicators directly affected gross rent error and the underpayment and overpayment

subsidies.  Specifically,  a  higher  household  annual  income  increased  both  the  amount  of  gross

rent error and the likelihood of subsidy underpayment. Alternatively, our results indicated that

an increase in the total amount received in allowances (e.g., medical, child care, and disability)

reduced the amount of gross rent error. We found a more circumspect result with household rent.

An increase in household rent indicated an increase in the amount of gross rent error; however, the

same increase in household rent also demonstrated a lower likelihood of subsidy underpayment

but did not have a significant effect on subsidy overpayment. This finding would suggest that as

the amount of rent on applications increased, the likelihood of underpayment decreased, but when

the subsidy underpayment did occur the amount of gross rent error increased with respect to the

amount of rent.

The results also demonstrated that a couple of the household membership variables were significantly

related to the subsidy errors. Households with an individual over age 62 were less likely to have

subsidy underpayment. Alternatively, households reporting any dependents were more likely to

have received a subsidy overpayment.

Project Characteristics and Practices. The impact of project characteristics and project practices

on improper payments remained elusive within the current data analysis. As in previous years,

most key indicators of project resources, staff capacity, training, certification procedures, computer

application, and a broad array of quality control efforts were not found to be statistically significant,

and no substantial relationships were found with rent error measures.

Of the numerous project practices examined, two continued to be significant when examined using

a multilevel modeling approach. Similar to FY 2014 results, FY 2015 results showed a relationship

between EIV training for new staff and rent error. Additionally, we found that as the caseloads of

certified staff increased so did the likelihood of subsidy overpayment.

As was the case in FY 2013 and FY 2014, the project practices are much less predictive of rent error

than project-caused errors and household characteristics. Further, the project practices that emerged

from the modeling seem to be inconsistent in their relationship with rent errors (e.g., increased

training associated with increased rent error). Investigating these counterintuitive effects could

HUDQC Final Report for FY 2015

IV-49

help identify whether the effect is truly associated with rent error or whether there are underlying factors that are confounded with rent error. This will improve clarity on specific challenges facing

certification staff that lead to processing errors and payment errors.

This new conceptual and modeling approach to examining the impact of household characteristics,

project practices and characteristics, and project-caused errors on the gross amount of rent error and

overpayment and underpayment of rent subsidies continues to enhance our understanding of these

rent errors. While our results are similar to prior studies, concurrent modeling of these outcomes

and indicators has changed our understanding of their effects on amount and direction of rent error.

However, our findings suggest that many of the household variables affect the amount and direction

of the rent error through the project-caused transcription and overdue recertification errors. Using

these characteristics (e.g., large sources of income, large households) to identify cases that could be

at risk for improper payment in advance of the (re)certification process could allow for increased

quality control of these cases, or the development of specific training around these at-risk units.

L. The 20 Largest PHAs Study

The 20 Largest PHAs Study aims to provide additional information about the 20 largest PHAs.

Included in this study were the 18 largest PHAs and the 2 largest state PHAs in the project-level

sample selected for the HUDQC Study. There were 32 households selected from most PHAs, but

two PHAs had more. Specifically, NY005, which had 189 households selected, and IL002, which

had 40 selected households. The study of the 20 largest PHAs ultimately included a total of 806

households.

Most PHAs represented both Public Housing and PHA-administered Section 8 households. MI901,

NY110, CA063, and NY904 only represented PHA-administered Section 8 households. Weights

for the 20 Largest PHAs Study were not calculated, and as a result all data presented in the exhibits

in this section that pertain to the 20 largest PHAs are not weighted.30

Administrative Error in the 20 Largest PHAs.   Exhibit IV-31a provides the percentage of

households that had overdue recertification and transcription errors and the percentage of income and

expense items that were verified by PHA staff using both written third-party verification only and

verbal or written third-party verification, documentation, or EIV/UIV. These types of administrative

errors were examined because they are typically associated with overall Gross Rent Error and Net

Rent Error.

●  Compared to all HUDQC Study PHAs selected, the 20 largest PHAs had a higher percentage

of overdue recertification errors (2 percent and 4 percent, respectively) and a slightly lower

transcription error rate (43 percent and 37 percent, respectively).

●  Overdue recertification errors had low rates in most PHAs.

●  While most of the 20 largest PHAs had slightly lower transcription error percentages than

the HUDQC Study mean of 37 percent, NY005 had a markedly higher transcription error

rate (68 percent).

30  For a more detailed discussion regarding weighting for the 20 Largest PHAs Study, please refer to

Appendix B.

IV-50

August 31, 2016

● The 20 largest PHAs verified items using third-party verbal or in writing, documentation, or

EIV/UIV at a slightly higher rate than the HUDQC Study overall, at 88 percent compared

to 87 percent.

● The 20 largest PHAs verified items using only third-party in-writing verification, slightly

less than the PHAs in the HUDQC Study overall (10 percent and 14 percent, respectively).

● DC001  verified  items  using  only  third-party  in-writing  verification  at  the  greatest  rate

(26 percent), while IL025 and PA002 used this method rarely (a rate of 2 percent for each).

Exhibit IV-31a

Administrative Errors in the 20 Largest PHAs

Percentage of Verified Items

Percentage

Third-Party Verbal

of Overdue

Percentage of

or In Writing,

Number of

Recertification

Transcription

Documentation,

Third-Party

PHA

Households

Error*

Error

or EIV/UIV

In Writing

CA002

32

—

50%

88%

0%

CA004

32

3%

34%

83%

6%

CA063

32

—

34%

91%

9%

DC001

32

6%

38%

92%

26%

IL002

40

5%

35%

89%

11%

IL025

32

—

47%

89%

2%

MA002

32

—

44%

88%

5%

MD002

32

6%

34%

95%

10%

MI901

32

3%

9%

85%

20%

NY005

189

5%

68%

83%

17%

NY110

32

6%

31%

96%

4%

NY904

32

3%

53%

88%

8%

OH001

32

6%

44%

92%

15%

OH003

32

3%

16%

78%

14%

OH004

32

—

31%

84%

11%

PA002

33

—

30%

87%

2%

TX005

32

3%

31%

93%

3%

TX009

32

3%

31%

89%

4%

WA001

32

—

38%

84%

14%

WA002

32

—

41%

76%

13%

Total

806

4%

37%

88%

10%

QC Study Total

2,400

2%

43%

87%

14%

Note: Data in this exhibit for   the 20 largest PHAs are not weighted (see   Appendix B), although HUDQC Study total data are weighted.

*Those PHAs without a percentage in this column had no cases overdue for recertifications.

Payment Error. Exhibit IV-31b provides payment error information. This exhibit includes proper

payments, underpayments and overpayments of tenant rents, and the mean Gross Rent Errors

by PHA.

HUDQC Final Report for FY 2015

IV-51

● Compared to PHAs in the HUDQC Study as a whole, the 20 largest PHAs had a slightly

lower percentage of households with proper payments (73 percent and 70 percent,

respectively), as well as a higher average gross dollar error (about $20 for the 20 largest

PHAs versus about $13 for the HUDQC Study).

● The PHA with the highest percentage of proper payments was MI901, which had proper

payments for 91 percent of households.

● OH004 had the lowest average gross dollar error, at $2.03.

Exhibit IV-31b

Dollar Rent Errors in the 20 Largest PHAs*

Average Gross

PHA

Underpayment

Proper Payment

Overpayment

Dollar Error

CA002

25%

66%

9%

$14.78

CA004

13%

72%

16%

$22.81

CA063

16%

72%

13%

$5.69

DC001

19%

69%

13%

$8.66

IL002

23%

73%

5%

$26.42

IL025

25%

66%

9%

$14.62

MA002

13%

81%

6%

$19.63

MD002

16%

72%

13%

$24.94

MI901

9%

91%

0%

$3.78

NY005

18%

62%

20%

$28.96

NY110

13%

84%

3%

$16.84

NY904

6%

69%

25%

$57.34

OH001

13%

84%

3%

$7.72

OH003

3%

88%

9%

$7.87

OH004

16%

84%

0%

$2.03

PA002

12%

36%

52%

$10.91

TX005

13%

75%

13%

$19.47

TX009

13%

81%

6%

$21.19

WA001

19%

66%

16%

$24.25

WA002

6%

50%

44%

$18.50

Total/Average

15%

70%

15%

$20.07

QC Study Total/Average

14%

73%

13%

$13.39

Note: Data in this exhibit for the 20 largest PHAs are not weighted (see Appendix B), although HUDQC Study total data are weighted.

*Row totals may not add up 100 percent to due to rounding.

IV-52

August 31, 2016

In FY 2015, a smaller proportion of proper payments seemed to lead to a higher average gross

dollar error within the PHA.

● The PHA with the lowest percentage of proper payments was PA006, with a 33 percent rate

of proper payment. PA002 had an Average Gross Dollar error rate of $11. This indicates

that although there was a large rate of mispayment within PA002, the amount of dollars in

errors was relatively low for most tenants.

● Alternatively, the PHA with the lowest average gross dollar amount, OH004 ($2.03), had a

higher rate of proper payments than the average across the 20 largest PHAs, at 84 percent.

These results imply that when the percentage of proper payments increases, the average gross dollar

error may decrease. Consequently, policies that increase proper payment rates may have some effect

on decreasing rent errors (and vice versa). These seemingly related problems may sometimes require

different approaches targeted to specific PHAs.

HUDQC Final Report for FY 2015

IV-53

V. Recommendations

The HUDQC Study was originally conducted in 2000 and has been conducted on an annual basis

since the FY 2003 review. A major goal of the study is to provide a snapshot of HUD national

improper payment error. However, another important objective is to understand the causes of error

and  identify  policies  that  reduce  improper  payments.  As  we  executed  the  study,  we  identified

general areas that could be improved in PHA/project rent calculation. We present these insights in

this chapter of the report. Section A outlines general policy recommendations that could potentially

reduce administrative error and tenant misreporting of income.

In addition to program recommendations, we examined how the QC studies can be improved. Each

year, changes and improvements are made in the execution of the study to achieve aims such as

increased efficiency, reduced burden on project staff and households, and a better understanding of

program practices and discretionary policies. Section B provides recommendations for improving

the data collection process and the quality of the data used in the analysis of improper payments.

A. Recommended Policy Actions

It should be noted that the study was not designed to provide recommendations regarding program

policies and procedures. However, findings from the study suggest general actions that should be

continued or policies that should be considered to maintain or improve PHA/project performance

in rent determination. Below we present recommendations that may improve administrative error

rates in HUD programs, based on insights we have gathered during this and previous studies.

1.  Continue Requiring the Use of EIV Reports. HUD should continue requiring the use of

Enterprise Income Verification (EIV) information in the process of rent determination. Data

systematically collected from the National Database of New Hires and the Social Security

Administration provide a strong method of identifying specific sources of income information.

The study shows that the majority of subsidy errors are associated with earned income.

HUD may also want to consider forming relationships with State programs, organizations,

and companies to collect other data not currently captured by the EIV system. Although

EIV provides a uniform and efficient method of verifying income sources that lessens the

burden on program administrators, caution must be exercised when using information from

the system. The data are extremely helpful in identifying unreported sources of income, but

they are not current and sometimes contain errors (including instances of identity theft and

incorrect identification of disability status). HUD’s EIV requirement should be coupled with

policies aimed at addressing the challenges of using EIV for verification.

2.  Perform Onsite Review of Rent Calculation. HUD should continue onsite monitoring of

program administration, and PHA/Owners should be held accountable for implementing HUD

regulations and calculating rent accurately. Onsite monitoring that includes reviews at both the

local and Federal levels is essential to improving accountability. PHA/Owners with excessive

errors should be required to develop corrective action plans and show improvement within

specified time periods. Improved HUD monitoring was likely a key factor in reducing subsidy

error between the 2000 study and the current study.

We recommend that HUD require PHA/Owners to perform their own QC reviews on income

determinations and rent calculations. Agencies that have aggressively sought to improve

HUDQC Final Report for FY 2015

V-1

V. Recommendations

the performance of their programs have demonstrated success in this area, and one of

the most frequently used error-reduction strategies includes the establishment of internal

QC review procedures. Based on the Project Staff Questionnaire (PSQ) survey, it appears

that programs that conduct QC on all their transactions have a significantly lower rent

determination error rate than programs that do not perform QC on all their files. Of course,

a comprehensive approach may not be feasible, given limited staffing resources, but even a

review of a small percentage of transactions may be beneficial in supporting the reduction

of rent determination error.

In addition to internal agency reviews, HUD regional offices can support field offices by

conducting a secondary review of transactions. This review would provide HUD Federal

staff with more on-the-ground insights into the issues and challenges faced by local program

administrators. In addition, this approach would demonstrate HUD’s concern regarding

program integrity and improper payments, thereby focusing PHA/Owner attention on

accurately determining tenant income and rent.

3.  Continue to Streamline the Program Requirements. Continue to simplify Federal laws,

regulations, and HUD requirements to the extent possible. The new regulations outlined in

Notice PIH 2016-05 provide much needed clarification on previous regulations, such as the

time period associated with the Earned Income Disregard calculations. The implementation

of triennial certification for fixed income households will save time for PHA staff, allowing

them to spend more time conducting required reexaminations, following up on suspected

cases of fraud, and conducting more internal reviews of tenant files. Office of Housing

should implement similar changes.

4.  Create an Online Community to Share Best Practices and Tools. HUD should provide

PHA/Owners with an online venue to support the sharing of best practices for its assisted

housing programs. A Web-based resource could facilitate communication between HUD

and program administrators regarding identifying ways to improve and address challenges

related to proposed policies. Comprehensive supporting documents, including forms for

interviewing tenants, obtaining verification, and determining rent, could be posted to the site

for downloading. Manuals and training materials describing how to implement requirements

and accurately calculate rent could also be available electronically, with online webcasts

providing an additional training resource for local program offices.

HUD experts and local housing staff should be given the opportunity to work together to

develop tools and systems that reduce rent error. Many local PHA/Owners have already

developed forms, training materials, manuals, automated systems, and monitoring processes

that enable them to provide accurate, efficient service to their tenants. HUD should create

a platform for organizations to learn from each other.

5.  Develop a technical assistance platform for rent calculation staff. HUD should develop a

technical assistance hotline for project/PHA staff to call about particularly difficult policies

related to completing (re)certifications. The assistance could be a Web-based support system,

where the PHA/project staff would be given a time estimate of when they will receive an

answer after submitting a question, or a live telephone system staffed by experienced HUD

policy advisors. This open dialogue would help avoid common errors that contribute to rent

error. Additionally, HUD should keep detailed records for each call to determine if there

V-2

August 31, 2016

V. Recommendations

were commonly asked questions, and provide written guidance on these issues to all PHA/

projects for future use.

6.  Develop a calculation worksheet for use in annualizing income. This electronic worksheet

would be programmed to calculate annual income to be used for rent calculation. It would

also serve as documentation of the methodology used by the PHA/project staff to annualize

income. A physical copy could be kept in the tenant file after it is dated and signed by the

certification staff. This would help to avoid calculation errors and thus reduce rent error.

B. Modifying the Quality Control Study

In addition to providing general program recommendations to improve error rates, we endeavored to

improve the HUDQC Study that provides the estimates of the error rates. The current methodology

used by ICF to conduct the HUDQC Study is based on meeting established study objectives and

builds on insights from previous studies. The following recommendations serve to expand the

utility of the data collected, support HUD’s research goals, and improve the overall efficiency of

ongoing QC studies.

7.  Continue to Measure Improper Payment Associated With Billing Error. HUD should

conduct billing error studies to obtain a more accurate assessment of improper payments.  In

the  FY 2014 HUD Agency Financial Report, billing error estimates are based on FY 2004

data for the Public Housing program and FY 2009 data for Owner-administered programs. In

FY 2015, ICF conducted billing studies for both the Office of Public and Indian Housing and

the Office of Housing. This work should continue to be conducted. Current error estimates

could be obtained by conducting primary data collection or by using statistical modeling

to update the existing data. However, an updated study will provide HUD with a better

assessment of billing error associated with rental assistance programs in order to understand

one of the main contributors to improper payments. The information from these billing

studies could strengthen financial management controls so that HUD can better detect and

prevent improper payments.

8.  Incorporate Additional Objectives in the HUDQC Study. Data collected through the

HUDQC Study provide details that are not available through other HUD sources (e.g., PIC/

TRACS) that could be used to track trends, such as the extent to which income and expense

items are verified or the number of sources of employment income for a particular household

or household member. Furthermore, because a statistically valid nationwide sample

of projects and households is created for the study, other HUD-related topics could be

investigated using the HUDQC Study’s research mechanisms and data collection processes.

The rental integrity monitoring (RIM) review validation, identified in the July 2013 issuance

of the HUD Research Roadmap for FY 2014–FY 2018, is a task that could be incorporated

into the HUDQC Study’s data collection process. Additional topics could include a review

of the changing demographics of HUD tenants, participant satisfaction surveys, and a more

in-depth review and evaluation of MTW programs.

9.  Continue to Conduct a Utility Allowance Comparison Study. In response to tightening

budgets and overall concerns with energy efficiency, HUD should undertake a study to

better understand utility costs and consumption in subsidized housing. ICF conducted this

work in FY 2015, and this work should continue. HUD should consider conducting an

HUDQC Final Report for FY 2015

V-3

in-depth quality control study of how utility allowance values are calculated and used in rent

calculation. This study could involve collecting data from utility companies regarding utility

use for a given fiscal year and comparing actual consumption with the utility allowance

subsidy calculated by program administrators.

The FY 2015 Utility Allowance Comparison Study found that current HUD Utility Schedule

Model (HUSM) users produce incorrect allowances via the tool. HUD should consider

making improvements to the HUSM tool to increase accuracy of data entry and calculated

utility allowances. Additionally, performing project-level reviews and approvals of utility

allowance levels would improve accountability in updating allowances in a timely manner

and would likely reduce subsidy error in utility allowances.

10. Learn More About PHA/Project Policies and Practices.  Each PHA establishes its own

policies, procedures, and forms for collecting information that is ultimately used to calculate

tenant rent. The differences in these practices should have some (possibly major) impact

on rent error, yet the analysis of the project practices and characteristics collected by the

PSQ does not demonstrate the expected impact. We recommend conducting focus groups,

interviews, and discussions with program administrators to identify additional PHA/project-

level factors that may impact error. This information could be used to revise the PSQ to

include questions focused on the specific practices expected to influence errors. The analysis

of more detailed, project-level data would assist in this process.

11. Continue to Perform the HUDQC Study. The HUDQC Study provides a consistent

ongoing method of monitoring, managing, and improving HUD rent determination processes.

The ongoing evaluation of HUD rental housing assistance programs is essential to program

management and improvement, and rigorous research is important for understanding how

well HUD programs are reaching their goals for the communities served. The primary

objective of the HUDQC Study is to measure rent calculation and improper payment error;

however, the study also gives HUD the opportunity to learn more about methods to reduce

rent calculation errors and better manage current and changing conditions at PHAs/projects.

Annual evaluations facilitate more accurate, cross-year comparisons of rent errors. They

also allow data collection and data analysis staff to develop specific expertise in HUD policy

areas, supporting the development of tailored solutions for improving data quality.

V-4

August 31, 2016

Appendix A: Rent Calculations

Appendix A: Rent Calculations

1.  Public Housing

a.  Obtain the Total Tenant Payment (TTP).

b.  Determine whether the family includes any ineligible noncitizens. IF YES, continue.

IF NO, go to d.

c.  Determine whether the family includes any citizens or eligible noncitizens. IF YES,

go to #6 (continuation of assistance). IF NO, go to #7 (temporary deferral).

MARKER (marks the return point after determining continuation of assistance or temporary

deferral status)

d.  Determine whether the tenant selected Flat Rent. IF NO, go to e. IF YES, the QC RENT

equals the Flat Rent. Go to g.

e.  Obtain the Utility Allowance.

f.  The amount of the tenant’s rent (QC RENT) is the lower of: a. (TTP) minus e. (Utility

Allowance), or the Flat Rent.1

g.  Determine whether any additional Moving to Work rent calculation policies apply.

IF YES, recalculate the tenant’s rent (QC RENT) according to the policy outlined in

the Public Housing Authority (PHA) Admissions and Continued Occupancy Policy

(ACOP), Administrative Plan, or Other Policy Guidance Documents; continue. IF

NO, continue.

h.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

2.  Section 8 Voucher Program

a.  Obtain TTP.

b.  Obtain the Gross Rent.

c.  Obtain Utility Allowance.

d.  Determine whether a. (TTP) is greater than b. (Gross Rent). IF YES, set TTP to Gross

Rent.  IF NO, TTP is equal to a.

e.  Obtain Payment Standard2 (the Payment Standard is based on the lower of the Unit

[actual] Bedroom Size and Family [eligible] Bedroom Size).

f.  Obtain the household’s Adjusted Monthly Income.

g.  Subtract e. (Payment Standard) from b. (Gross Rent). If the Payment Standard is higher

than the Gross Rent, use 0.

1   If there is no Flat Rent, the QC RENT will be the lower of the Ceiling Rent or a. (TTP) minus e. (Utility

Allowance) to determine the dollar amount of error. If there is also no Ceiling Rent, the QC RENT will be

a. (TTP) minus e. (Utility Allowance).

2   For Project-Based Vouchers, the Payment Standard equals the Gross Rent.

HUDQC Final Report for FY 2015

A-1

Appendix A: Rent Calculations

h.  Add d. (TTP) to g. (Gross Rent minus Payment Standard).

i.  Determine whether this is the initial occupancy for this dwelling unit. (Item 12b on the

Form HUD-50058 is yes). IF YES, continue.  IF NO, the Family Share is equal to h.

Go to l.

j.  Calculate 40 percent of f. (household’s Adjusted Monthly Income).

k.  Determine whether j. (40 percent of Adjusted Monthly Income) is equal to or greater

than h. (TTP plus Gross Rent minus Payment Standard). IF YES, the Family Share

equals h; continue. IF NO, procedural error. Family Share is equal to h; continue.

l.  Determine whether the family includes any ineligible noncitizens. IF YES, continue.

IF NO, go to n.

m.  Determine whether the family includes any citizens or eligible noncitizens. IF YES,

go to #6 (continuation of assistance).  IF NO, go to #7 (temporary deferral).

MARKER (marks the return point after determining continuation of assistance or temporary

deferral status)

n.  Subtract c. (Utility Allowance) from h. (Family Share). This is the QC RENT.

o.  Determine whether any additional Moving to Work rent calculation policies apply.

IF YES, recalculate the tenant’s rent (QC RENT) according to the policy outlined

in the PHA ACOP, Administrative Plan, or Other Policy Guidance Documents;

continue. IF NO, continue.

p.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

3.  Section 8 Enhanced Voucher

a.  Determine whether household is receiving an Enhanced Voucher. IF YES, continue.

IF NO, use #2 (the regular Section 8 Voucher formula).

b.  Obtain the TTP.

c.  Obtain the Gross Rent.

d.  Determine the lesser of b. (TTP) or c. (Gross Rent).

e.  Determine whether the family includes any ineligible noncitizens. IF YES, continue.

IF NO, go to g.

f.  Determine whether the family includes any citizens or eligible noncitizens. IF YES,

go to #6 (continuation of assistance).  IF NO, go to #7 (temporary deferral).

MARKER (marks the return point after determining continuation of assistance or temporary

deferral status)

g.  Obtain the Utility Allowance.

h.  Subtract g. (Utility Allowance) from d. (the lesser of TTP or Gross Rent). This is the

Family Rent to Owner (QC RENT).

A-2

August 31, 2016

Appendix A: Rent Calculations

i.  Determine whether any additional Moving to Work rent calculation policies apply.

IF YES,  recalculate the tenant’s rent (QC RENT) according to the policy outlined

in the PHA ACOP, Administrative Plan, or Other Policy Guidance Documents;

continue.  IF NO, continue.

j.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

4.  Project-Based Section 8, Section 202, Section 811, Section 8 Moderate Rehabilitation

a.  Obtain the Gross Rent (Gross Rent equals the Contract Rent plus the Utility Allowance).

b.  Obtain the TTP.

c.  Determine whether the family includes any ineligible noncitizens. IF YES, continue.

IF NO, go to e.

d.  Determine whether the family includes any citizens or eligible noncitizens. IF YES,

go to #6 (continuation of assistance).  IF NO, go to #7 (temporary deferral).

MARKER (marks the return point after determining continuation of assistance or temporary

deferral status)

e.  Obtain the Utility Allowance.

f.  Determine whether Subsidy Type on Form HUD-50059 is PRAC. IF NO, continue.

IF YES, go to h.

g.  Subtract e. (Utility Allowance) from b. (TTP) or a. (Gross Rent), whichever is lower.

This is the QC RENT. Go to i.

h.  Subtract e. (Utility Allowance) from b. (TTP). This is the QC RENT.

i.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

5.  Manufactured Home Space Rental for Section 8 Vouchers

a.  Obtain the Rent to Owner.

b.  Obtain the owner maintenance and management charges for the space.

c.  Obtain the Utility Allowance.

d.  Add together a. (Rent to Owner), b. (owner maintenance and management charges), and

c. (utility allowance). This is the Space Rent.

e.  Obtain the TTP.

f.  Obtain the Payment Standard.

g.  Subtract f. (Payment Standard) from d. (Space Rent). If Space Rent is less than the

Payment Standard, use 0.

h.  Add e. (TTP) to g. (the amount by which the Space Rent exceeds the Payment Standard).

This is the Family Share.

HUDQC Final Report for FY 2015

A-3

i.  Determine whether this is the initial occupancy for this dwelling unit. (Item 12b on

Form HUD-50058). IF YES, continue.  IF NO, the Family Share equals h. Go to m.

j.  Obtain the household’s Adjusted Monthly Income.

k.  Calculate 40 percent of the household’s Adjusted Monthly Income.

l.  Determine whether k. (40 percent of Adjusted Monthly Income) is equal to or greater than

h. (TTP plus Space Rent minus Payment Standard). IF YES, the Family Share is equal to

h.; go to m.  IF NO, procedural error. The family is not entitled to assistance in this unit.

m.  Determine whether the family includes any ineligible noncitizens. IF YES, continue.

IF NO, go to o.

n.  Determine whether the family includes any citizens or eligible noncitizens. IF YES,

go to #6 (continuation of assistance).  IF NO, go to #7 (temporary deferral).

MARKER (marks the return point after determining continuation of assistance or temporary

deferral status)

o.  Subtract c. (Utility Allowance) from h. (Family Share) to determine Tenant Rent to

Owner (QC RENT).

p.  Determine whether any additional Moving to Work rent calculation policies apply.

IF YES, recalculate the tenant’s rent (QC RENT) according to the policy outlined

in the PHA ACOP, Administrative Plan, or Other Policy Guidance Documents;

continue.  IF NO, continue.

q.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

Special Calculations for Household With Ineligible Noncitizens

6.  Continuation of Assistance

a.  Determine  whether  the  family  was  receiving  assistance  on  June  19,  1995.  IF  YES,

continue. IF NO, the FAMILY is eligible for prorated assistance; go to #8 (proration

formula for Public Housing).

b.  Determine whether the FAMILY head or spouse is a citizen or eligible noncitizen.

IF YES, continue.  IF NO, the FAMILY is eligible for prorated assistance;  go to #8

(proration formula for Public Housing).

c.  Determine whether the FAMILY includes any ineligible members other than the

head, spouse, and child or parent of the head or spouse. IF NO, continue.  IF YES,

the FAMILY is eligible for prorated assistance; go to #8 (proration formula for

Public Housing).

d.  Determine whether the FAMILY was granted continuation of assistance before November

29, 1996. IF YES, the FAMILY is eligible for full continuation of assistance.  Return

to MARKER for the appropriate program type.  IF NO, the FAMILY is eligible for

prorated assistance; go to #8 (proration formula for Public Housing).

A-4

August 31, 2016

7.  Temporary Deferral of Termination of Assistance

a.  Determine whether Temporary Deferral of Termination of Assistance has been granted.

IF YES, continue.  IF NO, go to c.

b.  Determine whether 18 months have passed since Temporary Deferral was granted.

IF YES, continue.  IF NO, the Family continues to be eligible for Temporary Deferral

of Termination of Assistance; return to MARKER for the appropriate program

type.

c.  Determine whether the FAMILY includes a refugee under Section 207 of the Immigration

and Naturalization Act or an individual seeking asylum under Section 208 of that Act.

IF NO, continue. IF YES, the Family is entitled to ongoing Deferral of Termination

of Assistance; go to MARKER for the appropriate program type.

d.  Determine whether the FAMILY was receiving assistance on June 19, 1995. IF NO,

continue. IF YES, the Family is eligible for Temporary Deferral of Termination of

Assistance; go to MARKER for the appropriate program type.

e.  Determine whether the FAMILY is exercising its hearing rights (waiting for a decision

from INS or PHA/Owner appeal). IF NO, continue. IF YES, go to MARKER for the

appropriate program type.

f.  Determine whether the PHA is making reasonable efforts to evict. IF YES, go

to MARKER for the appropriate program type.  IF NO, procedural  error,

HOUSEHOLD IS INELIGIBLE.

8.  Proration Formula for Public Housing

a.  Determine whether this is a Public Housing case. IF YES, continue.  IF NO, go to #9

(proration formula for all Section 8 programs).

b.  Determine the number of FAMILY members.

c.  Determine the number of eligible FAMILY members.

d.  Obtain the TTP.

e.  Obtain the 95th percentile of Gross Rents for similarly sized Public Housing units in

order to determine the Public Housing maximum rent.3

f.  Determine whether the Family pays a Flat Rent. IF YES, continue. IF NO, go to i.

g.  Obtain the Flat Rent.

h.  If g. (Flat Rent) is greater than or equal to e. (Maximum Rent), there is no prorated

rent. Use the Flat Rent; go to n. If g. (Flat Rent) is less than the e. (Maximum Rent),

subtract the Flat Rent from the Maximum Rent. This is the Family’s Maximum Subsidy.

Go to j.

i.  Subtract d. (TTP) from e. (Maximum Rent) to determine Maximum Subsidy.

3  If Maximum Rent is not available, Fair Market Rent is used as a substitution for Maximum Rent.

HUDQC Final Report for FY 2015

A-5

j.  Divide h. or i. (Maximum Subsidy) by b. (number of FAMILY members) and multiply

by c. (number of eligible FAMILY members) to determine the Eligible Subsidy for the

FAMILY.

k.  Subtract j. (Eligible Subsidy) from e. (Maximum Rent) to obtain the prorated TTP.

l.  Obtain the Utility Allowance.

m.  The amount of the tenant’s rent (QC RENT) is k. (prorated TTP) minus l. (Utility

Allowance).

n.  Determine whether any additional Moving to Work rent calculation policies apply.

IF YES, recalculate the tenant’s rent (QC RENT) according to the policy outlined

in the PHA ACOP, Administrative Plan, or Other Policy Guidance Documents;

continue. IF NO, continue.

o.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

9.  Proration Formula for All Section 8 Programs

a.    For Voucher Projects Only:  Obtain the Rent to Owner.

b.   Obtain the Utility Allowance.

c.   Obtain the Gross Rent.

Voucher: Gross Rent = Rent to Owner plus the Utility Allowance.

Owner-administered: Gross Rent = Contract Rent plus the Utility Allowance.

d.   Obtain the TTP.

e.    For Voucher Projects Only: Obtain the Payment Standard.

f.    Obtain the Housing Assistance Payment (HAP).

Owner-administered: HAP = Gross Rent minus TTP.

Voucher: HAP = Gross Rent or Payment Standard (whichever is less) minus the TTP.

Enhanced Voucher: HAP = Gross Rent minus the Payment Standard.

g.   Record the number of FAMILY members.

h.   Record the number of eligible FAMILY members.

i.    Divide f. (HAP) by g. (total number of FAMILY members), and then multiply the result

by h. (number of eligible FAMILY members) to obtain the prorated HAP.

j.   Determine if Manufactured Home Space Rental. IF NO, continue.  IF YES, return to

MARKER for the appropriate program type.

k.   Subtract i. (prorated HAP) from c. (Gross Rent) to obtain the prorated Family Share.

l.  Subtract b. (Utility Allowance) from k. (prorated Family Share) to determine the prorated

QC RENT.

A-6

August 31, 2016

m.   For PHA-administered Projects Only: Determine whether any additional Moving to

Work rent calculation policies apply. IF YES, recalculate the tenant’s prorated rent

(QC RENT) according to the policy outlined in the PHA ACOP, Administrative

Plan, or Other Policy Guidance Documents; continue.  IF NO,  continue.

n.  Determine whether the QC RENT equals the ACTUAL RENT. IF YES, no error.

IF NO, dollar error.

HUDQC Final Report for FY 2015

A-7

Appendix B: Weighting Procedure

Appendix B: Weighting Procedure

This appendix describes the procedures followed in weighting the project sample.

Study Population.  The universe of the Improper Payment for Quality Control for Rental Subsidy

Determination (HUDQC) Study included all projects and households located in the continental

United States, Alaska, Hawaii, and Puerto Rico. In fiscal year (FY) 2015, Moving to Work (MTW)

Public Housing Authorities (PHAs) were included in the study population.

The following programs were included in the sample:

● PHA-administered Public Housing (Public Housing)

● PHA-administered Section 8 (PHA-administered Section 8)

○ Moderate Rehabilitation

○ Housing Choice Voucher program

● Office of Housing-administered projects (Owner-administered)

○ Section 8 New Construction/Substantial Rehabilitation

○ Section 8 Loan Management

○ Section 8 Property Disposition

○ Section 202 Project Rental Assistance Contracts (PRAC)

○ Section 202/162 Project Assistance Contracts

○ Section 811 PRAC

The initial universe files used to draw the sample occasionally reflected out-of-date or incorrect

information, including out-of-scope projects such as demolished projects, projects undergoing

renovation, projects that were no longer assisted, projects that had merged or split, and other

special circumstances. Many of these projects were identified prior to drawing the sample. For

example, in FY 2015, at the request of HUD, projects newly converted to Owner-administered

assistance through the Rental Assistance Demonstration (RAD) program were excluded from the

sampling frame, due to unique rent calculation rules while households phase into the new program

type. However, other out-of-scope projects were identified later during data collection. Depending

on the circumstance of those identified during data collection, sampling decisions were made to

either replace the project, to subselect the project, or to make adjustments during weighting. The

use of replacement projects for out-of-scope projects complicated the sample weight calculations.

The determination of an actual probability of selection for these replacements was impossible to

make. A sampling weight that is proportional to what the probability would have been, had the

project been selected originally, was used as a reasonable estimate.

Population Totals. The same population totals were used in studies for FY 2012–2014, and in order

to provide accurate estimates based on the current assisted housing population, these population

totals were updated for FY 2015. The use of the same population counts from year to year has

had the advantage of increasing comparability of gross dollar estimates; any change from year to

year would not have been due to a change in the number of households in the program, but to

HUDQC Final Report for FY 2015

B-1

Appendix B: Weighting Procedure

an actual change in the average gross dollar error or percentage of households. In FY 2012, the

population counts used to produce the weights were updated from those used in FY 2011. As

programs may grow or shrink over time, it is desirable to update population counts. Estimates of

total dollar amounts and estimates of the proportion of the population represented by each program

type run the risk of not being representative of the current population if the population counts are

outdated or if the population changes significantly. Due to the inclusion of the MTW PHAs in

FY 2012, the nature of the population itself had changed. Because the FY 2011 population totals

and sample did not include the MTW population, using FY 2011 totals to produce FY 2012 weights

would produce invalid estimates.

Estimates of averages and percentages within program types have the advantage of being comparable

regardless of changes in population counts from year to year. This approach of using the same

population counts for multiple study years was used for FY 2005 through FY 2010 and again from

FY 2012 to FY 2014 to allow for comparability across years.

Exhibit B-1 provides the population totals by program type for the FY 2012 through

FY 2015 studies.

Exhibit B-1

Population Totals Used for Weighting by Program Type

Administration Type

FY 2012–FY 2014

FY 2015

Population

Population

Public Housing, non-MTW

1,040,708

959,766

Public Housing, MTW

114,088

101,924

PHA-administered Section 8, non-MTW

1,935,597

1,916,735

PHA-administered Section 8, MTW

263,125

292,561

Owner-administered

1,378,158

1,382,453

Total

4,731,676

4,653,439

Weighting Methodology.  The procedure to determine the final weights involved several steps,

including: 1) calculating the project weight ( w ); 2) calculating the household weight ( w );

1

3

3) accounting for nonresponding households ( fn); 4) poststratifying ( fp); and, 5) finally, trimming

the weights.

1.  Calculating the Project Weight ( w ). The first step to determine the final weights was calculating 1

the project weight by compiling the sampling probabilities calculated during the cluster and project

sampling and the initial data collection process. These probabilities were then used to calculate

each project’s probability of selection. The probability of selection of a project was the product of

the following:

1)  The probability of selection of the cluster ( p ) 1

2)  The probability of selection of the subcluster if the cluster was divided ( p ) 2

3)  The probability of selection of the project from its respective cluster ( p )

3

B-2

August 31, 2016

Appendix B: Weighting Procedure

Each cluster was sampled with probabilities proportional to size. The measure of size used was the

number of households adjusted to obtain equal expectation for the three major types of programs

in the study. The number of households of each program in a cluster was multiplied by an inflation

factor to make all three numbers equal. The probability of selection of the cluster ( p ) was calculated

1

in three steps. First, the proportion of the households in each of the three programs in a particular

cluster was obtained. These proportions were defined as the number of households in each program

within a cluster divided by the number nationwide (program’s population count). Next, the three

proportions in each cluster were averaged, and finally, the proportions were multiplied by 60, the

number of clusters to be selected nationwide.

In some instances, clusters were geographically too large to collect data in a cost-effective manner.

To accommodate this logistical problem, clusters were divided into two or more subclusters or

smaller geographic areas. A subcluster was then sampled from the group of subclusters using

probabilities proportional to size. This resulted in the same probability that would have ensued had

the division taken place before drawing the sample, or the probability of selection of the subcluster

( p ). If the cluster was not divided into smaller clusters, then the subcluster probability of selection

2

was one. The formula to calculate the project weight was:

Clusters with probabilities greater than one could have been selected more than once (Sampling

with Minimal Replacement). These clusters were certainty clusters, meaning that their selection

into the sample was guaranteed. For the purposes of calculating the project weight, the certainty

clusters’ probability of selection was set to one.

The probability of selection of a project from its respective cluster ( p ) was calculated in two steps.

3

First, the number of households in a program type within a project was divided by the total number

of households in a program type within the project’s cluster. This proportion was then multiplied

by the number of projects in a program type to be selected from the cluster. The PHA-administered

Section 8 projects could have had a probability greater than one for sampling purposes (meaning

they could be sampled more than once). However, for the other two major program types, if

the calculated probability exceeded one, it was set to one, and all the other probabilities were

readjusted so that they added to the allocation for the program in the cluster. For weighting purposes,

probabilities greater than one among PHA-administered Section 8 projects were set to one.

2.  Calculating the Household Weight ( w ). The second step to determine the final weights was

3

to calculate the household weight. To calculate the household weight, the number of households

in the project ( N ) and the number of households sampled from the project ( n ) were identified.

p

p

The household probability of selection within the sampled project was the number of sampled

households divided by the number of households in the project ( p ):

4

HUDQC Final Report for FY 2015

B-3

Appendix B: Weighting Procedure

The household within project weight ( w ) was the inverse of the probability of selecting the

2

household within the sampled project:

The household base weight ( w ) was the product of the project weight and the household within

3

project weight:

3.  Account for Nonresponding Households ( fn). The third step in the weighting process was

to account for nonresponding households within the sampled project. To do this, the number of

eligible sampled households ( np ), the number of responding households ( n ), and the eligibility-

e

pr

adjusted household weight was needed. The sum of the eligibility-adjusted household weights for

all eligible households in the project and the sum of eligibility-adjusted household weights for only

the responding households in a project was then calculated. A nonresponse adjustment factor ( fn)

was calculated as:

The nonresponse adjusted household weight ( w ) was the eligibility-adjusted household weight

4

multiplied by the nonresponse adjustment factor:

Poststratification ( fp). The fourth step in the weighting process was poststratification. The sample was designed to obtain similar numbers of households in each of the following three program types:

1)  Public Housing projects

2)  PHA-administered Section 8 projects

3)  Owner-administered projects

Population totals for each of the programs were obtained from the FY 2012 sampling frame. The

population estimates after weighting did not correspond exactly to these FY 2012 population totals

and required adjustments. The weights were adjusted to sum to the known external population

totals, so the sum of the weights would have been the same had a different sample been selected.

In the past, this was due partially to special circumstances, such as the exclusion of geographic

areas affected by the 2005 hurricanes and the Owner-administered projects from Alaska that were

excluded from the frame but included during the weighting process. Alaska was excluded from

the sample frame in the past because the state typically did not meet the parameters for creating

a sample cluster. Prior to FY 2012, all Public Housing and PHA-administered Section 8 projects

in the state were out of the scope of the study since they were part of the MTW program, leaving

just Owner-administered projects. With the inclusion of MTW projects, more Alaska projects were

B-4

August 31, 2016

considered in the scope of the study, and, therefore, Alaska was once again included. In FY 2015,

Alaska was included in the frame but was not selected.

To poststratify the weights, the nonresponse adjusted household weights within program type

were summed to estimate the population totals from the HUD sample. For example, the sum of

weights for all Owner-administered households in the sample is an estimate of the total number

of Owner-administered households in the nation. A poststratification factor ( fp) was calculated by

dividing the known external population totals (

) by the estimated population totals

from the HUD sample (

):

A poststratification factor was calculated for each program type. This factor was then multiplied

to the household weight within each program type, ensuring that the sum of the household weights

by program type was the same as the external population totals.

Trimming the Weights.  The final step was the trimming of the weights. Weights more than three

times the median weight were set to three times the median weight, and all the weights were

readjusted. Large weights usually resulted from incorrect frame information.

Effective Sample Size Due to Weighting.  In FY 2015, the weights led to an effective sample size

(because of the weighting) of 757 (down from an actual size of 799) for the Owner-administered

projects, 774 for the Public Housing projects (down from 805), and 777 for the PHA-administered

Section 8 projects (down from 796). The effective sample size is the size of a random sample that

would yield confidence intervals of the same size as the current sample. The design effect calculates

what the sample size would be when the variance produced by weighting is accounted for, and

this calculation yielded the same effective sample sizes as those cited above. The effective sample

size will often be smaller than the actual sample, partly because of clustering and partly because

of weighting.

Variance Estimation.  Standard errors were obtained for a number of estimates using a delete-a-group

Jackknife procedure. This was implemented by using 20 replicate groups and creating 20 sets of

replicate weights. This procedure is available starting with SAS 9.4, and is considered more robust

with respect to design characteristics than the Taylor Series method.1

The 20 Largest PHAs Weighting.  As in previous studies, the data for the 20 largest PHAs sample

were not weighted. The sample is approximately a self-weighting sample. The term  self-weighting

refers to a sample where all units being sampled (in this case households) have the same weight,

assuming that the frame is accurate and that a 100 percent response is achieved.

1  Kott, P. S. (1998). Using the Delete-a-Group Jackknife Variance Estimator in Practice.  Proceedings

of the Annual Meeting of the American Statistical Association, Section on Survey Research Methods

(pp. 763–768). Alexandria, VA: American Statistical Association.

HUDQC Final Report for FY 2015

B-5

Appendix B: Weighting Procedure

A self-weighting sample has several advantages, including:

● Permitting more precise estimates for the 20 largest PHAs. To the extent that the sample

departs from equal weights, the design effect will increase, causing correspondingly less

precise estimates.

● Facilitating reporting because unweighted means and proportions for the sample will be

estimates of the same means and proportions for the weighted population so as not to

confuse the reader.

B-6

August 31, 2016

Appendix C: Source Tables

Source Tables Based on Quality Control Data

Throughout these Source Tables, empty cells indicate that either the result was zero

or the analysis was not applicable.

The following tables include all data collected for the Quality Control study.

C-2

Appendix C: Source T

HUDQC FY 2015

Table 1a. Verification of QC Rent Components

Third-Party Verbal or in Writing, Documentation, or EIV/UIV

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

71

(4.6%)

84

(5.4%)

1,395

(90.0%)

Pension, Etc.

8

(0.3%)

86

(3.0%)

2,745

(96.7%)

ables

Public Assistance

15

(4.1%)

2

(0.5%)

352

(95.5%)

Other Income

100

(12.6%)

81

(10.3%)

612

(77.1%)

Asset Income

5

(1.2%)

36

(8.2%)

395

(90.6%)

Child Care Expense

166

(100.0%)

Disability Expense

10

(100.0%)

Medical Expense

3

(0.3%)

43

(3.4%)

1,238

(96.4%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 1b. Verification of QC Rent Components

Third Party in Writing

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

969

(61.2%)

129

(8.2%)

485

(30.6%)

Pension, Etc.

2,237

(78.8%)

290

(10.2%)

312

(11.0%)

Public Assistance

342

(77.8%)

3

(0.7%)

94

(21.5%)

Other Income

682

(73.5%)

72

(7.7%)

174

(18.8%)

Asset Income

130

(29.5%)

135

(30.7%)

175

(39.8%)

Child Care Expense

71

(42.9%)

95

(57.1%)

August 31, 2016

Disability Expense

4

(38.5%)

6

(61.5%)

Medical Expense

355

(27.6%)

393

(30.6%)

536

(41.8%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 1c. Verification of QC Rent Components

Third Party in Writing or EIV/UIV

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

898

(56.7%)

141

(8.9%)

545

(34.4%)

Pension, Etc.

690

(24.3%)

416

(14.6%)

1,735

(61.1%)

Public Assistance

211

(47.9%)

3

(0.7%)

226

(51.3%)

Other Income

621

(66.9%)

67

(7.3%)

240

(25.8%)

2015

Asset Income

125

(28.5%)

136

(31.0%)

178

(40.5%)

Child Care Expense

71

(42.9%)

95

(57.1%)

Disability Expense

4

(38.5%)

6

(61.5%)

Medical Expense

250

(19.5%)

342

(26.6%)

692

(53.9%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 1d. Verification of QC Rent Components

Third Party Verbal

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

1,556

(98.2%)

4

(0.2%)

24

(1.5%)

Pension, Etc.

2,830

(99.7%)

6

(0.2%)

2

(0.1%)

Public Assistance

438

(99.6%)

2

(0.4%)

Other Income

907

(97.7%)

4

(0.5%)

17

(1.8%)

Appendix C: Source T

Asset Income

437

(99.6%)

2

(0.4%)

Child Care Expense

164

(98.9%)

2

(1.1%)

Disability Expense

10

(100.0%)

Medical Expense

1,267

(98.7%)

15

(1.2%)

2

(0.1%)

2016.08.17  [Weighted]

ables

C-3

C-4

Appendix C: Source T

HUDQC FY 2015

Table 1e. Verification of QC Rent Components

Documentation

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

707

(44.7%)

136

(8.6%)

740

(46.8%)

Pension, Etc.

1,814

(63.9%)

363

(12.8%)

662

(23.3%)

ables

Public Assistance

315

(71.6%)

1

(0.3%)

123

(28.0%)

Other Income

523

(56.4%)

68

(7.3%)

337

(36.3%)

Asset Income

200

(45.5%)

135

(30.6%)

105

(23.8%)

Child Care Expense

97

(58.1%)

70

(41.9%)

Disability Expense

6

(61.5%)

4

(38.5%)

Medical Expense

710

(55.3%)

334

(26.0%)

240

(18.7%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 1f. Verification of QC Rent Components

EIV (Enterprise Income Verification)

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

1,584

(100.0%)

Pension, Etc.

1,128

(39.7%)

471

(16.6%)

1,240

(43.7%)

Public Assistance

436

(99.2%)

4

(0.8%)

Other Income

928

(100.0%)

Asset Income

439

(100.0%)

Child Care Expense

166

(100.0%)

Disability Expense

10

(100.0%)

August 31, 2016

Medical Expense

1,030

(80.3%)

179

(13.9%)

75

(5.8%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 1g. Verification of QC Rent Components

UIV (Upfront Income Verification)

Not Verified

Partially Verified

Fully Verified

Rent Component

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

# of Cases (in 1,000s)

Row % of Cases

Earned Income

1,471

(94.9%)

21

(1.3%)

58

(3.7%)

Pension, Etc.

2,804

(98.8%)

31

(1.1%)

4

(0.1%)

Public Assistance

241

(65.4%)

128

(34.6%)

2015

Other Income

723

(91.1%)

14

(1.7%)

56

(7.1%)

Asset Income

13

(100.0%)

Medical Expense

15

(100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 2. Percent of Households by Payment Type and Program Type

Underpayment

Proper Payment

Overpayment

Total

# of

# of

# of

# of

Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of Program Type

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

Public Housing

139

(13.1%)

(22.1%)

799

(75.2%)

(23.3%)

124

(11.7%)

(20.8%)

1,062

(100.0%)

(22.8%)

PHA-administered

Section 8

326

(14.8%)

(51.8%)

1,560

(70.6%)

(45.5%)

323

(14.6%)

(54.0%)

2,209

(100.0%)

(47.5%)

Total

465

(14.2%)

(73.9%)

2,359

(72.1%)

(68.8%)

447

(13.7%)

(74.9%)

3,271

(100.0%)

(70.3%)

Owner-administered

165

(11.9%)

(26.2%)

1,068

(77.2%)

(31.2%)

150

(10.9%)

(25.1%)

1,382

(100.0%)

(29.7%)

Owner-administered

Total

165

(11.9%)

(26.2%)

1,068

(77.2%)

(31.2%)

150

(10.9%)

(25.1%)

1,382

(100.0%)

(29.7%)

Appendix C: Source T

Total

630

(13.5%) (100.0%)

3,427

(73.6%) (100.0%)

597

(12.8%) (100.0%)

4,653

(100.0%) (100.0%)

2016.08.17  [Weighted]

ables

C-5

C-6

Appendix C: Source T

HUDQC FY 2015

Table 2(S). Percent of Households by Payment Type and Program Type

(Proper Payment Based on Exact Match of Actual and QC Rent)

Underpayment

Proper Payment

Overpayment

Total

# of

# of

# of

# of

Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of Program Type

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

Public Housing

198

(18.6%)

(22.4%)

671

(63.2%)

(23.5%)

193

(18.2%)

(21.1%)

1,062

(100.0%)

(22.8%)

ables

PHA-administered

Section 8

445

(20.1%)

(50.4%)

1,279

(57.9%)

(44.8%)

485

(22.0%)

(53.1%)

2,209

(100.0%)

(47.5%)

Total

642

(19.6%)

(72.8%)

1,950

(59.6%)

(68.3%)

678

(20.7%)

(74.2%)

3,271

(100.0%)

(70.3%)

Owner-administered

241

(17.4%)

(27.3%)

905

(65.5%)

(31.7%)

236

(17.1%)

(25.8%)

1,382

(100.0%)

(29.7%)

Owner-administered Total

241

(17.4%)

(27.3%)

905

(65.5%)

(31.7%)

236

(17.1%)

(25.8%)

1,382

(100.0%)

(29.7%)

Total

883

(19.0%) (100.0%)

2,856

(61.4%) (100.0%)

915

(19.7%) (100.0%)

4,653

(100.0%) (100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 3. Dollar Rent Error by Program Type

Actual Rent (Monthly)

QC Rent (Monthly)

Gross Rent Error (Monthly)

Sum

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Dollar

Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount   Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount

Public Housing

1,062

(22.8%)

292,018

275.05

1,062

(22.8%)

293,178

276.14

1,062

(22.8%)

15,274

14.39

PHA-administered

Section 8

2,209

(47.5%)

586,239

265.35

2,209

(47.5%)

600,714

271.90

2,209

(47.5%)

33,398

15.12

Total

3,271

(70.3%)

878,258

268.50

3,271

(70.3%)

893,893

273.28

3,271

(70.3%)

48,672

14.88

Owner-administered

1,382

(29.7%)

340,759

246.49

1,382

(29.7%)

345,912

250.22

1,382

(29.7%)

13,646

9.87

Owner-administered

Total

1,382

(29.7%)

340,759

246.49

1,382

(29.7%)

345,912

250.22

1,382

(29.7%)

13,646

9.87

Total

4,653

(100.0%) 1,219,016

261.96

4,653

(100.0%) 1,239,804

266.43

4,653 (100.0%)

62,318

13.39

August 31, 2016

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 4. Dollar Error Amount by Payment Type and Program Type

Underpayment (Monthly)

Overpayment (Monthly)

QC Rent (Monthly)

Sum

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Dollar

Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount   Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount

Public Housing

139

(22.1%)

8,238

59.36

124

(20.8%)

7,035

56.57

1,062

(22.8%)

293,178

276.14

PHA-administered

Section 8

326

(51.8%)

23,954

73.44

323

(54.0%)

9,444

29.26

2,209

(47.5%)

600,714

271.90

2015

Total

465

(73.9%)

32,193

69.24

447

(74.9%)

16,479

36.86

3,271

(70.3%)

893,893

273.28

Owner-administered

165

(26.2%)

9,419

57.21

150

(25.1%)

4,228

28.18

1,382

(29.7%)

345,912

250.22

Owner-administered

Total

165

(26.2%)

9,419

57.21

150

(25.1%)

4,228

28.18

1,382

(29.7%)

345,912

250.22

Total

630

(100.0%)

41,612

66.10

597

(100.0%)

20,707

34.68

4,653

(100.0%) 1,239,804

266.43

2016.08.17  [Weighted]

HUDQC FY 2015

Table 4(S). Dollar Error Amount by Payment Type and Program Type

(Proper Payment Based on Exact Match of Actual and QC Rent)

Underpayment (Monthly)

Overpayment (Monthly)

QC Rent (Monthly)

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Sum Dollar  Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount (in  Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

1,000)

Amount

Public Housing

198

(22.4%)

8,387

42.43

193

(21.1%)

7,179

37.20

1,062

(22.8%)

293,178

276.14

PHA-administered

Section 8

445

(50.4%)

24,255

54.56

485

(53.1%)

9,780

20.15

2,209

(47.5%)

600,714

271.90

Appendix C: Source T

Total

642

(72.8%)

32,642

50.83

678

(74.2%)

16,959

25.00

3,271

(70.3%)

893,893

273.28

Owner-administered

241

(27.3%)

9,605

39.91

236

(25.8%)

4,452

18.84

1,382

(29.7%)

345,912

250.22

Owner-administered Total

241

(27.3%)

9,605

39.91

236

(25.8%)

4,452

18.84

1,382

(29.7%)

345,912

250.22

Total

883

(100.0%)

42,248

47.85

915

(100.0%)

21,411

23.40

4,653

(100.0%) 1,239,804

266.43

2016.08.17  [Weighted]

ables

C-7

C-8

Appendix C: Source T

HUDQC FY 2015

Table 5. Gross and Net Rent Error by Program Type

Gross Rent Error (Monthly)

Net Rent Error (Monthly)

QC Rent (Monthly)

Sum

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Dollar

Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount   Dollar

Cases   Col. % of  Amount   Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount

Public Housing

1,062

(22.8%)

15,274

14.39

1,062

(22.8%)

-1,203

-1.13

1,062

(22.8%)

293,178

276.14

ables

PHA-administered

Section 8

2,209

(47.5%)

33,398

15.12

2,209

(47.5%) -14,511

-6.57

2,209

(47.5%)

600,714

271.90

Total

3,271

(70.3%)

48,672

14.88

3,271

(70.3%) -15,713

-4.80

3,271

(70.3%)

893,893

273.28

Owner-administered

1,382

(29.7%)

13,646

9.87

1,382

(29.7%)

-5,191

-3.76

1,382

(29.7%)

345,912

250.22

Owner-administered Total

1,382

(29.7%)

13,646

9.87

1,382

(29.7%)

-5,191

-3.76

1,382

(29.7%)

345,912

250.22

Total

4,653

(100.0%)

62,318

13.39

4,653

(100.0%) -20,905

-4.49

4,653

(100.0%) 1,239,804

266.43

2016.08.17  [Weighted]

HUDQC FY 2015

Table 5(S). Gross and Net Rent Error by Program Type

(Proper Payment Based on Exact Match of Actual and QC Rent)

Gross Rent Error (Monthly)

Net Rent Error (Monthly)

QC Rent (Monthly)

Sum

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Dollar

Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount   Dollar

Cases   Col. % of  Amount   Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount

Public Housing

1,062

(22.8%)

15,566

14.66

1,062

(22.8%)

-1,209

-1.14

1,062

(22.8%)

293,178

276.14

PHA-administered

Section 8

2,209

(47.5%)

34,035

15.41

2,209

(47.5%)

-14,475

-6.55

2,209

(47.5%)

600,714

271.90

Total

3,271

(70.3%)

49,601

15.16

3,271

(70.3%)

-15,684

-4.79

3,271

(70.3%)

893,893

273.28

Owner-administered

1,382

(29.7%)

14,057

10.17

1,382

(29.7%)

-5,153

-3.73

1,382

(29.7%)

345,912

250.22

Owner-administered Total

1,382

(29.7%)

14,057

10.17

1,382

(29.7%)

-5,153

-3.73

1,382

(29.7%)

345,912

250.22

August 31, 2016

Total

4,653

(100.0%)

63,659

13.68

4,653

(100.0%)

-20,837

-4.48

4,653

(100.0%) 1,239,804

266.43

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 6. Certifications and Recertifications by Administration Type

Certifications

Recertifications/Non-Overdue

Recertifications/Overdue

Total

# of

# of

# of

# of

Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of Program Type

(in 1,000) Cases

Cases (in 1,000) Cases

Cases (in 1,000) Cases

Cases (in 1,000) Cases

Cases

Public Housing

136

(12.8%)

(22.9%)

895

(84.3%)

(22.4%)

30

(2.9%)

(42.3%)

1,062

(100.0%)

(22.8%)

PHA-administered

Section 8

262

(11.9%)

(44.1%)

1,909

(86.4%)

(47.9%)

38

(1.7%)

(53.0%)

2,209

(100.0%)

(47.5%)

Total

398

(12.2%)

(67.1%)

2,805

(85.7%)

(70.3%)

68

(2.1%)

(95.3%)

3,271

(100.0%)

(70.3%)

2015

Owner-administered

196

(14.1%)

(32.9%)

1,184

(85.6%)

(29.7%)

3

(.2%)

(4.7%)

1,382

(100.0%)

(29.7%)

Owner-administered

Total

196

(14.1%)

(32.9%)

1,184

(85.6%)

(29.7%)

3

(.2%)

(4.7%)

1,382

(100.0%)

(29.7%)

Total

594

(12.8%) (100.0%)

3,988

(85.7%) (100.0%)

72

(1.5%)

(100.0%)

4,653

(100.0%) (100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 7. Percent of Newly Certified Households Meeting Certification Criteria

Met Criterion

Did Not Meet Criterion

Certification Criteria

# of Households (in 1,000s)

% of Households

# of Households (in 1,000s)

% of Households

Citizenship

594

(100.0%)

Social Security Number

587

(98.9%)

7

(1.1%)

Consent Form

573

(96.6%)

20

(3.4%)

Low and Very Low Income

594

(100.0%)

Meets All Eligibility Criteria

568

(95.7%)

25

(4.3%)

2016.08.17  [Weighted]

Appendix C: Source T

ables

C-9

C-10

Appendix C: Source T

HUDQC FY 2015

Table 7b. Percent of Newly Certified Households Meeting Certification Criteria by Program Type

Met Criterion

Did Not Meet Criterion

# of Households

# of Households

Certification Criteria

(in 1,000s)

% of Households

(in 1,000s)

% of Households

Citizenship

136

(100.0%)

Social Security Number

135

(98.9%)

1

(1.1%)

ables

Public Housing

Consent Form

126

(92.7%)

10

(7.3%)

Low and Very Low Income

136

(100.0%)

Meets All Eligibility Criteria

126

(92.7%)

10

(7.3%)

Citizenship

262

(100.0%)

Social Security Number

262

(100.0%)

PHA-administered Section 8

Consent Form

253

(96.7%)

9

(3.3%)

Low and Very Low Income

262

(100.0%)

Meets All Eligibility Criteria

253

(96.7%)

9

(3.3%)

Citizenship

196

(100.0%)

Social Security Number

190

(97.4%)

5

(2.6%)

Owner-administered

Consent Form

194

(99.2%)

2

(.8%)

Low and Very Low Income

196

(100.0%)

Meets All Eligibility Criteria

189

(96.6%)

7

(3.4%)

2016.08.17  [Weighted]

August 31, 2016

HUDQC Final Report for FY

HUDQC FY 2015

Table 8. Dollar Error Amount by Payment Type and Case Type

Underpayment (Monthly)

Overpayment (Monthly)

QC Rent (Monthly)

Sum Dollar

Sum Dollar

Sum Dollar

# of Cases  Col. % of  Amount   Avg. Dollar # of Cases  Col. % of  Amount   Avg. Dollar # of Cases  Col. % of  Amount   Avg. Dollar Case Type

(in 1,000s)

Cases

(in 1,000s) Amount (in 1,000s)

Cases

(in 1,000s) Amount (in 1,000s)

Cases

(in 1,000s) Amount

Certification

Total

83

(13.2%)

5,182

62.28

82

(13.7%)

2,327

28.42

594

(12.8%)

140,958

237.50

Non-Overdue

520

(82.6%)

34,514

66.34

493

(82.5%)

14,800

30.05

3,988

(85.7%)

1,077,248

270.11

Recertification Overdue

26

(4.1%)

1,915

73.46

23

(3.8%)

3,580

157.71

72

(1.5%)

21,598

301.20

2015

Total

546

(86.8%)

36,429

66.68

515

(86.3%)

18,380

35.68

4,060

(87.2%)

1,098,846

270.66

Total

630

(100.0%)

41,612

66.10

597

(100.0%)

20,707

34.68

4,653

(100.0%)

1,239,804

266.43

2016.08.17  [Weighted]

HUDQC FY 2015

Table 8(S). Dollar Error Amount by Payment Type and Case Type

(Proper Payment Based on Exact Match of Actual and QC Rent)

Underpayment (Monthly)

Overpayment (Monthly)

QC Rent (Monthly)

Sum Dollar

Sum Dollar

Sum Dollar

# of Cases   Col. % of  Amount   Avg. Dollar # of Cases   Col. % of  Amount   Avg. Dollar # of Cases   Col. % of  Amount   Avg. Dollar Case Type

(in 1,000s)

Cases

(in 1,000s) Amount (in 1,000s)

Cases

(in 1,000s) Amount (in 1,000s)

Cases

(in 1,000s) Amount

Certification

Total

115

(13.0%)

5,251

45.86

121

(13.2%)

2,417

20.00

594

(12.8%)

140,958

237.50

Non-Overdue

733

(83.0%)

35,051

47.81

770

(84.2%)

15,413

20.02

3,988

(85.7%) 1,077,248

270.11

Recertification Overdue

35

(4.0%)

1,946

55.29

24

(2.6%)

3,582

148.82

72

(1.5%)

21,598

301.20

Total

768

(87.0%)

36,997

48.15

794

(86.8%)

18,994

23.92

4,060

(87.2%) 1,098,846

270.66

Appendix C: Source T

Total

883

(100.0%)

42,248

47.85

915

(100.0%)

21,411

23.40

4,653

(100.0%) 1,239,804

266.43

2016.08.17  [Weighted]

C-1

ables

1

C-12

Appendix C: Source T

HUDQC FY 2015

Table 9. Largest Component Error for Households With Rent Error (Annual Dollars)

Rent Component

# of Households (in 1,000s)

Col. % of Households

Sum Dollar Amount (in 1,000s)

Avg. Dollar Amount

Earned Income

408

(33.3%)

1,575,799

3,860

Pension, Etc.

188

(15.4%)

484,639

2,572

Public Assistance

50

(4.0%)

105,865

2,133

Other Income

158

(12.9%)

387,683

2,453

ables

Asset Income

30

(2.4%)

17,865

596

Dependent Allowance

57

(4.6%)

29,620

522

Elderly Household Allowance

69

(5.6%)

27,457

400

Child Care Allowance

24

(2.0%)

51,653

2,148

Disability Allowance

Medical Allowance

191

(15.6%)

173,009

905

No Error

52

(4.2%)

0

0

Total

1,227

(100.0%)

2,853,591

2,326

2016.08.17  [Weighted]

HUDQC FY 2015

Table 10. Total and Largest Dollar Error by Program Type for Households With Rent Errors

Total Dollar In Error

Largest Dollar Error

Sum Dollar

Sum Dollar

# of Cases

Col. % of

Amount

Avg. Dollar

# of Cases

Col. % of

Amount

Avg. Dollar

Program Type

(in 1,000s)

Cases

(in 1,000s)

Amount

(in 1,000s)

Cases

(in 1,000s)

Amount

Public Housing

263

(21.5%)

895,080

3,401.41

263

(21.5%)

745,524

2,833.08

PHA-administered

Section 8

649

(52.9%)

1,674,109

2,580.06

649

(52.9%)

1,472,832

2,269.86

Total

912

(74.4%)

2,569,189

2,817.05

912

(74.4%)

2,218,356

2,432.37

Owner-administered

315

(25.7%)

763,830

2,427.56

315

(25.7%)

635,235

2,018.87

August 31, 2016

Owner-administered

Total

315

(25.7%)

763,830

2,427.56

315

(25.7%)

635,235

2,018.87

Total

1,227

(100.0%)

3,333,019

2,717.15

1,227

(100.0%)

2,853,591

2,326.31

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 11. QC Rent Components by Payment Type and Administration Type

PHA-administered

Owner-administered

Total

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

Rent Component

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

Earned Income

235

(7.2%)

(81.6%)

53

(3.9%)

(18.4%)

289

(6.2%)

(100.0%)

Pension, Etc.

118

(3.6%)

(73.9%)

42

(3.0%)

(26.3%)

160

(3.4%)

(100.0%)

Public Assistance

26

(.8%)

(100.0%)

26

(.6%)

(100.0%)

Other Income

109

(3.3%)

(82.6%)

23

(1.7%)

(17.4%)

131

(2.8%)

(100.0%)

2015

Asset Income

37

(1.1%)

(57.3%)

28

(2.0%)

(42.7%)

65

(1.4%)

(100.0%)

Underpayment

Dependent Allowance

34

(1.0%)

(88.0%)

5

(.3%)

(12.0%)

38

(.8%)

(100.0%)

Elderly Household

Allowance

16

(.5%)

(90.2%)

2

(.1%)

(9.8%)

18

(.4%)

(100.0%)

Child Care Allowance

16

(.5%)

(76.6%)

5

(.4%)

(23.4%)

21

(.5%)

(100.0%)

Disability Allowance

Medical Allowance

66

(2.0%)

(48.3%)

71

(5.1%)

(51.7%)

137

(3.0%)

(100.0%)

No Error

17

(.5%)

(70.9%)

7

(.5%)

(29.1%)

24

(.5%)

(100.0%)

Earned Income

228

(7.0%)

(85.9%)

38

(2.7%)

(14.1%)

266

(5.7%)

(100.0%)

Pension, Etc.

285

(8.7%)

(63.7%)

162

(11.7%)

(36.3%)

447

(9.6%)

(100.0%)

Public Assistance

58

(1.8%)

(63.8%)

33

(2.4%)

(36.2%)

91

(1.9%)

(100.0%)

Other Income

171

(5.2%)

(79.5%)

44

(3.2%)

(20.5%)

215

(4.6%)

(100.0%)

Asset Income

88

(2.7%)

(61.3%)

56

(4.0%)

(38.7%)

143

(3.1%)

(100.0%)

Proper Payment

Dependent Allowance

22

(.7%)

(80.6%)

5

(.4%)

(19.4%)

28

(.6%)

(100.0%)

Elderly Household

Allowance

16

(.5%)

(83.5%)

3

(.2%)

(16.5%)

20

(.4%)

(100.0%)

Child Care Allowance

13

(.4%)

(100.0%)

13

(.3%)

(100.0%)

Appendix C: Source T

Disability Allowance

Medical Allowance

156

(4.8%)

(53.1%)

138

(10.0%)

(46.9%)

294

(6.3%)

(100.0%)

No Error

1,617

(49.4%)

(69.0%)

725

(52.5%)

(31.0%)

2,342

(50.3%)

(100.0%)

C-13

ables

C-14

Appendix C: Source T

HUDQC FY 2015

Table 11. QC Rent Components by Payment Type and Administration Type (continued)

PHA-administered

Owner-administered

Total

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

Rent Component

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

Earned Income

154

(4.7%)

(83.6%)

30

(2.2%)

(16.4%)

184

(4.0%)

(100.0%)

Pension, Etc.

108

(3.3%)

(72.9%)

40

(2.9%)

(27.1%)

149

(3.2%)

(100.0%)

ables

Public Assistance

34

(1.0%)

(79.7%)

9

(.6%)

(20.3%)

42

(.9%)

(100.0%)

Other Income

102

(3.1%)

(83.3%)

20

(1.5%)

(16.7%)

123

(2.6%)

(100.0%)

Asset Income

28

(.9%)

(59.0%)

20

(1.4%)

(41.0%)

48

(1.0%)

(100.0%)

Overpayment

Dependent Allowance

34

(1.0%)

(70.7%)

14

(1.0%)

(29.3%)

48

(1.0%)

(100.0%)

Elderly Household

Allowance

46

(1.4%)

(56.3%)

36

(2.6%)

(43.7%)

82

(1.8%)

(100.0%)

Child Care Allowance

22

(.7%)

(100.0%)

22

(.5%)

(100.0%)

Disability Allowance

Medical Allowance

85

(2.6%)

(58.1%)

62

(4.5%)

(41.9%)

147

(3.2%)

(100.0%)

No Error

27

(.8%)

(100.0%)

27

(.6%)

(100.0%)

Total with Rent Error Calculation

3,271

(100.0%)

(70.3%)

1,382

(100.0%)

(29.7%)

4,653

(100.0%)

(100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 12a. Elderly/Disabled Allowances

Non-Elderly/Disabled Household

Elderly/Disabled Household

Total

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

Allowances

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

No Allowance

1,946

(99.2%)

(100.0%)

1,946

(41.8%)

(100.0%)

Incorrect Allowance

16

(.8%)

(13.2%)

104

(3.9%)

(86.8%)

120

(2.6%)

(100.0%)

August 31, 2016

Correct Allowance

2,588

(96.1%)

(100.0%)

2,588

(55.6%)

(100.0%)

Total

1,961

(100.0%)

(42.1%)

2,692

(100.0%)

(57.9%)

4,653

(100.0%)

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 12b. Dependent Allowances

Households Without Dependent(s)

Households With Dependent(s)

Total

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

# of Cases

Col. % of

Row % of

Allowances

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

(in 1,000s)

Cases

Cases

No Allowance

2,781

(99.8%)

(100.0%)

2,781

(59.8%)

(100.0%)

Incorrect Allowance

4

(.2%)

(3.8%)

110

(5.9%)

(96.2%)

114

(2.5%)

(100.0%)

Correct Allowance

1,758

(94.1%)

(100.0%)

1,758

(37.8%)

(100.0%)

2015

Total

2,785

(100.0%)

(59.9%)

1,868

(100.0%)

(40.1%)

4,653

(100.0%)

(100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 13. Calculation Errors on Form HUD-50058/50059

Form HUD-50058

Form HUD-50059

Total

# of Households

# of Households

# of Households

Items

# of Errors

(in 1,000s)

# of Errors

(in 1,000s)

# of Errors

(in 1,000s)

Household Composition

173

171

0

173

171

Net Family Assets and Income

378

310

161

73

540

383

Allowances and Adjusted Income

1,542

1,380

0

1,542

1,380

Family Rent and Subsidy Information

545

302

0

545

302

2016.08.17  [Weighted]

Appendix C: Source T

HUDQC FY 2015

Table 14. Consistency Errors on Form HUD-50058/50059

Form HUD-50058

Form HUD-50059

Total

# of Households

# of Households

# of Households

Items

# of Errors

(in 1,000s)

# of Errors

(in 1,000s)

# of Errors

(in 1,000s)

General Information

46

46

203

159

250

205

Household Composition

313

139

344

316

657

454

C-15

ables

Net Family Assets and Income

111

58

0

111

58

Allowances and Adjusted Income

323

307

7

7

330

314

Family Rent and Subsidy Information

69

69

5

5

74

74

2016.08.17  [Weighted]

C-16

Appendix C: Source T

HUDQC FY 2015

Table 15a. Verification of Form HUD-50058/50059 Rent Components

Third-Party Verbal or in Writing, Documentation, or EIV

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

215

(13.7%)

410

(26.1%)

949

(60.3%)

1,575

(100.0%)

ables

Pension, Etc.

143

(5.0%)

368

(13.0%)

2,328

(82.0%)

2,839

(100.0%)

Public Assistance

93

(21.5%)

60

(14.0%)

278

(64.5%)

432

(100.0%)

Other Income

323

(35.0%)

191

(20.7%)

410

(44.4%)

924

(100.0%)

Asset Income

41

(10.3%)

42

(10.5%)

316

(79.2%)

399

(100.0%)

Child Care Expense

3

(1.8%)

37

(24.8%)

110

(73.4%)

151

(100.0%)

Disability Expense

9

(80.3%)

2

(19.7%)

12

(100.0%)

Medical Expense

83

(8.8%)

213

(22.7%)

641

(68.4%)

937

(100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 15b. Verification of Form HUD-50058/50059 Rent Components

Third Party in Writing

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

1,253

(79.5%)

74

(4.7%)

248

(15.8%)

1,575

(100.0%)

Pension, Etc.

2,815

(99.2%)

4

(.2%)

19

(.7%)

2,839

(100.0%)

Public Assistance

362

(83.9%)

10

(2.2%)

60

(13.9%)

432

(100.0%)

August 31, 2016

Other Income

790

(85.5%)

31

(3.3%)

103

(11.2%)

924

(100.0%)

Asset Income

257

(64.5%)

14

(3.5%)

128

(32.1%)

399

(100.0%)

Child Care Expense

75

(49.8%)

14

(9.6%)

61

(40.6%)

151

(100.0%)

Disability Expense

5

(47.6%)

4

(32.7%)

2

(19.7%)

12

(100.0%)

Medical Expense

801

(85.5%)

20

(2.2%)

116

(12.3%)

937

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 15c. Verification of Form HUD-50058/50059 Rent Components

Third Party in Writing or EIV/UIV

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

1,191

(75.6%)

101

(6.4%)

283

(18.0%)

1,575

(100.0%)

Pension, Etc.

1,317

(46.4%)

196

(6.9%)

1,325

(46.7%)

2,839

(100.0%)

2015

Public Assistance

230

(53.2%)

34

(7.9%)

168

(38.9%)

432

(100.0%)

Other Income

724

(78.4%)

53

(5.8%)

146

(15.8%)

924

(100.0%)

Asset Income

256

(64.1%)

14

(3.5%)

129

(32.4%)

399

(100.0%)

Child Care Expense

75

(49.8%)

14

(9.6%)

61

(40.6%)

151

(100.0%)

Disability Expense

5

(47.6%)

4

(32.7%)

2

(19.7%)

12

(100.0%)

Medical Expense

642

(68.5%)

54

(5.8%)

241

(25.7%)

937

(100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 15d. Verification of Form HUD-50058/50059 Rent Components

Third Party Verbal

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Appendix C: Source T

Earned Income

1,550

(98.5%)

3

(.2%)

22

(1.4%)

1,575

(100.0%)

Pension, Etc.

2,834

(99.8%)

4

(.2%)

2,839

(100.0%)

Public Assistance

430

(99.6%)

2

(.4%)

432

(100.0%)

Other Income

905

(98.0%)

18

(2.0%)

924

(100.0%)

Asset Income

399

(100.0%)

399

(100.0%)

Child Care Expense

149

(98.8%)

2

(1.2%)

151

(100.0%)

Disability Expense

12

(100.0%)

12

(100.0%)

C-17

ables

Medical Expense

935

(99.8%)

2

(.2%)

937

(100.0%)

2016.08.17  [Weighted]

C-18

Appendix C: Source T

HUDQC FY 2015

Table 15e. Verification of Form HUD-50058/50059 Rent Components

Documentation

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

# of Cases

# of Cases

# of Cases

Rent Component

(in 1,000s)

Row % of Cases

(in 1,000s)

Row % of Cases

(in 1,000s)

Row % of Cases

(in 1,000s)

Row % of Cases

Earned Income

669

(42.5%)

282

(17.9%)

624

(39.6%)

1,575

(100.0%)

ables

Pension, Etc.

1,984

(69.9%)

92

(3.2%)

763

(26.9%)

2,839

(100.0%)

Public Assistance

297

(68.8%)

26

(6.1%)

108

(25.1%)

432

(100.0%)

Other Income

549

(59.5%)

132

(14.3%)

243

(26.3%)

924

(100.0%)

Asset Income

253

(63.3%)

13

(3.2%)

133

(33.4%)

399

(100.0%)

Child Care Expense

80

(53.2%)

23

(15.3%)

48

(31.6%)

151

(100.0%)

Disability Expense

6

(52.4%)

5

(47.6%)

12

(100.0%)

Medical Expense

568

(60.6%)

99

(10.6%)

270

(28.8%)

937

(100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015

Table 15f. Verification of Form HUD-50058/50059 Rent Components

EIV (Enterprise Income Verification)

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

# of Cases

# of Cases

# of Cases

Rent Component

(in 1,000s)

Row % of Cases

(in 1,000s)

Row % of Cases

(in 1,000s)

Row % of Cases

(in 1,000s)

Row % of Cases

Earned Income

1,575

(100.0%)

1,575

(100.0%)

Pension, Etc.

1,455

(51.3%)

147

(5.2%)

1,237

(43.6%)

2,839

(100.0%)

Public Assistance

428

(99.1%)

1

(.3%)

2

(.5%)

432

(100.0%)

August 31, 2016

Other Income

924

(100.0%)

924

(100.0%)

Asset Income

399

(100.0%)

399

(100.0%)

Child Care Expense

151

(100.0%)

151

(100.0%)

Disability Expense

12

(100.0%)

12

(100.0%)

Medical Expense

826

(88.1%)

25

(2.6%)

87

(9.3%)

937

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 15g. Verification of Form HUD-50058/50059 Rent Components

UIV (Upfront Income Verification)

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

1,513

(96.1%)

27

(1.7%)

35

(2.2%)

1,575

(100.0%)

2015

Pension, Etc.

2,835

(99.9%)

4

(.1%)

2,839

(100.0%)

Public Assistance

303

(70.1%)

23

(5.4%)

106

(24.5%)

432

(100.0%)

Other Income

863

(93.4%)

23

(2.5%)

38

(4.1%)

924

(100.0%)

Asset Income

388

(97.3%)

4

(.9%)

7

(1.8%)

399

(100.0%)

Child Care Expense

151

(100.0%)

151

(100.0%)

Disability Expense

12

(100.0%)

12

(100.0%)

Medical Expense

922

(98.4%)

4

(.4%)

12

(1.2%)

937

(100.0%)

2016.08.17  [Weighted]

Appendix C: Source T

C-19

ables

C-20

Appendix C: Source T

HUDQC FY 2015

Table 15h. Verification of Form HUD-50058/50059 Rent Components

Third-Party Verbal or in Writing, Documentation, or EIV

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of  # of Cases (in  Row % of

# of Cases

Row % of

Rent Component by Program Type

(in 1,000s)

Cases

(in 1,000s)

Cases

1,000s)

Cases

(in 1,000s)

Cases

Earned Income

83

(21.7%)

104

(27.2%)

195

(51.1%)

381

(100.0%)

ables

Pension, Etc.

37

(6.2%)

98

(16.7%)

454

(77.1%)

589

(100.0%)

Public Assistance

24

(20.6%)

19

(16.8%)

72

(62.6%)

115

(100.0%)

Other Income

86

(41.3%)

30

(14.6%)

91

(44.0%)

207

(100.0%)

Public Housing

Asset Income

8

(11.7%)

14

(20.0%)

48

(68.3%)

70

(100.0%)

Child Care Expense

11

(30.7%)

25

(69.3%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

29

(15.1%)

48

(24.6%)

117

(60.3%)

193

(100.0%)

Earned Income

110

(12.3%)

226

(25.3%)

557

(62.3%)

894

(100.0%)

Pension, Etc.

66

(5.3%)

167

(13.5%)

1,001

(81.2%)

1,233

(100.0%)

Public Assistance

35

(15.6%)

34

(15.1%)

155

(69.3%)

224

(100.0%)

Other Income

156

(29.5%)

134

(25.3%)

238

(45.2%)

528

(100.0%)

PHA-administered Section 8

Asset Income

17

(12.4%)

13

(9.9%)

104

(77.7%)

134

(100.0%)

Child Care Expense

3

(3.3%)

23

(28.4%)

55

(68.3%)

81

(100.0%)

Disability Expense

4

(62.5%)

2

(37.5%)

6

(100.0%)

Medical Expense

23

(8.4%)

80

(29.8%)

165

(61.8%)

268

(100.0%)

Earned Income

22

(7.4%)

80

(26.7%)

197

(65.9%)

300

(100.0%)

Pension, Etc.

40

(4.0%)

103

(10.1%)

873

(85.9%)

1,017

(100.0%)

Public Assistance

34

(37.2%)

7

(7.8%)

51

(55.0%)

92

(100.0%)

Other Income

81

(43.2%)

27

(14.3%)

80

(42.6%)

189

(100.0%)

Owner-administered

Asset Income

16

(8.4%)

14

(7.4%)

164

(84.2%)

194

(100.0%)

Child Care Expense

3

(10.2%)

30

(89.8%)

34

(100.0%)

August 31, 2016

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

31

(6.6%)

86

(18.0%)

359

(75.5%)

476

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 15i. Verification of Form HUD-50058/50059 Rent Components

Third Party in Writing

Verification

No Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component by Program Type

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

264

(69.3%)

32

(8.4%)

85

(22.2%)

381

(100.0%)

Pension, Etc.

589

(100.0%)

589

(100.0%)

2015

Public Assistance

93

(80.5%)

4

(3.5%)

18

(16.0%)

115

(100.0%)

Other Income

172

(83.0%)

11

(5.4%)

24

(11.7%)

207

(100.0%)

Public Housing

Asset Income

51

(72.3%)

5

(6.7%)

15

(21.0%)

70

(100.0%)

Child Care

Expense

14

(37.9%)

5

(13.3%)

17

(48.8%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

176

(91.1%)

4

(1.9%)

14

(7.0%)

193

(100.0%)

Earned Income

797

(89.1%)

27

(3.0%)

70

(7.8%)

894

(100.0%)

Pension, Etc.

1,222

(99.1%)

3

(.2%)

9

(.7%)

1,233

(100.0%)

Public Assistance

194

(86.3%)

6

(2.5%)

25

(11.2%)

224

(100.0%)

Other Income

483

(91.5%)

12

(2.3%)

33

(6.2%)

528

(100.0%)

PHA-administered Section 8

Asset Income

119

(88.4%)

7

(5.2%)

9

(6.4%)

134

(100.0%)

Child Care

Expense

58

(71.5%)

8

(9.8%)

15

(18.6%)

81

(100.0%)

Disability Expense

4

(62.5%)

2

(37.5%)

6

(100.0%)

Medical Expense

258

(96.5%)

9

(3.5%)

268

(100.0%)

Earned Income

192

(63.9%)

15

(4.9%)

93

(31.2%)

300

(100.0%)

Pension, Etc.

1,004

(98.8%)

2

(.2%)

11

(1.1%)

1,017

(100.0%)

Appendix C: Source T

Public Assistance

76

(82.1%)

16

(17.9%)

92

(100.0%)

Other Income

135

(71.4%)

7

(3.9%)

47

(24.7%)

189

(100.0%)

Owner-administered

Asset Income

88

(45.1%)

2

(1.1%)

105

(53.8%)

194

(100.0%)

Child Care

Expense

3

(10.3%)

2

(5.0%)

29

(84.7%)

34

(100.0%)

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

366

(77.0%)

17

(3.5%)

93

(19.5%)

476

(100.0%)

C-21

2016.08.17  [Weighted]

ables

C-22

Appendix C: Source T

HUDQC FY 2015

Table 15j. Verification of Form HUD-50058/50059 Rent Components

Third Party in Writing or EIV/UIV

Verification

No Verification

Dollar Amount Not Matched Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of  # of Cases (in  Row % of

# of Cases

Row % of

Rent Component by Program Type

(in 1,000s)

Cases

(in 1,000s)

Cases

1,000s)

Cases

(in 1,000s)

Cases

Earned Income

258

(67.8%)

35

(9.2%)

87

(23.0%)

381

(100.0%)

ables

Pension, Etc.

271

(46.0%)

58

(9.9%)

260

(44.1%)

589

(100.0%)

Public Assistance

42

(36.7%)

16

(14.1%)

57

(49.2%)

115

(100.0%)

Other Income

158

(76.4%)

15

(7.1%)

34

(16.6%)

207

(100.0%)

Public Housing

Asset Income

50

(70.6%)

5

(6.7%)

16

(22.8%)

70

(100.0%)

Child Care Expense

14

(37.9%)

5

(13.3%)

17

(48.8%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

121

(62.6%)

16

(8.3%)

56

(29.2%)

193

(100.0%)

Earned Income

758

(84.8%)

44

(4.9%)

92

(10.3%)

894

(100.0%)

Pension, Etc.

560

(45.4%)

83

(6.7%)

590

(47.9%)

1,233

(100.0%)

Public Assistance

126

(56.3%)

16

(7.2%)

82

(36.6%)

224

(100.0%)

Other Income

441

(83.6%)

28

(5.4%)

58

(11.0%)

528

(100.0%)

PHA-administered Section 8

Asset Income

119

(88.4%)

7

(5.2%)

9

(6.4%)

134

(100.0%)

Child Care Expense

58

(71.5%)

8

(9.8%)

15

(18.6%)

81

(100.0%)

Disability Expense

4

(62.5%)

2

(37.5%)

6

(100.0%)

Medical Expense

204

(76.2%)

14

(5.1%)

50

(18.7%)

268

(100.0%)

Earned Income

175

(58.2%)

22

(7.2%)

104

(34.5%)

300

(100.0%)

Pension, Etc.

486

(47.8%)

55

(5.4%)

475

(46.8%)

1,017

(100.0%)

Public Assistance

61

(66.2%)

2

(1.9%)

29

(31.9%)

92

(100.0%)

Other Income

125

(66.1%)

10

(5.5%)

54

(28.4%)

189

(100.0%)

Owner-administered

Asset Income

88

(45.1%)

2

(1.1%)

105

(53.8%)

194

(100.0%)

Child Care Expense

3

(10.3%)

2

(5.0%)

29

(84.7%)

34

(100.0%)

August 31, 2016

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

317

(66.6%)

24

(5.1%)

135

(28.3%)

476

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 15k. Verification of Form HUD-50058/50059 Rent Components

Third-Party Verbal

No Verification

Verification

Dollar Amount Not Matched

Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component by Program Type

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

369

(96.8%)

3

(.7%)

10

(2.5%)

381

(100.0%)

Pension, Etc.

589

(100.0%)

589

(100.0%)

2015

Public Assistance

115

(100.0%)

115

(100.0%)

Other Income

202

(97.5%)

5

(2.5%)

207

(100.0%)

Public Housing

Asset Income

70

(100.0%)

70

(100.0%)

Child Care Expense

34

(94.9%)

2

(5.1%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

193

(100.0%)

193

(100.0%)

Earned Income

889

(99.4%)

5

(.6%)

894

(100.0%)

Pension, Etc.

1,231

(99.8%)

2

(.2%)

1,233

(100.0%)

Public Assistance

224

(100.0%)

224

(100.0%)

Other Income

515

(97.5%)

13

(2.5%)

528

(100.0%)

PHA-administered Section 8

Asset Income

134

(100.0%)

134

(100.0%)

Child Care Expense

81

(100.0%)

81

(100.0%)

Disability Expense

6

(100.0%)

6

(100.0%)

Medical Expense

268

(100.0%)

268

(100.0%)

Earned Income

293

(97.7%)

7

(2.3%)

300

(100.0%)

Pension, Etc.

1,015

(99.8%)

2

(.2%)

1,017

(100.0%)

Public Assistance

90

(98.1%)

2

(1.9%)

92

(100.0%)

Other Income

189

(100.0%)

189

(100.0%)

Appendix C: Source T

Owner-administered

Asset Income

194

(100.0%)

194

(100.0%)

Child Care Expense

34

(100.0%)

34

(100.0%)

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

475

(99.7%)

2

(.3%)

476

(100.0%)

2016.08.17  [Weighted]

C-23

ables

C-24

Appendix C: Source T

HUDQC FY 2015

Table 15l. Verification of Form HUD-50058/50059 Rent Components

Documentation

Verification

No Verification

Dollar Amount Not Matched Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component by Program Type

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

236

(61.8%)

55

(14.3%)

91

(23.9%)

381

(100.0%)

ables

Pension, Etc.

417

(70.8%)

22

(3.7%)

150

(25.5%)

589

(100.0%)

Public Assistance

97

(83.8%)

3

(2.7%)

16

(13.5%)

115

(100.0%)

Other Income

146

(70.3%)

12

(6.0%)

49

(23.7%)

207

(100.0%)

Public Housing

Asset Income

40

(57.2%)

5

(7.4%)

25

(35.4%)

70

(100.0%)

Child Care Expense

24

(67.1%)

6

(17.4%)

6

(15.5%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

144

(74.7%)

15

(7.6%)

34

(17.7%)

193

(100.0%)

Earned Income

270

(30.2%)

172

(19.3%)

452

(50.6%)

894

(100.0%)

Pension, Etc.

879

(71.3%)

51

(4.1%)

304

(24.6%)

1,233

(100.0%)

Public Assistance

133

(59.3%)

18

(7.9%)

73

(32.7%)

224

(100.0%)

Other Income

258

(48.9%)

103

(19.5%)

167

(31.6%)

528

(100.0%)

PHA-administered Section 8

Asset Income

44

(32.5%)

6

(4.7%)

84

(62.8%)

134

(100.0%)

Child Care Expense

26

(31.8%)

15

(18.5%)

40

(49.7%)

81

(100.0%)

Disability Expense

2

(37.5%)

4

(62.5%)

6

(100.0%)

Medical Expense

102

(38.0%)

53

(19.8%)

113

(42.1%)

268

(100.0%)

Earned Income

164

(54.8%)

55

(18.4%)

81

(26.9%)

300

(100.0%)

Pension, Etc.

689

(67.7%)

19

(1.9%)

309

(30.4%)

1,017

(100.0%)

Public Assistance

67

(72.9%)

5

(5.9%)

20

(21.2%)

92

(100.0%)

Other Income

145

(77.0%)

17

(8.8%)

27

(14.2%)

189

(100.0%)

Owner-administered

Asset Income

169

(86.8%)

1

(.7%)

24

(12.5%)

194

(100.0%)

August 31, 2016

Child Care Expense

30

(89.7%)

2

(5.2%)

2

(5.1%)

34

(100.0%)

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

322

(67.6%)

32

(6.6%)

123

(25.8%)

476

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 15m. Verification of Form HUD-50058/50059 Rent Components

EIV (Enterprise Income Verification)

Verification

No Verification

Dollar Amount Not Matched Dollar Amount Matched

Total

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

Rent Component by Program Type

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

381

(100.0%)

381

(100.0%)

Pension, Etc.

286

(48.6%)

51

(8.7%)

251

(42.6%)

589

(100.0%)

2015

Public Assistance

113

(98.0%)

2

(2.0%)

115

(100.0%)

Other Income

207

(100.0%)

207

(100.0%)

Public Housing

Asset Income

70

(100.0%)

70

(100.0%)

Child Care Expense

36

(100.0%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

144

(74.3%)

11

(5.9%)

38

(19.9%)

193

(100.0%)

Earned Income

894

(100.0%)

894

(100.0%)

Pension, Etc.

615

(49.9%)

64

(5.2%)

554

(44.9%)

1,233

(100.0%)

Public Assistance

223

(99.4%)

1

(.6%)

224

(100.0%)

Other Income

528

(100.0%)

528

(100.0%)

PHA-administered Section 8

Asset Income

134

(100.0%)

134

(100.0%)

Child Care Expense

81

(100.0%)

81

(100.0%)

Disability Expense

6

(100.0%)

6

(100.0%)

Medical Expense

224

(83.6%)

11

(4.3%)

33

(12.2%)

268

(100.0%)

Earned Income

300

(100.0%)

300

(100.0%)

Pension, Etc.

554

(54.5%)

31

(3.0%)

432

(42.5%)

1,017

(100.0%)

Public Assistance

92

(100.0%)

92

(100.0%)

Other Income

189

(100.0%)

189

(100.0%)

Appendix C: Source T

Owner-administered

Asset Income

194

(100.0%)

194

(100.0%)

Child Care Expense

34

(100.0%)

34

(100.0%)

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

459

(96.3%)

2

(.4%)

16

(3.3%)

476

(100.0%)

2016.08.17  [Weighted]

C-25

ables

C-26

Appendix C: Source T

HUDQC FY 2015

Table 15n. Verification of Form HUD-50058/50059 Rent Components

UIV (Upfront Income Verification)

Verification

No Verification

Total

Rent Component by Program Type

Dollar Amount Not Matched

Dollar Amount Matched

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

# of Cases

Row % of

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

(in 1,000s)

Cases

Earned Income

375

(98.5%)

3

(.8%)

3

(.8%)

381

(100.0%)

ables

Pension, Etc.

589

(100.0%)

589

(100.0%)

Public Assistance

67

(58.2%)

12

(10.6%)

36

(31.2%)

115

(100.0%)

Other Income

195

(94.1%)

4

(1.7%)

9

(4.2%)

207

(100.0%)

Public Housing

Asset Income

68

(96.5%)

2

(3.5%)

70

(100.0%)

Child Care Expense

36

(100.0%)

36

(100.0%)

Disability Expense

2

(100.0%)

2

(100.0%)

Medical Expense

192

(99.2%)

1

(.8%)

193

(100.0%)

Earned Income

855

(95.7%)

17

(1.9%)

22

(2.5%)

894

(100.0%)

Pension, Etc.

1,229

(99.7%)

4

(.3%)

1,233

(100.0%)

Public Assistance

158

(70.5%)

9

(4.1%)

57

(25.4%)

224

(100.0%)

Other Income

489

(92.6%)

16

(3.1%)

23

(4.3%)

528

(100.0%)

PHA-administered Section 8

Asset Income

130

(96.6%)

5

(3.4%)

134

(100.0%)

Child Care Expense

81

(100.0%)

81

(100.0%)

Disability Expense

6

(100.0%)

6

(100.0%)

Medical Expense

260

(97.3%)

2

(.9%)

5

(1.9%)

268

(100.0%)

Earned Income

283

(94.3%)

7

(2.3%)

10

(3.4%)

300

(100.0%)

Pension, Etc.

1,017

(100.0%)

1,017

(100.0%)

Public Assistance

78

(84.1%)

2

(1.9%)

13

(14.0%)

92

(100.0%)

Other Income

179

(94.7%)

3

(1.6%)

7

(3.7%)

189

(100.0%)

Owner-administered

Asset Income

191

(98.1%)

4

(1.9%)

194

(100.0%)

Child Care Expense

34

(100.0%)

34

(100.0%)

August 31, 2016

Disability Expense

4

(100.0%)

4

(100.0%)

Medical Expense

470

(98.6%)

7

(1.4%)

476

(100.0%)

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015

Table 16a. QC Rent Component for Households With QC Rent Error (>$5)

Form HUD-50058

Form HUD-50059

Total

# of Households

# of Households

# of Households

Rent Component

(in 1,000s)

% of Households

(in 1,000s)

% of Households

(in 1,000s)

% of Households

No Error

2,882

(88.1%)

1,299

(94.0%)

4,181

(89.8%)

Earned Income

With Error

389

(11.9%)

83

(6.0%)

473

(10.2%)

No Error

3,044

(93.1%)

1,300

(94.1%)

4,345

(93.4%)

Pension, Etc.

With Error

227

(6.9%)

82

(5.9%)

309

(6.6%)

2015

No Error

3,211

(98.2%)

1,374

(99.4%)

4,585

(98.5%)

Public Assistance

With Error

60

(1.8%)

9

(.6%)

68

(1.5%)

No Error

3,060

(93.6%)

1,339

(96.9%)

4,400

(94.5%)

Other Income

With Error

211

(6.4%)

43

(3.1%)

254

(5.5%)

No Error

3,205

(98.0%)

1,335

(96.6%)

4,540

(97.6%)

Asset Income

With Error

66

(2.0%)

47

(3.4%)

113

(2.4%)

No Error

3,231

(98.8%)

1,377

(99.6%)

4,608

(99.0%)

Child Care Expense

With Error

40

(1.2%)

5

(.4%)

45

(1.0%)

No Error

3,271

(100.0%)

1,382

(100.0%)

4,653

(100.0%)

Disability Expense

With Error

No Error

3,067

(93.8%)

1,271

(92.0%)

4,338

(93.2%)

Medical Expense

With Error

204

(6.2%)

111

(8.0%)

315

(6.8%)

No Error

2,446

(74.8%)

1,107

(80.1%)

3,553

(76.3%)

All Components

With Error

825

(25.2%)

276

(20.0%)

1,101

(23.7%)

Total

3,271

(100.0%)

1,382

(100.0%)

4,653

(100.0%)

Appendix C: Source T

2016.08.17  [Weighted]

C-27

ables

C-28

Appendix C: Source T

HUDQC FY 2015

Table 16b. QC Error Households With Missing Verification in Tenant File

Form HUD-50058

Form HUD-50059

Total

# of Households

# of Households

# of Households

Rent Component

(in 1,000s)

% of Households

(in 1,000s)

% of Households

(in 1,000s)

% of Households

Verified

202

(51.9%)

52

(62.5%)

254

(53.8%)

Earned Income

Not Verified

187

(48.1%)

31

(37.5%)

218

(46.2%)

ables

Verified

157

(69.4%)

59

(71.3%)

216

(69.9%)

Pension, Etc.

Not Verified

69

(30.6%)

24

(28.7%)

93

(30.1%)

Verified

20

(33.8%)

20

(29.6%)

Public Assistance

Not Verified

40

(66.2%)

9

(100.0%)

48

(70.4%)

Verified

113

(53.5%)

14

(33.0%)

127

(50.0%)

Other Income

Not Verified

98

(46.5%)

29

(67.0%)

127

(50.0%)

Verified

28

(43.2%)

30

(63.1%)

58

(51.6%)

Asset Income

Not Verified

37

(56.8%)

17

(36.9%)

55

(48.4%)

Verified

24

(59.0%)

3

(69.0%)

27

(60.1%)

Child Care Expense

Not Verified

16

(41.0%)

2

(31.0%)

18

(39.9%)

Verified

Disability Expense

Not Verified

Verified

44

(21.3%)

40

(36.1%)

84

(26.5%)

Medical Expense

Not Verified

161

(78.7%)

71

(63.9%)

232

(73.5%)

2016.08.17  [Weighted]

August 31, 2016

HUDQC Final Report for FY

HUDQC FY 2015

Table 17a. Administrative Error: Number and Percent of Households, Average Dollars in Error

For Non-MTW Households With Recalculated Form HUD-50058/50059 Rent Error by Administrative Error Type

Non-MTW Households with Recalculated Form HUD-50058/50059 Rent Error

Error Type

# of Households in Error

% of Households in Error

Average Gross Dollar Error

Transcription Error

144

(49.3%)

11.70

No Transcription Error

148

(50.7%)

14.50

Consistency Error

46

(15.6%)

15.42

No Consistency Error

247

(84.4%)

12.70

2015

Allowances Calculation Error

5

(1.8%)

69.72

No Allowances Calculation Error

287

(98.2%)

12.10

Income Calculation Error

9

(3.2%)

40.06

No Income Calculation Error

283

(96.8%)

12.24

Other Calculation Error

21

(7.0%)

19.66

No Other Calculation Error

272

(93.0%)

12.63

Overdue Recertification

6

(2.1%)

5.37

On-time Recertification

229

(78.1%)

12.13

Certification

58

(19.8%)

17.88

Any Administrative/Procedural Error

166

(56.7%)

10.88

No Administrative/Procedural Error

127

(43.3%)

16.06

Total Households

293

(100.0%)

13.12

Note: Data presented above exclude Moving to Work households.

2016.08.17  [Weighted]

Appendix C: Source T

C-29

ables

C-30

Appendix C: Source T

HUDQC FY 2015

Table 17b. Administrative Error: Number and Percent of Households, Average Dollars in Error

For Households With QC Rent Error by Administrative Error Type

Households with QC Rent Error

Error Type

# of Households in Error

% of Households in Error

Average Gross Dollar Error

Transcription Error

1,002

(81.7%)

50.72

No Transcription Error

224

(18.3%)

51.16

ables

Consistency Error

270

(22.0%)

45.95

No Consistency Error

956

(78.0%)

52.17

Allowances Calculation Error

24

(2.0%)

68.36

No Allowances Calculation Error

1,202

(98.0%)

50.45

Income Calculation Error

39

(3.2%)

93.81

No Income Calculation Error

1,188

(96.8%)

49.39

Other Calculation Error

75

(6.1%)

89.09

No Other Calculation Error

1,151

(93.9%)

48.30

Overdue Recertification

49

(4.0%)

112.67

On-time Recertification

1,013

(82.6%)

48.69

Certification

165

(13.5%)

45.48

Any Administrative/Procedural Error

1,047

(85.4%)

51.18

No Administrative/Procedural Error

179

(14.6%)

48.63

Total Households

1,227

(100.0%)

50.80

2016.08.17  [Weighted]

August 31, 2016

HUDQC Final Report for FY

HUDQC FY 2015

Table 18. Administrative Error: Number and Percent of Households, Average Dollars in Error

For All Households by Administrative Error Type

Gross QC Rent Error

Net QC Rent Error

Average Dollar

Average Dollar

Error Type

# of Households

% of Households

Error

# of Households

% of Households

Error

Transcription Error

1,949

(41.9%)

26.08

1,949

(41.9%)

-8.25

No Transcription Error

2,704

(58.1%)

4.25

2,704

(58.1%)

-1.79

Consistency Error

976

(21.0%)

12.73

976

(21.0%)

-4.69

2015

No Consistency Error

3,677

(79.0%)

13.57

3,677

(79.0%)

-4.44

Allowances Calculation Error

52

(1.1%)

31.69

52

(1.1%)

-23.44

No Allowances Calculation Error

4,601

(98.9%)

13.18

4,601

(98.9%)

-4.28

Income Calculation Error

108

(2.3%)

33.89

108

(2.3%)

-15.49

No Income Calculation Error

4,545

(97.7%)

12.90

4,545

(97.7%)

-4.23

Other Calculation Error

171

(3.7%)

39.38

171

(3.7%)

8.57

No Other Calculation Error

4,483

(96.3%)

12.40

4,483

(96.3%)

-4.99

Overdue Recertification

72

(1.5%)

76.64

72

(1.5%)

23.22

On-time Recertification

3,988

(85.7%)

12.36

3,988

(85.7%)

-4.94

Certification

594

(12.8%)

12.65

594

(12.8%)

-4.81

Any Administrative/Procedural Error

2,479

(53.3%)

21.62

2,479

(53.3%)

-7.04

No Administrative/Procedural Error

2,174

(46.7%)

4.01

2,174

(46.7%)

-1.58

Total

4,653

(100.0%)

13.39

4,653

(100.0%)

-4.49

2016.08.17  [Weighted]

Appendix C: Source T

C-31

ables

C-32

Appendix C: Source T

HUDQC FY 2015

Table 19. Occupancy Standards on Form HUD-50058/50059

Public Housing

PHA-administered Section 8

Owner-administered

Total

# of

# of

# of

# of

Number of Bedrooms by

Households

% of

Households

% of

Households

% of

Households

% of

Occupancy Standard

(in 1,000s)

Households

(in 1,000s)

Households

(in 1,000s)

Households

(in 1,000s)

Households

0

1

(1.3%)

3

(3.5%)

2

(2.5%)

6

(2.4%)

1

2

(.6%)

1

(.2%)

5

(.6%)

9

(.5%)

ables

2

6

(1.9%)

23

(2.9%)

5

(1.9%)

33

(2.5%)

Under-Housed

3

5

(2.3%)

23

(3.7%)

2

(1.6%)

30

(3.1%)

4

2

(3.2%)

2

(.9%)

5+

2

(7.6%)

2

(5.1%)

All Units

16

(1.5%)

52

(2.4%)

14

(1.0%)

82

(1.8%)

0

82

(98.7%)

70

(96.5%)

78

(97.5%)

230

(97.6%)

1

357

(99.4%)

592

(99.8%)

890

(99.4%)

1,838

(99.5%)

2

221

(73.2%)

536

(68.0%)

192

(74.9%)

950

(70.5%)

Correct

3

186

(78.3%)

440

(71.6%)

97

(81.3%)

722

(74.4%)

4

31

(58.9%)

66

(65.2%)

17

(56.3%)

115

(61.9%)

5+

1

(8.1%)

8

(29.3%)

9

(22.1%)

All Units

878

(83.9%)

1,712

(77.9%)

1,274

(92.1%)

3,864

(83.5%)

0

1

2

75

(24.8%)

229

(29.1%)

59

(23.2%)

364

(27.0%)

Over-Housed

3

46

(19.4%)

152

(24.7%)

20

(17.1%)

218

(22.5%)

4

20

(37.9%)

35

(34.8%)

13

(43.7%)

69

(37.2%)

5+

11

(91.9%)

18

(63.1%)

1

(100.0%)

31

(72.7%)

All Units

152

(14.6%)

434

(19.7%)

95

(6.8%)

681

(14.7%)

2016.08.17  [Weighted]

August 31, 2016

HUDQC Final Report for FY

HUDQC FY 2015

Table 19a. Frequency and Percent of All Households

by Number of Bedrooms and Number of Household Members

Number of Household Members

Number of

1

2

3

4

5

6

7

8

9

10

11

Bedrooms

N

%

N

%

N

%

N

%

N

%

N

%

N

%

N

%

N

%

N

%

N

%

0

230 97.6%

3 1.3%

3 1.1%

1

1675 90.7%

163 8.8%

5 0.3%

2 0.1%

2 0.1%

2

364 27.0%

579 43.0%

273 20.3%

98 7.3%

24 1.8%

7 0.5%

2 0.2%

2015

3

88 9.0%

130 13.4%

318 32.8%

237 24.4%

114 11.8%

53 5.5%

23 2.4%

6 0.6%

2 0.2%

4

5 2.5%

9 4.6%

16 8.5%

40 21.5%

50 27.0%

33 17.7%

18 9.5%

14 7.7%

2 0.9%

5+

1 3.4%

2 5.9%

4 9.2%

15 35.8%

8 18.4%

6 13.2%

3 6.6%

1 2.4%

2 5.1%

2016.08.17  [Weighted]

Appendix C: Source T

C-33

ables

Source Tables Based on Tenant File Data

Source Tables Based on Tenant File Data

Throughout these Source Tables, empty cells indicate that either the result was zero

or the analysis was not applicable.

The following tables are based solely on data collected from tenant files.

HUDQC Final Report for FY

HUDQC FY 2015 [Tenant File]

Table 2. Percent of Households by Payment Type and Program Type

Underpayment

Proper Payment

Overpayment

Total

# of

# of

# of

# of

Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of Program Type

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

Public Housing

139

(13.1%)

(23.9%)

811

(76.4%)

(23.4%)

112

(10.5%)

(18.6%)

1,062 (100.0%)

(22.8%)

PHA-administered

Section 8

305

(13.8%)

(52.4%)

1,620

(73.3%)

(46.7%)

284

(12.8%)

(47.2%)

2,209 (100.0%)

(47.5%)

Total

444

(13.6%)

(76.2%)

2,431

(74.3%)

(70.1%)

395

(12.1%)

(65.8%)

3,271 (100.0%)

(70.3%)

2015

Owner-administered

139

(10.0%)

(23.8%)

1,038

(75.1%)

(29.9%)

206

(14.9%)

(34.2%)

1,382 (100.0%)

(29.7%)

Owner-administered

Total

139

(10.0%)

(23.8%)

1,038

(75.1%)

(29.9%)

206

(14.9%)

(34.2%)

1,382 (100.0%)

(29.7%)

Total

583

(12.5%) (100.0%)

3,469

(74.6%) (100.0%)

601

(12.9%) (100.0%)

4,653 (100.0%) (100.0%)

2016.08.17  [Weighted]

HUDQC FY 2015 [Tenant File]

Table 2(S). Percent of Households by Payment Type and Program Type

(Proper Payment Based on Exact Match of Actual and DC Rent)

Underpayment

Proper Payment

Overpayment

Total

# of

# of

# of

# of

Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of  Cases   Row % of  Col. % of Program Type

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

(in 1,000)

Cases

Cases

Public Housing

196

(18.5%)

(24.0%)

690

(65.0%)

(23.3%)

176

(16.6%)

(20.0%)

1,062

(100.0%) (22.8%)

PHA-administered

Section 8

405

(18.3%)

(49.5%)

1,367

(61.9%)

(46.3%)

437

(19.8%)

(49.7%)

2,209

(100.0%) (47.5%)

Total

601

(18.4%)

(73.5%)

2,056

(62.9%)

(69.6%)

614

(18.8%)

(69.7%)

3,271

(100.0%) (70.3%)

Owner-administered

217

(15.7%)

(26.5%)

899

(65.0%)

(30.4%)

267

(19.3%)

(30.3%)

1,382

(100.0%) (29.7%)

Appendix C: Source T

Owner-administered

Total

217

(15.7%)

(26.5%)

899

(65.0%)

(30.4%)

267

(19.3%)

(30.3%)

1,382

(100.0%) (29.7%)

Total

818

(17.6%) (100.0%)

2,955

(63.5%) (100.0%)

881

(18.9%) (100.0%)

4,653

(100.0%) (100.0%)

2016.08.17  [Weighted]

C-35

ables

C-36

Appendix C: Source T

HUDQC FY 2015 [Tenant File]

Table 3. Dollar Rent Error by Program Type

Actual Rent (Monthly)

C Rent (Monthly)

Gross Rent Error (Monthly)

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Sum Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount   Avg. Dollar

Program Type

(in 1,000) Cases (in 1,000) Amount (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases

(in 1,000)

Amount

Public Housing

1,062

(22.8%)

292,018

275.05

1,062

(22.8%)

293,477

276.42

1,062

(22.8%)

15,188

14.31

ables

PHA-administered

Section 8

2,209

(47.5%)

586,239

265.35

2,209

(47.5%)

592,732

268.29

2,209

(47.5%)

34,158

15.46

Total

3,271

(70.3%)

878,258

268.50

3,271

(70.3%)

886,209

270.93

3,271

(70.3%)

49,346

15.09

Owner-administered

1,382

(29.7%)

340,759

246.49

1,382

(29.7%)

338,366

244.76

1,382

(29.7%)

15,480

11.20

Owner-administered Total

1,382

(29.7%)

340,759

246.49

1,382

(29.7%)

338,366

244.76

1,382

(29.7%)

15,480

11.20

Total

4,653 (100.0%) 1,219,016

261.96

4,653 (100.0%) 1,224,575

263.15

4,653 (100.0%)

64,826

13.93

2016.08.17  [Weighted]

HUDQC FY 2015 [Tenant File]

Table 4. Dollar Error Amount by Payment Type and Program Type

Underpayment (Monthly)

Overpayment (Monthly)

DC Rent (Monthly)

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Sum Dollar

Avg.

Cases   Col. % of  Amount

Dollar

Cases  Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Program Type

(in 1,000) Cases

(in 1,000) Amount (in 1,000) Cases

(in 1,000) Amount (in 1,000) Cases

(in 1,000) Amount

Public Housing

139

(23.9%)

8,346

60.02

112

(18.6%)

6,842

61.30

1,062

(22.8%)

293,477

276.42

PHA-administered

Section 8

305

(52.4%)

20,338

66.61

284

(47.2%)

13,820

48.72

2,209

(47.5%)

592,732

268.29

Total

444

(76.2%)

28,684

64.55

395

(65.8%)

20,662

52.27

3,271

(70.3%)

886,209

270.93

Owner-administered

139

(23.8%)

6,521

47.06

206

(34.2%)

8,959

43.51

1,382

(29.7%)

338,366

244.76

Owner-administered Total

139

(23.8%)

6,521

47.06

206

(34.2%)

8,959

43.51

1,382

(29.7%)

338,366

244.76

Total

583 (100.0%)

35,205

60.39

601 (100.0%)

29,621

49.27

4,653 (100.0%) 1,224,575

263.15

August 31, 2016

2016.08.17  [Weighted]

HUDQC Final Report for FY

HUDQC FY 2015[Tenant File]

Table 4(S). Dollar Error Amount by Payment Type and Program Type

(Proper Payment Based on Exact Match of Actual and DC Rent)

Underpayment (Monthly)

Overpayment (Monthly)

DC Rent (Monthly)

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Sum Dollar  Avg.

Cases   Col. % of  Amount   Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount

Public Housing

196

(24.0%)

8,483

43.30

176

(20.0%)

6,976

39.60

1,062

(22.8%)

293,477

276.42

PHA-administered

Section 8

405

(49.5%)

20,596

50.84

437

(49.7%)

14,103

32.24

2,209

(47.5%)

592,732

268.29

2015

Total

601

(73.5%)

29,078

48.38

614

(69.7%)

21,079

34.35

3,271

(70.3%)

886,209

270.93

Owner-administered

217

(26.5%)

6,730

31.07

267

(30.3%)

9,122

34.14

1,382

(29.7%)

338,366

244.76

Owner-administered Total

217

(26.5%)

6,730

31.07

267

(30.3%)

9,122

34.14

1,382

(29.7%)

338,366

244.76

Total

818 (100.0%)

35,808

43.80

881 (100.0%)

30,201

34.29

4,653 (100.0%) 1,224,575

263.15

2016.08.17  [Weighted]

HUDQC FY 2015 [Tenant File]

Table 5. Gross and Net Rent Error by Program Type

Gross Rent Error (Monthly)

Net Rent Error (Monthly)

DC Rent (Monthly)

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Sum Dollar  Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount (in  Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

1,000)

Amount

Public Housing

1,062

(22.8%)

15,188

14.31

1,062

(22.8%)

-1,504

-1.42

1,062

(22.8%)

293,477

276.42

PHA-administered

Section 8

2,209

(47.5%)

34,158

15.46

2,209

(47.5%)

-6,518

-2.95

2,209

(47.5%)

592,732

268.29

Total

3,271

(70.3%)

49,346

15.09

3,271

(70.3%)

-8,022

-2.45

3,271

(70.3%)

886,209

270.93

Appendix C: Source T

Owner-administered

1,382

(29.7%)

15,480

11.20

1,382

(29.7%)

2,437

1.76

1,382

(29.7%)

338,366

244.76

Owner-administered Total

1,382

(29.7%)

15,480

11.20

1,382

(29.7%)

2,437

1.76

1,382

(29.7%)

338,366

244.76

Total

4,653 (100.0%)

64,826

13.93

4,653 (100.0%)

-5,585

-1.20

4,653 (100.0%) 1,224,575

263.15

2016.08.17  [Weighted]

C-37

ables

C-38

Appendix C: Source T

HUDQC FY 2015 [Tenant File]

Table 5(S). Gross and Net Rent Error by Program Type

(Proper Payment Based on Exact Match of Actual and DC Rent)

Gross Rent Error (Monthly)

Net Rent Error (Monthly)

DC Rent (Monthly)

Sum

Sum

Sum

# of

Dollar

Avg.

# of

Dollar

Avg.

# of

Dollar

Avg.

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount

Dollar

Cases   Col. % of  Amount   Dollar

Program Type

(in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount (in 1,000)

Cases

(in 1,000) Amount

ables

Public Housing

1,062

(22.8%)

15,458

14.56

1,062

(22.8%)

-1,507

-1.42

1,062

(22.8%)

293,477

276.42

PHA-administered

Section 8

2,209

(47.5%)

34,699

15.71

2,209

(47.5%)

-6,493

-2.94

2,209

(47.5%)

592,732

268.29

Total

3,271

(70.3%)

50,157

15.33

3,271

(70.3%)

-8,000

-2.45

3,271

(70.3%)

886,209

270.93

Owner-administered

1,382

(29.7%)

15,852

11.47

1,382

(29.7%)

2,392

1.73

1,382

(29.7%)

338,366

244.76

Owner-administered Total

1,382

(29.7%)

15,852

11.47

1,382

(29.7%)

2,392

1.73

1,382

(29.7%)

338,366

244.76

Total

4,653 (100.0%)

66,009

14.18

4,653 (100.0%)

-5,607

-1.21

4,653 (100.0%) 1,224,575

263.15

2016.08.17  [Weighted]

August 31, 2016

Appendix D: Consistency and Calculation Errors

Appendix D: Consistency and Calculation Errors

Exhibit D-1

Form HUD-50058—Consistency Errors

Form HUD-50058 Item

Error

General Information

1c.

Program

Must equal P, CE, VO, or MR

2a.

Type of Action

Must equal 1 through 15

2b.

Effective Date of Action

Cannot be earlier than Date of Admission to the Program (2h)

Household Composition

3g.

Sex

Must equal M or F

3h.

Relationship

Must equal H, S, K, F, Y, E, L, or A

3i.

Citizenship

Must equal EC, EN, IN, PV

3k.

Race

Must equal 1 through 5

3m.  Ethnicity

Must equal 1 or 2

3u.

Family Subsidy Status

Must equal C, E, F, P

3v.

Effective Date

Should not be blank if 3u equals C

Net Family Assets and Income

6a.

Family Member No.

Must equal a number used in Section 3. Household

7a.

Family Member No.

Must equal a number used in Section 3. Household

7b.

Income Code

Must equal B, F, HA, M, W, G, IW, T, P, S, SS, C, E, I, N, or U

8a.

Total Annual Income

Must equal Total Annual Income recorded in 7i

8i.

Earnings Made Possible by

Must be ≤ the sum of Dollars per Year (7d) for Income Codes (7b) HA,

Disability Assistance Expense

F, W, B, or M

Allowances and Adjusted Income

8h.

Maximum Disability Allowance

Should only be completed if any member is disabled

•  Should be ≤ Maximum Disability Allowance (8h)

8j.

Allowable Disability Assistance

•  Should be 0 if Medical/Disability Threshold (8f) is > Maximum Dis-

Expense

ability Allowance (8h)

•  Should be 0 or blank if Maximum Disability Allowance (8h) is 0

or blank

8k.

Total Medical Expenses

Should only be completed if the head, spouse, or co-head is 62 or

over, or disabled; otherwise it should be blank

•  Should equal Total Annual Disability Assistance and Medical

Expense (8m) minus Medical/Disability Threshold (8f) if Allowable

Disability Assistance Expense (8j) is blank or if the Total Annual

Unreimbursed Disability Assistance Expense (8g) is less than the

8n.

Medical/Disability Assistance

Medical/Disability Threshold (8f)

Allowance

•  Should equal Total Annual Disability Assistance and Medical

Expense (8m) if Total Annual Unreimbursed Disability Assistance

Expense (8g) and Allowable Disability Assistance Expense (8j) is ≥

Medical/Disability Threshold (8f)

8p.

Elderly/Disabled Allowance

Should be $400 if head, spouse, or co-head is 62 or over, or disabled;

otherwise it should be 0 or blank

Must be completed if the household contains a member under age 18,

8s.

Dependent Allowance

a member who is disabled, or a full-time student (excluding the head,

spouse, foster child, or adult, or live-in attendant)

HUDQC Final Report for FY 2015

D-1

Appendix D: Consistency and Calculation Errors

Exhibit D-1

Form HUD-50058—Consistency Errors, continued

Form HUD-50058 Item

Error

8t.

Yearly Child Care Cost That Is

Not Reimbursed (Child Care

Should only be completed if any member is less than 13 years old

Allowance)

Family Rent and Subsidy Information

10a.  TTP (Public Housing and Turnkey

III)

11q.  TTP (Section 8: Project Based

Certificates and Vouchers)

12r.  TTP (Housing Choice Vouchers:

Tenant Based Vouchers)

Items 10a, 11q, 12r, 13j, or 14s must equal TTP (9j) or be blank

13j.  TTP (Section 8: Moderate Reha-

bilitation [Mod Rehab])

14s.  TTP (Manufactured Home Owner

Renting the Space)

•  If Program (1c) = P:

o  TTP (10a) must be completed

o  Flat Rent (10b), Tenant Rent (10f), or Mixed Family Tenant Rent

(10s) must be completed

o  Sections 11 through 14 must be blank

•  If Program (1c) = VO or C:

o  Section 11 or 12 must be completed

Rent Calculations (item numbers include

10a through 14ag)

o  Tenant Rent (11s or 12k) or Mixed Family Tenant Rent (11ak or

12ai) must be completed

o  Sections 10, 13, and 14 must be blank

•  If Program (1c) = MR:

o  Contract Rent to Owner must be completed

o  Tenant Rent (13k) or Mixed Family Tenant Rent (13x) must

be completed

o  Sections 10, 11, 12, and 14 must be blank

D-2

August 31, 2016

Appendix D: Consistency and Calculation Errors

Exhibit D-2

Form HUD-50058 MTW*—Consistency Errors

Form HUD-50058 MTW Item

Error

General Information

1c.

Program

Must equal P, PR, or T

2a.

Type of Action

Must equal 1 through 15

2b.

Effective Date of Action

Cannot be earlier than Date of Admission to the Program (2h)

Household Composition

3g.

Sex

Must equal M or F

3h.

Relationship

Must equal H, S, K, F, Y, E, L, or A

3i.

Citizenship

Must equal EC, EN, IN, PV

3k.

Race

Must equal 1 through 5

3m.  Ethnicity

Must equal 1 or 2

3u.

Family Subsidy Status

Must equal C, E, F, P

3v.

Eligibility Effective Date

Should not be blank if 3u equals C

Net Family Assets and Income

18a.  Family Member No.

Must equal a number used in Section 3. MTW Household

19a.  Family Member No.

Must equal a number used in Section 3. MTW Household

19b.  Income Code

Must equal B, F, HA, M, W, G, IW, T, P, S, SS, C, E, I, N, U, or X

*For the purpose of the study, we implemented a Moving to Work (MTW) exception if a case was flagged as using Form

HUD-50058 MTW. As a result, there were 112 MTW cases (representing 27 projects) that did not use Form HUD-50058 MTW, but did adhere to MTW policies. There were no non-MTW cases that used Form HUD-50058 MTW.

HUDQC Final Report for FY 2015

D-3

Appendix D: Consistency and Calculation Errors

Exhibit D-3

Form HUD-50059—Consistency Errors

Form HUD-50059 Item

Error

General Information

2.

Subsidy Type

Must equal 1 through 9

13.

Effective Date

Cannot be earlier than Date Tenant Moved into Project (16)

18.

Certification Type

Must equal 1 through 5

19.

Action Processed

Must equal 1 through 4, or blank

40.

Race of Head of Household

Must equal 1 through 4

41.

Ethnicity of Head of Household

Must equal 1 or 2

Household Composition

39.

Sex

Must equal M or F

44.

Special Status Code

Must equal E, S, H, F, I, J, or blank; should be E if Age > 61

46.

Eligibility Code (Citizenship)

Must equal EC, EN, IC, IN, IP, PV, or XX

Net Family Assets and Income

66.

Member No.—Income Info

Should not be greater than the total number of members listed in item

75.

Member No.—Asset Info

34 (Family Member Number)

Allowances and Adjusted Income

97.

Deduction for Dependents

Must be completed if Number of Dependents (55) is greater than 0

98.

Child Care Expense (work)

Should only be completed if any member is less than 13 years old

99.

Child Care Expense (school)

•  Should be ≤ Disability Expenses (101)

102.  Disability Allowance

•  Should be 0 if 3% of Annual Income (100) is > Total Disability

Assistance Expenses (101)

•  Should be 0 or blank if Total Disability Expenses (101) is 0 or blank

Should only be completed if the Special Status Code (43) for the head,

103.  Total Medical Expenses

spouse, or co-head = H or E, or if the head, spouse, or co-head is 62

years old or older

105.  Elderly Household Allowance

Should be $400 if the Special Status Code (43) for the head or spouse

or co-head = H or E; otherwise it should be 0 or blank

Family Rent and Subsidy Information

109.  Tenant Rent

Should equal the maximum of TTP (108) minus the Utility Allowance

(32) or 0, or be blank if the Utility Reimbursement (110) > 0

110.  Utility Reimbursement

Should be blank if Item 32 < Item 108

D-4

August 31, 2016

Exhibit D-4

Form HUD-50058—Calculation Errors

Form HUD-50058 Item

Error Calculation

Household Composition

3f.

Age

Must equal the age calculated based on Date of Birth (3e) and Effective

Date of Action (2b)

Must equal the number of household members who are under age 18,

8q.

Number of Dependents

have a disability, or are full-time students (other than head, spouse,

co-head, foster child/adult, or live-in aide)

Net Family Assets and Income

6f.

Total Asset Value

Must equal the sum of all values in Cash Value of Asset (6d)

Must equal Total Cash Value of Asset (6f) times the Passbook Rate (6h) if

6i.

Imputed Asset Income

Total Value of Assets (6f) is > $5,000. If Total Value of Assets (6f) is

≤ $5,000 Imputed Asset Income (6i) = 0

6j.

Income From Asset

Must equal the larger of Total Anticipated Income (6g) or Imputed Asset

Income (6i)

7g.

Total Non-Asset Income

Must equal the sum of all values in Income After Exclusions (7f)

7i.

Total Annual Income

Must equal Final Asset Income (6j) + Total Income Other Than

Assets (7g)

Allowances and Adjusted Income

8e.

Total Permissible Deductions

Must equal the sum of all values in Amount of Permissible Deduction (8d)

8f.

3% of Annual Income

Must equal 3% * Total Annual Income (8a)

Must equal Total Annual Unreimbursed Disability Assistance Expense

8h.

Disability Allowance

(8g) minus Medical/Disability Threshold (8f) if there is a disabled

household member and an earned income greater than or equal to the

disability expense

Must equal Total Annual Disability Assistance and Medical Expense (8m)

minus Medical/Disability Threshold (8f) if Allowable Disability Assistance

Expense (8j) is blank or Total Annual Unreimbursed Disability Assistance

8n.

Medical Allowance

Expense (8g) is less than Medical/Disability Threshold (8f); or equal

Total Annual Disability Assistance and Medical Expense (8m) if Total

Annual Unreimbursed Disability Assistance Expense (8g) and Allowable

Disability Assistance Expense (8j) is ≥ Medical/Disability Threshold (8f); if

the head, spouse, or co-head is elderly or disabled

8p.

Elderly/Disabled Allowance

Must equal $400 if head, spouse, or co-head is elderly or disabled

8s.

Dependent Allowance

Must equal Number of Dependents (8q) * $480

8t.

Child Care Costs

Must be 0 or blank if no household member is under age 13

Must equal Total Permissible Deductions (8e) + Medical/Disability

8x.

Total Allowance

Assistance Allowance (8n) + Elderly/Disability Allowance (8p) +

Dependent Allowance (8s) + Total Annual Unreimbursed Child Care

Costs (8t) + Total Annual Travel Cost to Work/School (8u)

8y.

Adjusted Annual Income

Must equal Total Annual Income (8a) minus Total Allowances (8x)

HUDQC Final Report for FY 2015

D-5

Appendix D: Consistency and Calculation Errors

Exhibit D-4

Form HUD-50058—Calculation Errors, continued

Form HUD-50058 Item

Error Calculation

Family Rent and Subsidy Information

Must equal the highest of TTP if Based on Annual Income (9c), TTP if

9j.

Total Tenant Payment

Based on Adjusted Annual Income (9f), Welfare Rent (9g), Minimum Rent

(9h), or Enhanced Voucher Minimum Rent (9i)

12p.  Gross Rent

Must equal Rent to Owner (12k) + Utility Allowance (12m)

Tenant Rent (item number varies

Tenant Rent must equal the recalculated tenant rent based on the Rent

by program)

Calculation rules provided in Appendix A

Note: With the exception of tenant rent, negative numbers are always converted to 0.

Exhibit D-5

Form HUD-50058 MTW—Calculation Errors

Form HUD-50058 MTW Item

Error Calculation

Household Composition

3f.

Age

Must equal the age calculated based on Date of Birth (3e) and Effective

Date of Action (2b)

Net Family Assets and Income

18f.  Total Asset Value

Must equal the sum of all values in Cash Value of Asset (18d)

Must equal Total Cash Value of Asset (18f) times the Passbook Rate

18i.  Imputed Asset Income

(18h) if Total Value of Assets (18f) is > $5,000. If Total Value of Assets

(18f) is ≤ $5,000 Imputed Asset Income (18i) = 0

18j.  Income From Asset

Must equal the larger of Total Anticipated Income (18g) or Imputed Asset

Income (18i)

19h.  Total Non-Asset Income

Must equal the sum of all values in Income After Exclusions (19f)

19i.  Total Annual Income

Must equal Final Asset Income (18j) + Total Income Other Than

Assets (19h)

Allowances and Adjusted Income

19k.  Adjusted Annual Income

Must equal Total Annual Income (19i) minus Total Deductions (19j)

Family Rent and Subsidy Information

21k.  Gross Rent

Must equal Rent to Owner (21i) + Utility Allowance/estimate (21j)

Note: With the exception of tenant rent, negative numbers are always converted to 0.

Exhibit D-6

Form HUD-50059—Calculation Errors

Form HUD-50059 Item

Error Calculation

Household Composition

48.

Age

Must equal the age calculated based on Date of Birth (42) and Effective

Date of Action (13)

53.

Number of Family Members

Must equal the number of family members listed

54.

Number of Non-Family Mem-

Must equal the number of family members listed with a relationship code

bers

of “L” or “F”

Must equal the number of household members who are under age 18,

55.

Number of Dependents

have a disability, or are full-time students (other than head, spouse,

co-head, foster child/adult, or live-in aide)

D-6

August 31, 2016

Exhibit D-6

Form HUD-50059—Calculation Errors, continued

Form HUD-50059 Item

Error Calculation

Net Family Assets and Income

81.

Total Asset Value

Must equal the sum of the asset values in Cash Value of Assets (78)

82.

Actual Income From Asset

Must equal the sum of the income values in Actual Yearly Income From

Assets (79)

84.

Imputed Asset Income

Must equal Total Asset Value (81) * 2%, if Total Value of Assets is >

$5,000

70.

Earned Income Sum

Must equal the sum of income values (in item 68) for items with codes B,

F, M, or W in Income Type Code (67)

71.

Pension Income Sum

Must equal the sum of the income values (in item 68) for items with codes

PE, SI, or SS in Income Type Code (67)

72.

Public Assistance Income Sum

Must equal the sum of the income values (in item 68) for items with codes

TA or G in Income Type Code (67)

73.

Other Income Sum

Must equal the sum of the income values (in item 68) for items with codes

CS, I, N, or U in Income Type Code (67)

74.

Total Non-Asset Income

Must equal Earned Income Sum (70) + Pension Income Sum (71) +

Public Assistance Income Sum (72) + Other Income Sum (73)

85.

Asset Income

Must equal the greater of Imputed Asset Income (84) or Actual Income

from Asset (82)

86.

Total Annual Income

Must equal Total Non-Asset Income (74) + Income from Asset (85)

Allowances and Adjusted Income

97.

Dependent Allowance

Must equal Number of Dependents (55) * $480

98.

Child Care Expense (work)

Must be 0 or blank if no household member is under age 13

99.

Child Care Expense (school)

100.  3% of Annual Income

Must equal Total Annual Income (86) * .03

Must equal Total Disability Expenses (101) minus 3% of Annual Income

102.  Disability Allowance

(100) if there is a disabled household member and if there is earned

income greater than or equal to the disability expense

Must equal Total Medical Expenses (103) minus 3% of Annual Income

(100) if Total Disability Expense (101) = 0; or if (Disability Deduction (102)

104.  Medical Allowance

= 0, then Medical Deduction (104) = Total Medical Expenses (103) +

Total Disability Expenses (101) - 3% of Annual Income (86), if the head,

spouse, or co-head is elderly or disabled

105.  Elderly Household Allowance

Must equal $400 if head, spouse, or co-head is elderly or disabled

Must equal Deduction for Dependents (97) + Child Care Expense

106.  Total Allowance

Allowance (98 + 99) + Allowance for Disability Expenses (101) +

Deduction for Medical Expenses (104) + Elderly Family Deduction (105)

107.  Adjusted Annual Income

Must equal Total Annual Income (86) minus Total Allowances (106)

Family Rent and Subsidy Information

33.

Gross Rent

Must equal Contract Rent (31) + Utility Allowance (32)

108.  Total Tenant Payment

Must equal the higher of 30% of Adjusted Income (107), 10% of Total

Annual Income (86), Welfare Rent (112), or $25 (Minimum Rent)

109.  Tenant Rent

Tenant Rent must equal the recalculated tenant rent based on the Rent

Calculation rules provided in Appendix A

Note: With the exception of tenant rent, negative numbers are always converted to 0.

HUDQC Final Report for FY 2015

D-7

Appendix E: Project Staff Questionnaire

Descriptive Analysis

Appendix E:  Project Staff Questionnaire Descriptive Analysis

The Project Staff Questionnaire (PSQ) was designed to obtain project-level information about the

characteristics and processes that support the Public Housing Authority (PHA) and project staff’s

calculation of rent during certification transactions, including initial and annual certifications. The

questionnaire aimed to identify structural procedures, standards, and policies that may hinder accurate

rent determination to uncover potential areas of improvement. The PSQ is a self-administered online

questionnaire sent to managers and executive directors of PHAs/projects included in the fiscal year

(FY) 2015 Quality Control (QC) Study.

A. Methodology

The PSQ was administered as a Web questionnaire using a survey package called SelectSurvey. In

February 2016, ICF staff contacted PHAs/projects via email with instructions on how to access and

complete the survey. Follow-up emails and telephone calls were made through the end of May to

PHAs/projects, reminding staff to complete the PSQ survey. Assistance was requested from HUD

to encourage some of the nonresponsive PHAs/projects to complete the questionnaire. Overall,

these efforts led to a response rate of 94 percent: 501 out of 531 PHAs/projects completed the PSQ.

After data collection, we examined the data to confirm the completeness and validity of responses.

PSQ surveys containing questionable responses or skip patterns were individually investigated

and data issues were resolved. Three cases were removed due to missing data, which resulted in a

total of 498 cases and a response rate of 94 percent.1 The PSQ responses were analyzed separately

for three major program types: Public Housing (179 projects), PHA-administered Section 8 (130

projects), and Owner-administered Section 8 (189 projects). This analysis was conducted using

SPSS, version 22.

The content of the FY 2015 PSQ was similar to the FY 2014 PSQ. It consisted of a combination of

open-ended and closed-ended questions. Topics included project characteristics, software usage,

training and development, performance management, and QC procedures. The results presented

in this report reflect the project’s response for the study period of November 1, 2014, to October

31, 2015.

B. Results

The results of the PSQ are presented in three sections that correspond to the three sections in the

survey:

1.  PHA/Project Characteristics. The PSQ surveyed respondents on PHA/project characteristics

that may help to explain differences in error rates. This section included questions about

the number and types of staff, number of project units/tenants and certifications conducted,

staff work experience, staff work assignments and workload, and staff use of software and

computer technology related to certification tasks.

2.  Training and Development.  The second section of the survey gathered information about

the nature and extent of rent calculation training for new and experienced certification staff,

1  Two of these projects only responded to the first question of the PSQ; the third project only responded

to the first six questions.

HUDQC Final Report for FY 2015

E-1

Appendix E: Project Staff Questionnaire Descriptive Analysis

procedures for implementing new policies related to rent calculation, and certification staff

work practices.

3.  Performance Management.  The PSQ also inquired about various aspects of performance

management of certification activities and QC reviews, including the timing of reviews,

methods used to select cases for review, the type of certification information reviewed for

QC, the prevalence of various types of rent calculation errors, and performance feedback

methods and timing.

Our highlights of the survey findings are presented in the sections that follow.

1. PHA/Project Characteristics

Type and Number of Staff. Beginning in FY 2008, the PSQ has collected information from projects

about whether or not they can provide information about the administration of rental assistance in the

project specifically selected for the QC Study review. Some PHAs/projects cannot provide information

about practices for a specific project because PHA/project staff work across multiple counties or across

a number of assisted housing units beyond the site or county selected for QC review. In FY 2015,

75 percent of PHAs/projects could provide information specific to the project selected for the QC

Study review (see Exhibit E-1a). PHAs/projects that could not provide project-specific information,

but offered information about their entire organization, indicated that they employed an average of

22 staff members who supported an average total of 3,320 units/households.2 These organizations

reported an average ratio of 78 assisted units/households per total staff. PHA-administered Section 8

projects had the highest average ratio of units per total staff at 129, Owner-administered projects had

the smallest average ratio at 32, and Public Housing projects were in the middle, with an average of

76 units per total staff in the organization.

In FY 2015, PHAs and management companies that could provide information on a

specific  project  averaged  11  employees  (see  Exhibit  E-1a).  With  respect  to  program  type,

PHA-administered Section 8 projects averaged 26 employees, followed by Public Housing with

9 employees and Owner-administered projects with 4 employees. On average, PHA/project staff

across all three program types supported 1,238 units over a 12-month period, with an average ratio

of 88 units per total staff. As above, PHA-administered Section 8 projects had the highest ratio of

units per total staff at 166, Owner-administered projects had the smallest ratio at 40, and Public

Housing sites averaged 88 units per total staff.

In addition to the general ratio of total employed staff to units/households served, the PSQ collected

information about certification staff members who performed move-in and annual certifications.

In FY 2015, PHAs/projects averaged 8 certification staff members who conducted an average of

2  In the analysis tables, medians are presented alongside averages because, for this data, the medians

may be a more accurate measure of central tendency. Since the data is consistently skewed to the right,

the averages tend to be much larger than the medians. For example, the average total number of staff at

entire organizations is 22.2, while the median is 5.0 staff members. In other words, while the average

is 22.2 staff, half of PHAs/projects employ five staff members or less. While the medians are presented

in the tables, only averages will be discussed in the analysis so that FY 2015 PSQ results can easily be

compared with FY 2014 PSQ results.

E-2

August 31, 2016

Appendix E: Project Staff Questionnaire Descriptive Analysis

164 move-in/initial certifications and 1,516 annual certifications over a 12-month period (see Exhibit

E-1a). PHA-administered Section 8 projects had the highest average number of certification staff at

23, but these sites also performed the most work overall, with an average of 531 initial certifications

and 5,065 annual certifications completed during the study period. Owner-administered projects

had the smallest average with 2 certification staff members who conducted an average of 20 initial

certifications and 112 annual certifications. Public Housing sites had an average of 4 certification

staff members and managed an average of 43 initial certifications and 422 annual certifications

conducted in FY 2015.

Exhibit E-1a

Number of Staff and Certifications, by Program Type

Program Type

PHA-

All

Number of Staff and

Public

administered

Owner-

Program

Number of Certifications Performed

Housing

Section 8

administered

Types

PHA/Project Can Provide Information About Staff and

Certifications at Individual Projects

76.5%

68.5%

76.7%

74.5%

Entire Organizations

Total Number of Staff the Entire

Mean

7.4

55.4

5.4

22.2

Organization Employs

Median

4.0

12.0

3.0

5.0

Total Number of Assisted Units

Mean

380.1

9,802.8

85.8

3,320.1

Supported by These Staff

Median

203.0

1,089.0

64.0

197.0

Mean

76.1

128.6

31.6

77.7

Units per Entire Organization Staff Ratio

Median

39.6

99.9

20.0

38.8

Individual Projects

Total Number of Staff the Individual

Mean

8.8

25.8

3.9

11.0

Project Employs

Median

4.0

11.0

3.0

4.0

Total Number of Assisted Units

Mean

470.1

4,198.7

129.7

1,237.5

Supported by These Staff

Median

256.0

1,908.0

100.0

200.0

Mean

87.6

166.0

39.6

87.8

Units per Individual Project Staff Ratio

Median

56.7

163.3

32.3

52.1

Entire Organizations and Individual Projects

Number of Certification Staff Who Work

Mean

3.6

23.2

2.4

8.3

at the PHA/Project

Median

3.0

9.0

2.0

3.0

Number of Initial/Move-in Certifications

Mean

42.7

531.0

19.6

163.7

Conducted Over a 12-Month Period

Median

33.5

189.0

11.5

31.0

Number of Annual Certifications

Mean

421.8

5,065.0

112.1

1,516.4

Conducted Over a 12-Month Period

Median

207.0

1,379.0

89.0

176.0

Note: Averages and medians were calculated based on the number of PHAs/projects that responded to the specific items.

Experienced Certification Staff and Certification Staff Turnover. Additionally, the PSQ collected

information specifically about the number of experienced certification staff at PHAs/projects. In the

survey, “experienced certification staff” was defined as certification staff members with more than

1 year of certification experience at the project. On average, PHA-administered Section 8 projects

HUDQC Final Report for FY 2015

E-3

Appendix E: Project Staff Questionnaire Descriptive Analysis

reported having 18 experienced staff, while Public Housing had 3 experienced certification staff and

Owner-administered projects had 2 experienced staff during the study period (see Exhibit E-1b).

PHAs/projects were also asked about the rate of certification staff turnover. Thirty-one percent of

PHAs/projects in the study indicated that they had at least 1 certification employee stop working

on certification activities at the specified project from November 1, 2014, to October 31, 2015

(see Exhibit E-1b), the same rate as in FY 2014. Among those PHAs/projects that had turnover,

an average of 3 certification staff stopped working on certification activities at the project during

the study period. PHA-administered Section 8 projects were most likely to have certification staff

turnover during the study period (44 percent) and reported the largest turnover (an average of

5  certification  staff).  Public  Housing  sites  were  least  likely  to  have  certification  staff  turnover

(26 percent) and reported turnover of 2 certification staff. Twenty-eight percent of Owner-administered

projects had some turnover, with an average of 1 certification staff member who stopped working

on certification activities at the specified project in FY 2015.

Exhibit E-1b

Experienced Certification Staff and Staff Turnover, by Program Type

Program Type

PHA-

All

Public   administered

Owner-

Program

Average Number of Certification Staff

Housing

Section 8

administered

Types

Experienced Certification Staff

Certification Staff With More Than 1 Year

Mean

3.1

18.2

2.2

6.7

of Experience

Median

3.0

7.0

2.0

3.0

Certification Staff Turnover

PHAs/Projects With At Least One Certification Staff

Member Who Stopped Working on Certification Activities

26.3%

43.8%

27.5%

31.3%

Number of Certification Staff Members

Mean

1.6

4.6

1.3

2.6

Who Stopped Working on Certification

Activities*

Median

1.0

2.0

1.0

1.0

Note: Averages, medians, and percentages were calculated based on the number of PHAs/projects that responded to the specific items.

* Averages and medians were calculated based on the PHAs/projects that had staff turnover.

Certification Staff Assignments and Workload. The FY 2015 PSQ asked PHAs/projects to indicate

how work was assigned to certification staff. The most frequently employed methods for assigning

cases were by certification transaction type (19 percent) and by random assignment of cases based

on staff availability (16 percent) (see Exhibit E-1c). Methods with less frequently reported use

included assignment of cases by certification activity type and by tenant last name (12 percent and

10 percent, respectively), while almost no projects assigned cases by household characteristics (less

than 1 percent). Although the question provided a variety of case assignment methods, nearly half

of PHAs/projects reported that they use case assignment methods other than those provided in the

survey (43 percent).

A look at program-specific breakdowns indicates that PHA-administered Section 8 projects were

by far the most likely to assign work alphabetically by tenant last name (35 percent), while just

1 percent of Owner-administered projects and Public Housing projects relied on this method

E-4

August 31, 2016

(see Exhibit E-1c). Owner-administered projects were most likely to assign work by certification

transaction type (20 percent) and by certification activity type (18 percent). Public Housing projects

were also most likely to assign cases by certification transaction type (22 percent), followed by

random assignment based on staff availability (18 percent).

Exhibit E-1c

Certification Staff Case Assignment Methods, by Program Type

Program Type

PHA-

All

Public

administered

Owner-

Program

Certification Staff Case Assignment Methods

Housing

Section 8

administered

Types

By Transaction Type

(e.g., some staff work on move-ins only, some staff

22.3%

14.6%

19.6%

19.3%

work on annual certifications only)

Random Assignment Based on Staff Availability

18.4%

10.8%

16.4%

15.7%

By Activity Type

(e.g., some certification staff perform interviews

and send out initial third-party verifications, while

8.4%

7.7%

18.0%

11.8%

other staff perform rent calculations and enter data

from Form HUD-50058/50059)

Alphabetical by Tenant Last Name

(e.g., households with a last name that starts with

any letter between “A” through “E” belong to one

1.1%

34.6%

1.1%

9.8%

certification staff member/staff team)

By Household Characteristic

(e.g., more complicated cases go to particular staff)

0.6%

0.0%

1.6%

0.8%

Other Assignment Method Not Listed

49.2%

32.3%

43.4%

42.6%

Note: Percentages were calculated based on the number of PHAs/projects that responded to the specific items.

In addition to methods of assigning work, PHAs/projects were asked to comment on the average

workload for certification staff at the specified project (see Exhibit E-1d). The results show that most

certification staff had either a manageable workload (56 percent) (i.e., a workload that is neither too

low nor too high) or a high workload (44 percent). In FY 2014, each category described 49 percent of

PHAs/projects, suggesting that more certification staff had a manageable workload in FY 2015. With

respect to program type, Owner-administered projects were most likely to report a manageable workload

(66 percent) from November 1, 2014, to October 31, 2015, while PHA-administered Section 8

projects were most susceptible to heavy workloads (59 percent).

HUDQC Final Report for FY 2015

E-5

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-1d

Certification Staff Average Workload, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Certification Staff Average Workload

Housing

Section 8

administered

Types

The Workload Was High

43.6%

58.5%

33.9%

43.8%

The Workload Was Not Too Low or Too High

55.9%

40.8%

65.6%

55.6%

The Workload Was Low

0.6%

0.8%

0.5%

0.6%

Note: Percentages were calculated based on the number of PHAs/projects that responded to the specific item.

Organizations Contracted to Perform Certification Activities.  Despite the moderate to heavy

workloads experienced by certification staff, a large majority of PHAs/projects (89 percent) indicated

that they do not contract out certification activities to outside organizations (see Exhibit E-1e). PHAs/

projects that did have a certification contractor in the past year were more likely to hire a private

company  or  nonprofit  organization  to  handle  certifications  (4  percent  each),  as  opposed  to  the

government (3 percent). With respect to program type, Public Housing projects were least likely to

use a contractor, with 92 percent of projects performing certifications themselves. PHA-administered

Section 8 and Owner-administered projects reported similar numbers (88 percent and 86 percent,

respectively).  Owner-administered  sites  that  did  contract  out  certification  activities  were  most

likely to choose a nonprofit organization (9 percent), while an equal number of PHA-administered

Section  8  projects  contracted  out  certification  activities  to  private  companies  and  government

agencies (5 percent each).

Exhibit E-1e

Organizations Contracted to Perform Certification Activities by Program Type

Program Type

PHA-

Organizations Contracted to Perform

Public

administered

Owner-

All Program

Certifications

Housing

Section 8

administered

Types

PHA/Project Does Not Contract Out Certification

Activities

92.2%

87.7%

85.7%

88.6%

Private Company

4.5%

4.6%

3.7%

4.2%

Nonprofit Organization

0.0%

3.1%

8.5%

4.0%

Government

3.4%

4.6%

2.1%

3.2%

Note: Percentages were calculated for PHAs/projects that responded to the specific item.

Utilization and Capabilities of Computer Software in the Certification Process.  Automated

systems and computer software continued to play an integral part in daily tasks at the PHAs/

projects surveyed. PHAs/projects were asked to describe the tasks for which certification staff

used computer systems. Virtually all of the PHAs/projects reported using computers to print

letters to tenants (98 percent) and Form HUD-50058/50059 (97 percent); calculate income,

expenses, and allowances (96 percent); and record tenant demographics (92 percent) (see Exhibit

E-1f). The majority of PHAs/projects also acknowledged using computer software to input

certification  interview  information  (81  percent),  determine  certification  dates/appointments

(70 percent), record other types of statistics (67 percent), and keep track of pending verification

E-6

August 31, 2016

documents (62 percent). Less than half of PHAs/projects reported using computer systems to store

electronic copies of verification documents (44 percent), conduct rent reasonableness comparisons

(40 percent), and conduct computer-assisted interviews with tenants (36 percent). The largest

increase from FY 2014 to FY 2015 was found in keeping track of pending verification documents

(from 59 percent to 62 percent), and the largest decrease in keeping other types of statistics

(72 percent to 67 percent).

Exhibit E-1f

Use of Computer Systems for Key Tasks, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Use of Computer Systems

Housing

Section 8

administered

Types

Printing Letters to Tenants

96.1%

99.2%

97.9%

97.6%

Printing Form HUD-50058/50059

96.6%

93.8%

98.9%

96.8%

Calculating Income, Expenses, and Allowances

95.0%

96.2%

95.8%

95.6%

Recording Tenant Age, Ethnicity, Family Size,

or Other Demographics

88.8%

97.7%

92.1%

92.4%

Inputting Certification Interview or Application

Responses

77.7%

86.9%

78.8%

80.5%

Determining Certification Appointment Dates

73.7%

73.8%

63.5%

69.9%

Keeping Other Types of Statistics

60.3%

76.2%

67.2%

67.1%

Keeping Track of Pending Verification

Documents

62.0%

60.8%

61.9%

61.6%

Storing Electronic Verification Documents

39.1%

53.8%

40.7%

43.6%

Conducting Rent Reasonableness

Comparisons

34.6%

79.2%

16.9%

39.6%

Conducting Computer-Assisted Interviews with

Tenants

39.1%

37.7%

32.3%

36.1%

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

In addition to the tasks shown in Exhibit E-1f, 96 percent of PHAs/projects indicated that they

use computer software to help calculate tenant rent, with PHA-administered Section 8 being most

likely (99 percent), and Public Housing sites being the least likely (91 percent) (see Exhibit E-1g).

Of that 96 percent, a large portion reported having software capable of assisting staff by storing

household-specific information from previous Form HUD-50058/50059s (96 percent), submitting

data to PIC/TRACS (92 percent), annualizing individual sources of income and/or expenses

(91  percent),  and  containing  pre-loaded  information  that  identifies  the  appropriate  payment

standard or utility allowance (86 percent). However, 70 percent of PHAs/projects reported that

their software was limited in its capabilities by requiring users to enter Form HUD-50058/50059

after its manual completion.

Program-specific results show that PHA-administered Section 8 and Owner-administered projects

were both very likely to report using software that is capable of submitting data to PIC/TRACS

(93 percent and 97 percent, respectively), while only 85 percent of Public Housing projects

had software with this capability (see Exhibit E-1g). Furthermore, PHA-administered Section 8

and Owner-administered projects were about equally likely to report using software containing

HUDQC Final Report for FY 2015

E-7

Appendix E: Project Staff Questionnaire Descriptive Analysis

pre-loaded  information  that  identifies  the  appropriate  payment  standard  or  utility  allowance

(90 percent and 89 percent, respectively), while only 80 percent of Public Housing projects had

software with this capability.

Exhibit E-1g

Functionalities of Computer Software, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Tasks Performed Using Computer Software

Housing

Section 8

administered

Types

Certification Staff Use Computer Software to

Help Calculate Tenant Rent

91.1%

99.2%

98.4%

96.0%

Functionalities

Stores Household-Specific Information From

Previous Form HUD-50058/50059s and Allows

91.4%

98.4%

97.8%

95.8%

Updating With Current Information*

Submits Data to PIC/TRACS*

84.7%

93.0%

96.8%

91.6%

Annualizes Individual Sources of Income/

Expenses When Information Is Entered*

87.1%

94.6%

93.0%

91.4%

Contains Pre-Loaded Information that Identifies

the Appropriate Payment Standard/Utility

Allowance for Each Household Based on

80.4%

89.9%

88.7%

86.2%

Information Entered*

User Must Enter Form HUD-50058/50059 Data

After Its Manual Completion*

68.1%

67.4%

74.2%

70.3%

Note: Percentages were calculated for PHAs/projects that responded to the specific item.

* Percentages were calculated based on PHAs/projects that indicated using computer software to help calculate rent.

2. Training and Development

Certification Staff Training and Development.  The PSQ collected information about the amount

and type of training provided to both new and experienced certification staff during the study period.

Fifty-nine percent of PHAs/projects reported having a training department or staff trainer that provided

guidance to staff working on rent calculation activities, a decrease from 67 percent in FY 2014.

Owner-administered programs were most likely to have a training department or staff trainer for

certification staff (71 percent), followed by Public Housing (53 percent) and PHA-administered

Section 8 projects (50 percent) (see Exhibit E-2a).

Training for New Staff.  The average number of hours of training provided to each new certification

staff member prior to performing rent calculations unassisted varied greatly by program type. On

average, PHA-administered Section 8 projects provided the most training hours to new certification

staff (69 hours), followed by Public Housing sites (60 hours), and Owner-administered programs

(25-hours) (see Exhibit E-2a). The overall average for all program types was 49 hours of training,

which is the lowest in the last 6 years (50 average hours were provided to staff in FY 2014,

E-8

August 31, 2016

55-average hours in FY 2013, 65 average hours in FY 2012, 89 hours on average in FY 2011, and

about 85 training hours in FY 2010).3

Exhibit E-2a

Presence of Training Staff in PHA/Project and

Average Training Hours for Certification Staff, by Program Type

Program Type

PHA-

All

Training Staff Present in PHA/Project and

Public

administered

Owner-

Program

Training Hours for Certification Staff

Housing

Section 8

administered

Types

Training Department

PHA/Project Has a Training Department or Staff

Trainer for Certification Staff

52.5%

50.0%

71.4%

59.0%

Training Hours

Number of Training Hours Before

Mean

59.6

68.6

24.8

48.8

Staff Can Perform Rent Calculations

Unassisted

Median

8.0

33.5

16.0

16.0

Note: Percentages, averages, and medians were calculated for PHAs/projects that responded to the specific item.

PHAs/projects were asked to explain in further detail how the number of training hours provided

to one certification staff member were typically allocated. PHAs/projects reported the approximate

number of rent calculation training hours that a typical new staff member is provided via various

training methods (see Exhibit E-2b). In FY 2012, FY 2011, and FY 2010, PHAs/projects were asked

to rank the three most frequently used training methods so that percentages of training utilization

within each PHA/project could be calculated. Beginning in FY 2013, any training hours reported for

the methods indicated utilization. The reported hours were used to calculate the average percentage

of PHAs/projects that have used the various methods during the study period, where zero hours

indicated that the method was not used.

The most commonly used training method was reading policies and procedural guides

on their own (96 percent), followed by shadowing or mentorship with experienced staff

(94 percent) and classroom-style training administered in-house (70 percent) (see Exhibit E-2b).

Web-based or recorded training video created in-house was the method least likely to be used

(28 percent). The use of most of the training methods presented has remained stable since

FY 2014. There was, however, a marked increase in the use of shadowing or mentorship with more

experienced staff (91 percent to 94 percent) and in classroom-style training administered by an

outside organization (56 percent to 63 percent) since FY 2014.

In FY 2015, all Public Housing projects prepared new certification employees by having them

read policies and procedural guides on their own (100 percent). This method was also used by the

vast majority of PHA-administered Section 8 and Owner-administered projects (96 percent and

94 percent, respectively). PHA-administered Section 8 projects were most likely to prepare new

certification  staff  by  having  them  shadow  experienced  staff  (99  percent).  Owner-administered

projects were the least likely to use a Web-based or recorded training video created by HUD

3  Training hour averages for FY 2012, FY 2011, and FY 2010 were calculated as the average for new

recertification staff and reassigned staff.

HUDQC Final Report for FY 2015

E-9

Appendix E: Project Staff Questionnaire Descriptive Analysis

(35 percent), while more than half of Public Housing and PHA-administered Section 8 projects

employed this method (56 percent and 51 percent, respectively). Conversely, Owner-administered

projects were more likely to use Web-based or recorded training videos created by an outside

organization (48 percent) or in-house (39 percent) than both Public Housing projects (41 percent and

25 percent, respectively) and PHA-administered Section 8 (37 percent and 16 percent, respectively).

Exhibit E-2b

Methods Used to Train New Certification Staff, by Program Type

Note: Percentages were calculated for those PHAs/projects that had new certification staff in FY 2015.

Although most PHAs/projects relied on new staff to self-train by reading policies and procedural

guides, an average of only 36 total hours per staff member was spent on this training method (see

Exhibit E-2c). PHAs/projects dedicated the most training hours to shadowing and mentorship with

more experienced staff, with an average of approximately 127 hours for each new certification

staff member from November 1, 2014, to October 31, 2015. While this was an increase from

FY 2014 (98 hours), it is consistent with the results from the FY 2013 PSQ (120 hours). The fewest

hours of training were spent using Web-based or recorded training videos created by HUD (3 hours)

and created in-house (3 hours).

E-10

August 31, 2016

Exhibit E-2c

Training Hours per Training Method for New Certification Staff, by Program Type

Program Type

PHA-

Public   administered

Owner-

All Program

Training Methods for New Staff

Housing

Section 8

administered

Types

Shadowing or Mentorship With More

Mean

106.1

207.8

83.7

127.0

Experienced Staff

Median

25.0

70.0

24.0

36.0

Reading Policies and Procedural Guides  Mean

26.7

77.3

12.8

36.1

on Their Own

Median

8.0

16.0

4.0

8.0

Classroom-Style Training Administered

Mean

23.4

33.4

14.1

22.7

In-House

Median

6.0

12.0

8.0

8.0

Classroom-Style Training Administered

Mean

15.2

21.4

10.6

15.2

by an Outside Organization

Median

4.5

24.0

8.0

8.0

Web-Based or Recorded Training Video

Mean

3.4

5.6

3.9

4.2

Created by an Outside Organization

Median

0.0

0.0

0.0

0.0

Web-Based or Recorded Training Video

Mean

1.5

0.7

5.5

2.8

Created In-House

Median

0.0

0.0

0.0

0.0

Web-Based or Recorded Training Video

Mean

3.5

3.9

1.4

2.8

Created by HUD

Median

1.0

1.0

0.0

0.0

Other Type of Training Activity Not

Mean

5.8

11.2

10.0

9.0

Mentioned Above

Median

0.0

0.0

0.0

0.0

Note: Averages and medians were calculated for those PHAs/projects that had new certification staff in FY 2014.

PHA/project training topics used to educate staff on conducting certifications are also important

in determining activities that support proper rent calculation. PHAs/projects reported the

approximate number of rent calculation training hours for a typical new staff member on various

training topics. These hours were used to calculate the average percentage of PHAs/projects

that provided various training topics related to certification activities from November 1, 2014,

to October 31, 2015, where zero hours indicated that the training topic was not presented to

certification staff. For each training topic area, most PHAs/projects (at least 88 percent) provided

training (see Exhibit E-2d). The average number of hours spent on each topic per staff member

was, for the most part, evenly distributed among the topic areas, with averages ranging from

24 to 36 hours during the study period, with the most time spent on entering Form HUD-50058/50059

and the least amount of time spent on Enterprise Income Verification (EIV) reports and security (see

Exhibit E-2e). This was a substantial increase in training hours for each training topic area from

FY 2014, when training hours ranged from 16 to 25 hours among the seven topics.

HUDQC Final Report for FY 2015

E-11

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-2d

Training Topic Areas for New Certification Staff, by Program Type

Note: Percentages were calculated for those PHAs/projects that had new certification staff in FY 2015.

* Deductions refer to medical, disability, and childcare deductions.

Among the three program types, Owner-administered projects were the most likely to provide

training in interviewing tenants and entering Form HUD-50058/50059 (92 percent each), while

PHA-administered Section 8 projects were the most likely to provide training in calculating

deductions, assets, and earned and fixed income sources, as well as in EIV reports and security

(98 percent, 98 percent, 99 percent, 98 percent, and 97 percent, respectively) (see Exhibit E-2d).

Furthermore, PHA-administered Section 8 projects spent, on average, the most training hours per

training topic area, ranging from 36 to 52 hours, with the exception of EIV reports and security

(see Exhibit E-2e).

E-12

August 31, 2016

Exhibit E-2e

Average Training Hours per Training Topic for New Certification Staff, by Program Type

Program Type

PHA-

All

Public

administered

Owner-

Program

Training Topics for New Staff

Housing

Section 8

administered

Types

Entering Form HUD-50058/50059  Mean

28.6

51.8

29.7

35.8

Information

Median

4.0

10.0

4.0

5.0

Calculating Earned Income

Mean

31.6

47.7

29.1

35.3

Sources

Median

7.0

14.0

4.0

8.0

Calculating Deductions (Medical,

Mean

32.2

38.9

29.0

32.9

Disability, Childcare)

Median

5.0

14.0

4.0

5.0

Mean

30.1

38.6

28.5

32.0

Calculating Fixed Income Sources Median

5.0

12.0

4.0

5.0

Mean

28.9

37.0

28.7

31.2

Calculating Assets

Median

4.0

12.0

4.0

5.0

Mean

30.1

35.5

26.8

30.4

Interviewing Tenants

Median

4.3

10.0

4.0

5.0

Mean

15.8

25.7

28.8

23.7

EIV Reports and EIV Security

Median

4.8

6.0

4.0

5.0

Mean

8.8

12.1

7.5

9.3

Other

Median

0.0

4.0

0.0

1.0

Note: Averages and medians were calculated for those PHAs/projects that had new certification staff in FY 2014.

Training for Experienced Staff. PHAs/projects were also asked to report the approximate number

of rent calculation training hours for a typical experienced staff member for each training method.

These reported hours were used to calculate the average percentage of PHAs/projects that have

used the various methods, where zero hours indicated that the method was not used. As for new

certification  staff,  PHAs/projects  were  most  likely  to  have  experienced  staff  read  policies  and

procedural guides on their own (86 percent), shadow more experienced staff (55 percent), and attend

classroom-style training administered by an outside organization (52 percent) (see Exhibit E-2f).

Compared to FY 2014, the data showed a pronounced increase in the use of Web-based or recorded

training videos created by HUD (22 percent to 38 percent) and by outside organizations (33 percent

to 39 percent). However, there was a dramatic decrease in the use of Web-based or recorded training

videos created in-house (34 percent to 20 percent). Additionally, there were slight decreases in the

use of shadowing more experienced staff (58 percent to 55 percent), reading policies and procedural

guides alone (88 percent to 86 percent), and classroom-style training administered in-house

(53 percent to 50 percent) (see Exhibit E-2f). The use of classroom-style training administered

in-house remained stable (52 percent).

HUDQC Final Report for FY 2015

E-13

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-2f

Methods Used to Train Experienced Certification Staff, by Program Type

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

An analysis by program type found that PHA-administered Section 8 projects were most likely to

provide policies and procedural guides to experienced staff for self-training (89 percent) and to use

classroom-style training administered in-house (55 percent) (see Exhibit E-2f). Owner-administered

projects were most likely to have an experienced employee attend classroom-style training

administered by an outside organization (60 percent) and use Web-based or recorded training

videos created by an outside organization (46 percent). All three program types were about

equally likely to have experienced staff shadow more experienced staff members (ranging from

54 percent to 55 percent). Additionally, Public Housing and PHA-administered Section 8

projects were about equally likely to use Web-based or recorded training videos created by HUD

(43 percent and 42 percent, respectively). In contrast, only 30 percent of Owner-administered

projects used this method.

E-14

August 31, 2016

Exhibit E-2g

Training Hours per Training Method for Experienced Certification Staff, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Training Methods for Experienced Staff

Housing

Section 8

administered

Types

Shadowing or Mentorship With

Mean

15.2

78.7

47.3

44.0

More Experienced Staff

Median

1.0

2.0

2.0

2.0

Reading Policies and Procedural  Mean

28.0

55.9

25.3

34.3

Guides on Their Own

Median

5.0

9.5

4.0

5.0

Classroom-Style Training

Mean

11.7

9.8

18.8

13.9

Administered by an Outside

Organization

Median

0.0

0.5

4.0

2.0

Classroom-Style Training

Mean

8.2

10.1

7.3

8.3

Administered In-House

Median

0.0

2.0

1.0

0.0

Web-Based or Recorded

Mean

2.5

3.5

3.3

3.1

Training Video Created by an

Outside Organization

Median

0.0

0.0

0.0

0.0

Web-Based or Recorded

Mean

3.1

2.5

1.8

2.4

Training Video Created by HUD

Median

0.0

0.0

0.0

0.0

Web-Based or Recorded

Mean

0.7

1.2

2.7

1.6

Training Video Created In-House Median

0.0

0.0

0.0

0.0

Mean

14.7

3.8

3.7

7.7

Other (Activity Not Listed)

Median

0.0

0.0

0.0

0.0

Note: Averages were calculated for PHAs/projects that responded to the specific items.

The average number of training hours provided to an experienced certification staff member was

considerably less than those provided to a new staff member. PHAs/projects dedicated the most

training hours to shadowing and mentorship with more experienced staff, with an average of

approximately 44 hours per experienced certification staff over the study period (see Exhibit E-2g).

This is an increase from an average of 29 hours in FY 2014, but is consistent with FY 2013 PSQ

results (43 hours). The remaining training for experienced staff throughout the study period consisted

of an average of 34 hours of self-training, 14 hours for classroom-style training administered by an

outside organization, and less than 10 hours for each remaining training method.

As mentioned above, PHA/project training topics used to educate staff on conducting certifications

are also important in determining activities that support proper rent calculation. PHAs/projects

reported the approximate number of rent calculation training hours for a typical experienced staff

member on various training topics. These hours served to calculate the average percentage of PHAs/

projects that provided various training topics related to certification activities from November 1,

2014, to October 31, 2015, where zero hours indicated that the training topic was not presented to

certification staff. At least 53 percent of PHAs/projects provided training to experienced certification

staff for each topic listed (see Exhibit E-2h). PHAs/projects were most likely to train experienced

certification staff in EIV reports and security (81 percent) and calculating deductions (66 percent),

earned income sources, and assets (65 percent each) (see Exhibit E-2h). Of the topics listed, PHAs/

projects were least likely to train on interviewing tenants (53 percent). With respect to program type,

HUDQC Final Report for FY 2015

E-15

Appendix E: Project Staff Questionnaire Descriptive Analysis

Owner-administered projects were the most likely to train on EIV reports and security (86 percent),

and PHA-administered Section 8 projects were the most likely to train on all other training topics.

Exhibit E-2h

Training Topic Areas for Experienced Certification Staff, by Program Type

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

* Deductions refer to medical, disability, and childcare deductions.

The average number of hours spent on each topic listed was fairly evenly distributed among the topic

areas (see Exhibit E-2i). On average, the most time was spent on entering Form HUD-50058/50059

information (10 hours) and the least amount of time on interviewing tenants (7 hours). Interestingly,

the most training time for experienced staff in FY 2014 was spent on interviewing tenants (12 hours).

In  comparison  to  a  new  certification  staff  member,  an  experienced  certification  staff  member

received approximately 25 fewer hours of training for each topic (see Exhibit E-2i). This was a

considerable increase from FY 2014, when experienced staff received 13 fewer hours of training

than new staff per topic. Furthermore, in FY 2013, experienced staff only received 7 fewer training

hours, on average, than new certification staff. This is primarily due to the substantial increase in

training hours for new staff, though the number of training hours for experienced staff did slightly

decrease as well.

With respect to program type, Public Housing sites provided the most training in calculating

earned  income  sources,  fixed  income  sources,  deductions,  assets,  and  in  interviewing  tenants

(11 hours each) (see Exhibit E-2i). PHA-administered Section 8 projects provided the most training

hours on entering Form HUD-50058/50059 (15 hours). Public Housing and PHA-administered

Section 8 project offered, on average, the same number of training hours in EIV reports and security

(8 hours). Owner-administered projects provided the least amount of training to experienced

certification staff, with an average of 6 hours or less spent on each topic.

E-16

August 31, 2016

Exhibit E-2i

Training Hours per Training Topic for

Experienced Certification Staff, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Training Topics for Experienced Staff

Housing

Section 8

administered

Types

Entering Form

Mean

10.9

14.5

5.5

9.8

HUD-50058/50059 Information

Median

1.0

1.0

1.0

1.0

Calculating Earned Income

Mean

11.4

10.6

5.1

8.8

Sources

Median

1.0

3.0

1.0

2.0

Calculating Fixed Income

Mean

11.3

8.6

5.1

8.3

Sources

Median

1.0

2.0

1.0

1.0

Calculating Deductions

Mean

11.1

8.1

4.9

8.0

(Medical, Disability, Childcare)

Median

1.0

3.0

1.0

2.0

Mean

11.3

7.3

4.8

7.8

Calculating Assets

Median

1.0

2.0

1.0

1.0

Mean

7.7

7.7

6.2

7.1

EIV Reports and EIV Security

Median

2.0

3.0

3.0

2.0

Mean

11.2

5.8

3.8

7.0

Interviewing Tenants

Median

0.0

1.0

1.0

1.0

Mean

4.2

6.2

4.8

5.0

Other

Median

0.0

0.0

0.0

0.0

Note: Averages and medians were calculated for PHAs/projects that responded to the specific items.

Policy Implementation.  In FY 2015, the PSQ aimed to capture information from PHAs/projects

on the typical time it takes to implement a new policy related to rent calculation. An analysis of

the qualitative data collected found that most PHAs/projects implement a new policy between

1 and 30 days once an Office of Public and Indian Housing (PIH) Notice or Housing Notice is

issued (44 percent) (see Exhibit E-2j). Many PHAs/projects also stated that implementation of a

new policy related to rent calculation began immediately after issuance of a notice (27 percent),

and only 4 percent of projects reported that they take longer than 90 days to implement a new rent

calculation policy.

With respect to program type, Owner-administered projects were most likely to immediately

implement a new policy and most likely to have full implementation of a new policy within

30 days (34 percent and 89 percent, respectively) (see Exhibit E-2j). By comparison, 60 percent

of PHA-administered Section 8 projects and 58 percent of Public Housing sites achieved full

implementation of a new policy within 30 days.

HUDQC Final Report for FY 2015

E-17

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-2j

Implementation Time for a New Rent Calculation Policy, by Program Type

Certification  Staff  Work  Behaviors.  Another goal of the FY 2015 PSQ was to gain insight

into PHA/project assessments of the quality of certification staff work behaviors. Ninety-eight

percent of PHAs/projects rated their certification staff as either organized or very organized when

working on certification activities, an increase from FY 2014 (95 percent) (see Exhibit E-2k).

Owner-administered projects were most likely to rate staff as very organized (59 percent), while

more Public Housing and PHA-administered Section 8 projects rated their staff as organized

(55 percent and 52 percent, respectively). Less than 1 percent of PHA-administered Section 8

projects, as well as no Public Housing sites or Owner-administered projects, rated their staff as

very unorganized.

Similarly, about 96 percent of PHAs/projects reported certification staff as having either good

or very good time management (see Exhibit E-2k). As with organization, Owner-administered

projects were most likely to select a rating of very good (57 percent), while Public Housing sites

and PHA-administered Section 8 projects were more likely to report good time management

(58 percent and 52 percent, respectively). Public Housing sites were most likely to report certification

staff as having poor time management (7 percent). No PHAs/projects reported certification staff as

having very poor time management.

Eighty percent of PHAs/projects indicated that their certification staff had a lot of attention to

detail (see Exhibit E-2k). This is an increase from FY 2014, when 72 percent of PHAs/projects

rated their staff as having a lot of attention to detail. Following trends from previous questions,

Owner-administered projects were the most likely to rate their staff as having a lot of attention to

detail (86 percent) when compared with Public Housing sites and PHA-administered Section 8

projects (74 percent and 78 percent, respectively). Overall, 19 percent of PHAs/projects rated staff

as having some attention to detail, and just 2 percent as having little attention to detail. Less than

1 percent of Public Housing sites, and no PHA-administered Section 8 or Owner-administered

projects, indicated that certification staff had very little or no attention to detail.

E-18

August 31, 2016

Exhibit E-2k

Certification Staff Work Behaviors, by Program Type

Program Type

PHA-

Certification Staff Organization, Attention to

Public

administered

Owner-

All Program

Detail, and Quality of Time Management

Housing

Section 8

administered

Types

Certification Staff Organization

Very Organized

41.3%

46.9%

58.7%

49.4%

Organized

55.3%

51.5%

39.7%

48.4%

Unorganized

3.4%

0.8%

1.1%

1.8%

Very Unorganized

0.0%

0.8%

0.0%

0.2%

Certification Staff Attention to Detail

A Lot

73.7%

77.7%

86.2%

79.5%

Some

23.5%

20.0%

12.7%

18.5%

Little

2.2%

2.3%

0.5%

1.6%

Very Little/None

0.6%

0.0%

0.0%

0.2%

Certification Staff Quality of Time Management

Very Good

35.2%

45.4%

56.6%

46.0%

Good

58.1%

52.3%

39.7%

49.6%

Poor

6.7%

2.3%

3.2%

4.2%

Very Poor

0.0%

0.0%

0.0%

0.0%

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

3. Performance Management

Quality Control Review and Timing of Reviews. The PSQ inquired about PHA/project practices on

the review of tenant files as a QC measure. Most PHAs/projects indicated that they reviewed move-in

and annual recertifications as a QC measure (89 percent) (see Exhibit E-3a). PHA-administered

Section 8 projects were most likely to review tenant files (95 percent), compared with Public Housing

sites and Owner-administered projects (87 percent each). The PHAs/projects that performed QC

reviews reviewed an average of 47 percent of all move-in and annual certification transactions,

and were most likely to conduct reviews prior to Form HUD-50058/50059 approval (27 percent)

or within 30 calendar days of submitting Form HUD-50058/50059 to PIC/TRACS (25 percent).4

Public Housing sites and Owner-administered projects both reviewed the majority of all

move-in  and  annual  certification  transactions  (52  percent  and  51  percent,  respectively),  while

PHA-administered Section 8 projects reviewed an average of 36 percent of transactions (see Exhibit

E-3a). Additionally, Owner-administered projects were most likely to conduct reviews prior to

Form HUD-50059 approval (40 percent). By comparison, 21 percent of Public Housing sites and

18 percent of PHA-administered Section 8 projects reviewed transactions prior to Form

HUD-50058 approval. Public Housing sites were the most likely to conduct QC reviews within

4  This percentage was calculated by combining the percentages of PHAs/projects that indicated that they

conducted QC reviews within 7 calendar days and within 30 calendar days of Form HUD-50058/50059

submission.

HUDQC Final Report for FY 2015

E-19

Appendix E: Project Staff Questionnaire Descriptive Analysis

30 calendar days (32 percent), followed closely by PHA-administered Section 8 projects

(30 percent), while Owner-administered projects were least likely to conduct reviews within

30 days (14 percent). Additionally, 22 percent of PHAs/projects indicated that QC reviews were

typically conducted between 6 months and 1 year after submitting Form HUD-50058/50059.

Exhibit E-3a

Quality Control Reviews and Timing of Reviews, by Program Type

Program Type

PHA-

Percentage and Timing of Quality Control

Public

administered

Owner-

All Program

Reviews

Housing

Section 8

administered

Types

Percentage of PHAs/Projects That Perform QC

Reviews of Move-in and Annual Certification

87.2%

95.4%

86.8%

89.2%

transactions

Average Percentage of Move-in and Annual

Certification Transactions Reviewed for QC*

51.8%

36.3%

51.3%

47.3%

Primary Period When QC Review Was Conducted

Prior to Form HUD-50058/50059 approval*

21.4%

17.7%

39.9%

27.2%

Within 7 calendar days of Form

HUD-50058/50059 submission*

5.8%

1.6%

1.2%

2.9%

Within 30 calendar days of Form

HUD-50058/50059 submission*

26.0%

28.2%

12.9%

21.8%

Within 60 calendar days of Form

HUD-50058/50059 submission*

11.0%

12.9%

6.1%

9.8%

Within 3 months of Form HUD-50058/50059

submission*

7.8%

11.3%

10.4%

9.8%

Within 6 months of Form HUD-50058/50059

submission*

5.8%

8.9%

6.1%

6.8%

Within 1 year of Form HUD-50058/50059

submission*

22.1%

19.4%

23.3%

21.8%

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

* Percentages were calculated for PHAs/projects that reviewed certification transactions as a QC measure.

Selecting Cases for Review. Of PHAs/projects that conducted QC reviews, 26 percent reported

that they reviewed not only move-in and annual certifications, but all tenant files as a QC measure

(see Exhibit E-3b). This is consistent with FY 2014 results. As in FY 2014, FY 2013, and

FY 2012, Owner-administered projects were most likely to review all cases (29 percent), and

PHA-administered Section 8 projects were least likely to do so (23 percent). The percentage of

Owner-administered projects that reviewed all tenant files decreased since FY 2014 (38 percent),

while PHA-administered Section 8 projects were more likely to complete comprehensive reviews

this year compared to FY 2014 (16 percent).

PHAs/projects that did not report performing QC reviews on all tenant files stated that they used

the following methods most frequently to select cases for QC: randomly selecting cases for review

(79  percent),  selecting  move-in  transactions  (50  percent),  and  selecting  annual  certification

transactions (40 percent) (see Exhibit E-3b). With respect to program type, PHA-administered

Section 8 projects were most likely to select tenant files randomly for QC review (89 percent). This

is a slight decrease from FY 2014, when 92 percent of PHA-administered Section 8 projects selected

E-20

August 31, 2016

tenant files randomly. Additionally, PHA-administered Section 8 projects were the most likely of the three program types to select annual certification transactions, transfer/move transactions, interim

transactions, files from new staff, files from staff with high error rates, and files based on household

characteristics for review (49 percent, 33 percent, 32 percent, 39 percent, 33 percent, and 8 percent,

respectively). Owner-administered projects were the most likely to select move-in transactions

(61 percent), as well as the most likely to use another method of selecting cases for review that was

not listed (10 percent).

Exhibit E-3b

Methods Used by PHAs/Projects to Select Cases for Review, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Methods Used to Select Cases for Review

Housing

Section 8

administered

Types

All Transactions That Were Processed Were

Reviewed by Another Staff Member

26.6%

22.6%

28.8%

26.3%

All Transactions Were Not Reviewed

Files Were Randomly Sampled*

77.0%

88.5%

72.4%

78.8%

Move-In Transactions Were Chosen for Review*

35.4%

54.2%

61.2%

50.2%

Annual Certification Transactions Were Chosen

for Review*

37.2%

49.0%

36.2%

40.3%

Transfer/Move Transactions Were Chosen for

Review*

17.7%

33.3%

21.6%

23.7%

Interim Transactions Were Chosen for Review*

22.1%

32.3%

16.4%

23.1%

Files Were Chosen From New Staff*

13.3%

38.5%

6.9%

18.5%

Files Were Chosen From Staff With High Error

Rates or Who Seem to Have More Trouble*

8.8%

33.3%

1.7%

13.5%

Files Were Chosen Based on Household Income,

Asset, and Expense Characteristics*

4.4%

8.3%

0.9%

4.3%

PHA/Project Chose Another Method of Selecting

Cases for Review That Is Not Listed*

6.2%

6.3%

10.3%

7.7%

Note: Percentages were calculated for PHAs/projects that indicated they review tenant files as a QC measure.

* Percentages were calculated based on PHAs/projects that indicated they did not review all (100%) tenant files.

File Reviewers. The majority of PHAs/projects indicated that they have a dedicated QC staff

to review data submitted on Form HUD-50058/50059 (68 percent) (see Exhibit E-3c). This is,

however, a decrease since FY 2014 (74 percent). PHA-administered Section 8 projects were

most likely to have QC staff (80 percent), compared with Public Housing sites (62 percent) and

Owner-administered projects (65 percent). PHAs/projects reported that, during the study period,

the review or monitoring of tenant files was conducted primarily by the team leader or supervisor

(83 percent) or internal QC staff (47 percent), who reviewed a yearly average of 266 and 233 tenant

files, respectively (see Exhibits E-3d and E-3c).

Public  Housing  sites  were  the  most  likely  to  rely  on  a  supervisor  or  HUD-affiliated  auditor

(87 percent and 26 percent, respectively) when compared to other program types.

Owner-administered projects were the most likely to use internal staff reviewers, other certification

staff,  contract  administers,  and  Office  of  the  Inspector  General  (OIG)  auditors  (55  percent,

HUDQC Final Report for FY 2015

E-21

Appendix E: Project Staff Questionnaire Descriptive Analysis

39 percent, 22 percent, and 11 percent, respectively). Supervisors and internal staff reviewers

at PHA-administered projects reviewed the most files, with an average of 681 and 643 files a

year, respectively. In contrast, supervisors and internal staff reviewers at Public Housing sites

reviewed an average of 165 and 117 files, respectively, and Owner-administered projects only 44 and

30 files, respectively. However, as seen in Exhibit E-1a, PHA-administered Section 8 staff have

a much larger workload of certifications to perform yearly, so it is reasonable that they also QC a

larger number of files.

Exhibit E-3c

Quality Control Staff and Average

Number of Tenant Files Reviewed, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Quality Control Staff and Files Reviewed

Housing

Section 8

administered

Types

Percentage of PHAs/projects With a Dedicated

QC Staff

62.0%

80.0%

65.1%

67.9%

Number of Files Reviewed

Mean

164.9

680.7

44.0

265.8

Supervisors or Team Leaders*

Median

42.0

85.5

15.0

30.0

Mean

116.7

642.9

29.5

232.9

Internal Staff Reviewers/QC*

Median

0.0

3.0

3.5

0.0

Mean

54.9

153.0

23.8

71.1

Contract Administrators*

Median

0.0

0.0

0.0

0.0

Mean

3.6

96.6

7.2

31.2

Other Certification Staff*

Median

0.0

0.0

0.0

0.0

Mean

3.4

8.3

4.2

5.1

HUD-Affiliated Auditors*

Median

0.0

0.0

0.0

0.0

Mean

0.6

1.5

2.5

1.6

OIG Auditors*

Median

0.0

0.0

0.0

0.0

Other Type of Reviewers Not

Mean

5.2

22.0

8.5

11.1

Listed Above*

Median

0.0

0.0

0.0

0.0

Note: Percentages, averages, and medians were calculated for PHAs/projects that responded to the specific items.

* Averages and medians were calculated for the PHAs/projects that indicated that they review tenant files as a QC measure.

E-22

August 31, 2016

Exhibit E-3d

Sources of Monitoring or Reviewing of Tenant Files, by Program Type

Note: Data presented in the figure were calculated for the PHAs/projects that indicated that they review tenant files as a QC

measure.

File Information Reviewed.  For those PHAs/projects that conducted dedicated QC reviews, the

most  common  file  information  typically  checked  during  QC  reviews  included  correct  income

calculations (91 percent), presence of verification documents in the tenant file (90 percent), proper

core household documentation in the tenant file (89 percent), correct medical expense calculations

(88 percent), and accurate completion of documentation in the tenant file (88 percent) (see Exhibit

E-3e). PHAs/projects were least likely to report that a general spot check of the file was performed

(15 percent), indicating that specific file information was confirmed during QC checks. This is a

decrease from FY 2014, when 19 percent of PHAs/projects performed general spot checks.

Exhibit  E-3e  provides  a  complete  breakdown  of  documentation,  verification,  calculations,

and other information typically checked during the QC process. The exhibit illustrates that

PHA-administered  Section  8  projects  were  most  likely  to  review  file  information  for  most

categories than other program types. However, Public Housing sites were most likely to review

proper  core  household  documentation  in  the  tenant  file  (90  percent),  and  Owner-administered

projects  were  most  likely  to  review  accurate  completion  of  documentation  in  the  tenant  file

(89  percent),  the  presence  of  verification  documents  (93  percent),  that  verification  documents

meet their program’s timeframe for acceptable documentation (81 percent), and that data on

the Form HUD-50058/50059 was properly entered (88 percent). Compared to FY 2014, the

percentage of PHAs/projects that reviewed items related to documentation increased for each

category. Additionally, more PHAs/projects reviewed correct income calculations (91 percent),

correct medical expense calculations (88 percent), and correct childcare expense calculations

(77 percent) than in FY 2014 (88 percent, 84 percent, and 72 percent, respectively).

HUDQC Final Report for FY 2015

E-23

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-3e

File Information Typically Reviewed During Quality Control, by Program Type

Program Type

PHA-

File Information Typically Reviewed

Public

administered

Owner-

All Program

During QC

Housing

Section 8

administered

Types

Documentation

Proper Core Household Documentation in the

Tenant File

89.8%

87.2%

89.1%

88.8%

Accurate Completion Of Documentation in the

Tenant File

87.9%

86.4%

88.5%

87.7%

Consistency of a Household’s Certification/

Interview Application to Tenant File

66.9%

80.8%

79.4%

75.4%

Documentation

Proper Unit Documentation in the Tenant File

63.7%

83.2%

70.9%

71.8%

Verification

Presence of Verification Documents in the

Tenant File

85.4%

92.0%

92.7%

89.9%

Appropriate Type of Verification

Documentation (Follows Verification Policy) Is

78.3%

89.6%

88.5%

85.2%

in the Tenant File

Verification Documents Meet Your Program’s

Timeframe for Acceptable Documentation

70.7%

79.2%

81.2%

77.0%

Calculations

Correct Income Calculation

88.5%

93.6%

90.9%

90.8%

Correct Medical Expense Calculation

87.3%

88.8%

87.9%

87.9%

Correct Disability Expense Calculation

86.6%

92.8%

80.6%

86.1%

Correct Childcare Expense Calculation

84.7%

92.0%

58.8%

77.2%

Other Information

Properly Entered Data on the Form

HUD-50058/50059

77.1%

83.2%

87.9%

82.8%

Accuracy of the Rent Adjustment or HAP

Amendment Letter in the Tenant File

51.6%

85.6%

72.1%

68.7%

General Spot Check of the File Is Performed

14.6%

12.8%

15.8%

14.5%

Correct Utility Allowance Amount Applied

to Rent Calculation, if Applicable to Your

N/A

90.4%

N/A

90.4%

Program Type

Correct Payment Standard Amount Applied to

Rent Calculation, if Applicable

N/A

89.6%

N/A

89.6%

Note: Percentages were calculated for PHAs/projects that indicated they review tenant files as a QC measure.

Prevalence of Various Types of Errors.  All PHAs/projects, regardless of whether they conduct

quality reviews, were asked to rate the frequency of errors made by the staff as a whole for various

types of rent calculation activities during the study period. They were asked whether errors in

verifications, errors in calculations, and human errors were made very often, often, sometimes,

or rarely. For each type of error related to verifications, at least 1 percent of the PHAs/projects

reported that the error occurred often or very often (see Exhibit E-3f). Missing, incomplete, or

incorrect verification of income was most likely to be reported as often or very often (4 percent).

These findings are consistent with FY 2014 results. Additionally, errors in verification of income

E-24

August 31, 2016

were sometimes found (15 percent of PHAs/projects), as were errors in verification of deductions

(16 percent), verification of assets (15 percent), and verification of allowances (11 percent).

By program type, PHA-administered Section 8 projects were most likely to report income

verification errors occurring often or very often (7 percent), while Public Housing projects were

the most likely to report the often or very often occurrence of errors in verifying assets, allowances,

and deductions (3 percent, 2 percent, and 3 percent, respectively) (see Exhibit E-3f). This is a shift

from FY 2013 and FY 2014, when PHA-administered Section 8 projects were the most likely to

report each type of verification error as occurring often or very often. Furthermore, in FY 2015,

PHA-administered Section 8 projects were most likely to report that errors in verification of income,

assets, allowances, and deductions were sometimes found (21 percent, 18 percent, 17 percent, and

23 percent, respectively). Owner-administered projects were least likely to report errors occurring

often or very often in each category; furthermore, they were least likely to report errors occasionally

occurring in each category.

Exhibit E-3f

Frequency of Missing, Incomplete, or Incorrect Verifications, by Program Type

(PH = Public Housing, VO = PHA-administered Section 8, OA = Owner-administered)

Income Verifications

Asset Verifications

HUDQC Final Report for FY 2015

E-25

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-3f

Frequency of Missing, Incomplete, or Incorrect Verifications, by Program Type, cont.

(PH = Public Housing, VO = PHA-administered Section 8, OA = Owner-administered)

Allowance Verifications

Deduction Verifications

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

Among errors related to calculations, PHAs/projects were most likely to report earned income

calculation errors as occurring often or very often (3 percent), as well as occasionally (23 percent).

This is a decrease from 5 percent and 26 percent in FY 2014, respectively. Errors were also found

often or very often in calculating fixed income (1 percent), assets (2 percent), and student income

or financial aid (2 percent) (see Exhibit E-3g). By program type, PHA-administered Section 8

projects were most likely to report making earned income calculation errors often or very often

(5 percent), while Public Housing sites were most likely to report making errors in calculating fixed

income and assets often or very often (2 percent and 3 percent, respectively). Owner-administered

projects were the most likely to report making errors often or very often in calculating student

income or financial aid (2 percent). PHA-administered Section 8 projects were the most likely to

report occasional errors for each type of calculation.

E-26

August 31, 2016

Exhibit E-3g

Frequency of Missing, Incomplete, or Incorrect Income Calculations, by Program Type

(PH = Public Housing, VO = PHA-administered Section 8, OA = Owner-administered)

Earned Income Calculations

Fixed Income Calculations

Asset Calculations

Student Income/Financial Aid Calculations

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

For errors related to expenses, 2 percent of PHAs/projects reported making medical expense

calculation errors often or very often, while less than 1 percent reported making childcare expense

calculation or disability expense calculation errors often or very often (see Exhibit E-3h). This

is a decrease since FY 2014, when at least 1 percent of PHAs/projects reported finding errors in

each category often or very often. Additionally, medical expense calculations were identified as

having the highest occasional error rate (17 percent). PHA-administered Section 8 projects reported

the highest rate of occasional errors in medical, childcare, and disability expense calculations

(22 percent, 14 percent, and 13 percent, respectively) compared with other program types.

HUDQC Final Report for FY 2015

E-27

Appendix E: Project Staff Questionnaire Descriptive Analysis

Exhibit E-3h

Frequency of Missing, Incomplete, or Incorrect Expense Calculations, by Program Type

(PH = Public Housing, VO = PHA-administered Section 8, OA = Owner-administered)

Medical Expense Calculations

Childcare Expense Calculations

Disability Expense Calculations

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

Three percent of PHAs/projects reported that errors occurred due to poor attention to

detail often or very often. With respect to program type, 3 percent of Public Housing and

Owner-administered projects reported that these errors occurred often or very often, along

with 4 percent of PHA-administered Section 8 projects (see Exhibit E-3i). This is a shift from

FY 2014, when no Owner-administered sites reported this issue often or very often, while Public

Housing and PHA-administered Section 8 sites reported relatively high numbers (7 percent

and 5 percent, respectively). Furthermore, these errors were considerably more likely to occur

occasionally  compared  to  other  types  of  calculation  and  verification  errors  (27  percent).  Late

annual certifications had the highest reported often or very often error rates of all rent calculation

E-28

August 31, 2016

activities among PHAs/projects (5 percent), with Public Housing projects being most likely to

have certification staff frequently complete late annual certification transactions (6 percent). On the

calculation of payment standards and utility allowances, errors were made very often or often at

1 percent and 2 percent of PHAs/projects, respectively. Payment standard and utility

allowance  errors  were  identified  as  sometimes  being  made  less  than  10  percent  of  the  time

(5 percent and 8 percent, respectively).

Exhibit E-3i

Frequency of Other Errors, by Program Type

(PH = Public Housing, VO = PHA-administered Section 8, OA = Owner-administered)

Errors Due to Poor Attention to Detail

Late Annual Certification

Incorrect Payment Standard

Incorrect Utility Allowance

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

Note: Traditional Payment Standards are not applicable to Public Housing or Owner-administered projects. However, a proportion of survey respondents within these programs responded to the question.

HUDQC Final Report for FY 2015

E-29

Appendix E: Project Staff Questionnaire Descriptive Analysis

Performance Feedback to Staff. In order to provide performance feedback to staff on errors found

during QC, PHAs/projects were most likely to have one-on-one conversations to discuss QC findings

with staff (73 percent) (see Exhibit E-3j. This is an increase from FY 2014, when 65 percent of

PHAs/projects provided feedback with staff in this way. Many PHAs/projects also recorded file

errors and made that information available to certification staff (49 percent), conducted team/group

meetings to discuss QC issues (39 percent), and provided general reports of QC findings to staff

(17 percent). As in FY 2014, PHA-administered Section 8 projects were most likely to use each

performance feedback method compared to other program types, while Owner-administered projects

were most likely to use a feedback method not listed (15 percent).

Exhibit E-3j

Methods of Feedback to Staff Regarding Errors Found During QC, by Program Type

Program Type

PHA-

All

Public

administered

Owner-

Program

Methods of Feedback

Housing

Section 8

administered

Types

One-On-One Conversations to Discuss QC

Findings (Phone or In-Person)

76.6%

78.0%

66.7%

73.4%

Specific Deficiencies for Each File Are Recorded

and Made Available to Certification Staff

40.3%

56.1%

51.6%

48.9%

Team/Group Meetings to Discuss QC Issues

39.6%

63.4%

20.1%

39.2%

A General Report on QC Findings Is Provided to

All Certification Staff

15.6%

22.8%

14.5%

17.2%

Other Feedback Method Not Mentioned Above

7.1%

8.1%

14.5%

10.1%

Note: Percentages were calculated for PHAs/projects that reviewed tenant files as a QC measure.

PHAs/projects were asked to comment on the frequency with which they provided feedback to

staff  on  their  performance  in  calculating  rent.  Twenty-five  percent  of  PHAs/projects  reported

they provided monthly feedback, followed by weekly (14 percent), annually (12 percent), daily

(11 percent), quarterly (10 percent), and semi-annually (4 percent) (see Exhibit E-3k). Only

7 percent of PHAs/projects reported that they did not provide any performance feedback to staff.

With respect to program type, housing projects were most likely to provide daily and weekly

feedback (12 percent and 19 percent, respectively), Owner-administered projects were most likely

to provide feedback quarterly (12 percent) and annually (18 percent), and PHA-administered Section

8 projects were most likely to provide monthly feedback (33 percent).

E-30

August 31, 2016

Exhibit E-3k

Frequency of Rent Calculation Performance Feedback to Staff, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Frequency of Feedback to Certification Staff

Housing

Section 8

administered

Types

Daily

12.3%

10.5%

9.8%

10.9%

Weekly

18.8%

16.1%

6.7%

13.6%

Monthly

22.7%

33.1%

22.1%

25.4%

Quarterly

9.1%

8.1%

11.7%

9.8%

Semi-Annually

2.6%

3.2%

4.9%

3.6%

Annually

7.8%

9.7%

18.4%

12.2%

Other Time Period

17.5%

17.7%

16.6%

17.2%

Performance Feedback Is Not Provided to Staff

9.1%

1.6%

9.2%

7.0%

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

PHAs/projects  were  asked  which  staff  were  responsible  for  file  corrections  when  errors  were

found during QC review. Overwhelmingly, the certification staff member who made the error was

responsible for making the file corrections (83 percent), while only 12 percent of PHAs/projects

required the file reviewer to make the correction (see Exhibit E-3l). PHA-administered Section 8

projects were most likely to have the certification staff member who made the error complete the file

corrections (91 percent), compared to Owner-administered projects (78 percent) and Public Housing

projects (82 percent). Owner-administered projects were most likely to have the QC reviewer correct

file errors (17 percent), compared to other program types (7 percent for PHA-administered Section

8 and 10 percent for Public Housing).

Exhibit E-3l

Staff Members Responsible for File Corrections During Quality Control, by Program Type

Program Type

PHA-

Staff Responsible for File Corrections When

Public   administered

Owner-

All Program

Errors Were Found During Quality Control

Housing

Section 8

administered

Types

The Certification Staff Who Made the Error

82.4%

91.0%

78.2%

83.3%

The Person Who Performed QC of the File

10.1%

6.6%

17.3%

11.7%

Other Correction Staff Member Not Mentioned Above

7.4%

2.5%

4.5%

4.9%

Note: Percentages were calculated for PHAs/projects that reviewed certification transactions as a QC measure.

Error Tracking and Mitigation. Twenty-eight percent of PHAs/projects reported that they do not

track any of the listed certification errors (see Exhibit E-3m). PHA-administered Section 8 projects

were least likely to report that they do not track certification errors (16 percent), compared to Owner-

administered projects (37 percent) and Public Housing projects (29 percent). PHAs/projects that

did track the types of certification errors monitored the following types of errors most frequently:

late annual certification transactions (78 percent), errors in earned income calculations (57 percent),

errors in calculating assets (56 percent), and errors in calculating medical expenses (54 percent).

HUDQC Final Report for FY 2015

E-31

Appendix E: Project Staff Questionnaire Descriptive Analysis

Public Housing sites were most likely to track late annual certification transactions (86 percent),

while PHA-administered Section 8 projects were most likely to track errors in all other categories,

at considerably higher rates than both Public Housing and Owner-administered projects.

Exhibit E-3m

Types of Rent Calculation Errors Tracked, by Program Type

Program Type

PHA-

Types of Certification Errors Tracked or

Public

administered

Owner-

All Program

Recorded

Housing

Section 8

administered

Types

None—The Project Does Not Track Any of the

Listed Information

28.5%

16.2%

36.5%

28.3%

Projects That Tracked Rent Calculation Errors

Late Annual Certification Transactions*

85.9%

80.7%

66.7%

77.9%

Errors in Earned Income Calculation*

43.8%

81.7%

47.5%

56.6%

Errors in Asset Calculation*

38.3%

72.5%

59.2%

55.7%

Errors in Medical Expense Calculation*

39.8%

71.6%

54.2%

54.3%

Errors in Maintaining Accurate Tenant File

Documentation That Is Not Related to Verification

44.5%

68.8%

40.0%

50.4%

of Income, Assets, and Expenses*

Errors Related to Accurate Verification of Income,

Assets, and Expenses*

39.8%

67.0%

46.7%

50.4%

Errors in Fixed Income Calculation*

31.3%

67.0%

45.8%

47.1%

Errors in Elderly/Disability Allowance

Determination*

33.6%

66.1%

33.3%

43.4%

Errors in Childcare Expense Calculation*

32.0%

67.0%

30.0%

42.0%

Errors in Dependent Allowance Determination*

31.3%

63.3%

30.8%

40.9%

Errors in Disability Expense Calculation*

29.7%

59.6%

30.8%

39.2%

* Percentages were calculated based on PHAs/projects that indicated they track rent calculation errors.

Overall, the majority of PHAs/projects, regardless of whether they conducted quality checks,

had a formal or informal goal-setting process in place during the study period (69 percent) (see

Exhibit E-3n). Furthermore, 73 percent of PHAs/projects required certification staff to review the

household’s previous Form HUD-50058/50059 before beginning a new certification transaction.

With respect to program type, Owner-administered projects were least likely to have a goal-setting

process  in  place,  but  most  likely  to  require  certification  staff  to  review  the  previous  Form

HUD-50058/50059 (64 percent and 81 percent, respectively). While neither of these strategies are

formal or thorough QC measures, implementing and enforcing these performance targets may help

reduce rent calculation error.

E-32

August 31, 2016

Exhibit E-3n

Percentage of PHAs/Projects With Certification Performance

Goals and Form HUD-50058/50059 Review Requirements, by Program Type

Program Type

PHA-

Public

administered

Owner-

All Program

Rent Calculation Error Mitigation Strategies

Housing

Section 8

administered

Types

Percentage of Certification Staff That Undergo a

Formal or Informal Goal-Setting Process Related

70.9%

71.5%

64.0%

68.5%

to Performing Certifications

PHA/Project Requires Certification Staff to

Review Household’s Previous Form HUD-

50058/50059 Prior to Starting New Certification

69.3%

66.9%

81.0%

73.1%

Transaction

Note: Percentages were calculated for PHAs/projects that responded to the specific items.

C. Conclusion

Overall, PSQ questions on PHA/project characteristics, certification staff training and development,

and performance management revealed a complex and interesting picture of PHAs/projects.

Demographically, there was a decrease in the number of units/households from past years,

with a slight increase in the average number of certification staff. This resulted in a declining

ratio of units per individual staff member. Since FY 2014, there has been a decrease in projects

reporting a too-high workload (49 percent to 44 percent), and an increase in projects reporting

a manageable workload (49 percent to 56 percent). The percentage of PHAs/projects that had

at least one staff member leave remained constant in FY 2015, though the average number of

staff that stopped working on certification activities at projects with turnover increased slightly

(2 staff to 3 staff). In addition, staff were much more likely than in previous years to be rated as

very organized (43 percent to 50 percent), to have very good time management (36 percent to

46 percent), and to pay a lot of attention to detail (72 percent to 80 percent). With respect to project

characteristics, virtually all of the PHA/project respondents indicated that they did not contract out

their certification activities. Additionally, almost all of the PHAs/projects indicated that they used

computer software to help calculate tenant rents and reported that the software conducts a wide

variety of tasks, with minimal limitations.

PHAs/projects were less likely to have a dedicated training department or staff trainer for certification

staff than in FY 2014 (67 percent to 59 percent). Additionally, there has been a pronounced

downward trend in the average number of hours of training provided to each new certification

staff member before performing rent calculations unassisted, with FY 2015 contributing the lowest

average in the last 6 years. Self-training and mentorship with more experienced staff were still the

most utilized training methods. For new certification staff, the training hours dedicated to mentorship

and classroom-style training administered by an outside organization increased substantially since

FY 2014, while the average number of training hours dedicated to other methods remained stable.

Additionally, there was a notable increase in training hours spent on each training topic area listed.

In contrast, for experienced certification staff, most training methods were used at fewer projects

than in FY 2014. Interestingly, there was an increase in the average number of training hours

allocated to mentoring with more experienced staff, self-training, and classroom-style training,

and a decrease in hours spent watching training videos. The gap between the average number of

HUDQC Final Report for FY 2015

E-33

Appendix E: Project Staff Questionnaire Descriptive Analysis

training hours on specific topics given to a new staff member and to an experienced staff member

has further widened.

Fewer PHAs/projects in FY 2015 had a dedicated QC staff than in FY 2014 (74 percent to

68 percent). With respect to implementing QC procedures, about the same percentage of PHA/

projects as found previously reported performing QC reviews, with the percentage of transactions

reviewed increasing slightly. The PSQ also provided some specific information on the experiences

and issues of PHAs/projects with the certification process. For instance, during the QC monitoring

of  certifications,  the  majority  of  PHAs/projects  reported  randomly  sampling  files  for  review

and,  furthermore,  that  the  most  effective  QC  technique  was  to  review  the  files  prior  to  Form

HUD-50058/50059 approval. As in FY 2014, most of these files were reviewed by supervisors

or team leaders. These results, among others, suggest that QC has become more systematic; for

example, only 15 percent reported that a general spot check was performed, compared to 19 percent

in FY 2014 and 40 percent in FY 2013. Overall, rent calculation errors were not found very often.

The most common errors made by certification staff were late annual certification transactions and

incorrect calculations of earned income. Compared to FY 2014, projects were less likely to report

errors occurring often or very often in most categories. When errors did occur, they were usually

discussed monthly in one-on-one conversations and corrected by the staff member who made the

error.

For future HUDQC studies, it would be helpful to develop and validate additional items that

specifically  target  potential  difficulties  or  barriers  to  conducting  training,  managing  staff

performance,  and  collecting  best  practices  for  error  mitigation.  Specifically,  a  dual  approach

could be used for training methods and hours. Questions about the desired amount of training

and the actual amount of training could determine the perceived importance of each topic and

method. The discrepancy between the two would likely be due to barriers that would be discussed

in an open-ended question. In addition, to provide a richer view of project practices to HUD,

the development of questions that directly link staffing and staff performance to certification and

QC procedures is desirable. Moreover, it may be useful to reinitiate questions that were asked in

previous years and dropped, particularly for project characteristics. Currently, the survey is lacking

a question on whether or not the PHA/project had new staff, which resulted in projects having to

answer questions that may not have applied. Furthermore, it may be constructive to ask about the

reasons behind staff turnover. While focus groups and cognitive interviewing may be optimal in

supporting the revision of the PSQ items by focusing attention on the specific circumstances and

issues faced by the PHAs/projects, open-ended questions also help identify and explain these issues.

For example, given the number of projects reporting that another case assignment method is used

(43 percent), transforming this question to an open-ended format may glean more insights to

assignment methods. Gathering detailed descriptions of these aspects of the certification process

would lead to a more complete and detailed picture of the issues faced by the PHAs/projects and

would provide a better link between PHA/project practices and characteristics and the estimation

of payment and income error.

E-34

August 31, 2016

Appendix F: Multivariate Analysis

Appendix F: Multivariate Analysis

A. Introduction

ICF International conducted multivariate analyses to identify project and household factors related

to rent error and project-caused errors in the certification/recertification process. The multivariate

analyses also aimed to address the extent to which error rates in projects that use an automated rent

calculation system differ from errors in those that do not and to determine whether error rates and

error costs had statistically significant difference between program types. Multivariate analysis

allows us to examine characteristics related to rent error and determine corrective actions that can

be taken to lessen improper payment.

Using measures of project characteristics and practices combined with household variables, the

multiple regression analysis sought to systematically assess the net effects of project and household

variables  on the rent error and project-caused errors.1 We addressed two research questions:

1.   Other things being equal, what project characteristics, project practices, and household

variables accounted for rent error and project-caused errors?

2.  What was the effect size (or relative strength) of project characteristics, project practice

features, project-caused errors, and household characteristics in accounting for gross rent

error?2

We developed a single conceptual model to address the stated research questions. Results from

previous years have demonstrated the relationship between project factors and project-caused errors

in connection to rent errors. In this year, we attempted to demonstrate the impact of the household

and project factors on the project-caused errors and how these factors work through project-caused

errors to impact rent error. It is hoped that these analyses will demonstrate specific areas on which

projects and quality control staffs can reduce project-caused errors and overall reduce rent error.

We used data obtained from our Project Staff Questionnaire (PSQ) and tenant data collection to

conduct this analysis.

Modeling Rent Errors.   We measured the dollar amount of rent error in terms of subsidy

overpayment, subsidy underpayment, and gross rent error. Subsidy underpayment is the dollar

error caused by the household paying more than it should have (tenant overpayment); thus, HUD’s

contribution was less than it should have been. Subsidy overpayment, conversely, is the dollar error

caused by the household’s contribution being less than it should have been (tenant underpayment),

and HUD’s contribution being more. Gross error is the dollar amount of either overpayment or

1   The term “net effect” refers to the relationship between a given independent variable and the outcome

variable, statistically controlling for other independent variables in the model. A net effect is the estimated

regression coefficient  b or slope for a given predictor in multiple regression modeling. The term does not

necessarily imply a causal effect, as this cross-sectional, survey-based design does not warrant causal

conclusions.

2   Estimation of the “effect size” for predictor variables requires valid measurement of each variable,

sensible model specifications, and a good model fit. In survey data analysis, however, it is always

challenging to obtain accurate measures of every variable and specify models that generate robust

estimates of effect sizes.

HUDQC Final Report for FY 2015

F-1

Appendix F: Multivariate Analysis

underpayment (in absolute value) for a given household. These three measures of rent error may

relate to project and household factors in different patterns; therefore, it is necessary to model each

rent error measure.

We modeled the rent error measures using a single multilevel, multivariate model that estimates both

the probability of overpayment and underpayment subsidy concurrently with amount of gross rent

error. In addition, we have also included in the model the relationships between household/project

characteristics and project-caused errors. Combining these analyses will help us better understand

the system of characteristics, practices, and project-caused errors that have either a direct or indirect

relationship on rent error measures.

Hypothetically, dollar amounts of rent errors are affected by four sets of factors: (1) project

characteristics, (2) project practices, (3) project-caused errors, and (4) household characteristics

(see Figure F-1). Project characteristics include organizational and staffing features (e.g., program

type, caseload for all staff, certification staff, and staff experience). Project practices cover different

ways to assign cases (i.e., by transaction type, certification activities, or complexity levels); training

hours and methods/topics for new and experienced staff; information technology applications;

contracting certification to outside entities; and a set of performance management or quality control

(QC) measures. QC measures range from setting performance goals, dedicating a department or

personnel to QC, or reviewing certification and recertification to a certain extent, to measuring

staff-perceived frequency of making various errors in the process.

Figure F-1

Conceptual Framework for Modeling Rent Errors

The project-caused errors will be used as predictors of rent error and as the outcomes to be modeled

through project variables and household characteristics. In previous years, we have examined

individual project-caused errors and have also developed composite scales of these errors in an

effort to understand the impact of these errors or types of errors on rent error measurement. This

year, we have tried to maintain the independence of these variables so that we are better able to

differentiate impact of these errors.

Also, as we suggested earlier, we have chosen to combine the two conceptual models from last

year to include project-caused errors not only as predictors of rent error but also as the intermediate

outcome of household characteristics and project characteristics and practices. In doing so, we

consider project-caused errors as an essential system within the model predicting rent error.

Project-caused errors were identified through investigation of household records conducted by

the  field  work.  By  default,  project-caused  errors  are  related  to  rent  errors  to  a  varying  extent.

F-2

August 31, 2016

Appendix F: Multivariate Analysis

Project-caused errors occur due to limitations in organizational resources, insufficient staff skills,

a lack of rigorous quality control, and complicated household financial situations, among other

problems. Although, in previous analyses, we have found differing effects upon rent errors, we

chose to model the independent project-caused errors to better understand the effect upon the overall

system. Examining the pattern in which project and household factors account for project-caused

errors may help housing management reduce such errors.

B. Data and Variables

We combined household and project data to conduct data processing, including examining data

quality, missing data imputation, data editing and rescaling, and derived or composite variable

construction. Before testing final multiple regression models, we conducted extensive initial data

analysis in iteration with data processing to examine data quality of key measures, the bivariate

relationship between predictor variables and outcome variables, and issues in preliminary multiple

regression models.

We matched the household records with the affiliated projects using the project identification code.

The resulting data set contains 2,400 household cases affiliated with 531 projects.3 However, only

498 projects that were affiliated with these household responded to the PSQ survey. Replicate

weights were attached to each record to calculate delete-a-group Jackknife variance estimates that

account for the stratified, multistage cluster selection of the tenant sample.

The Household Data. The household survey gathered detailed information about household

characteristics as well as final measure of rent errors. An algorithm was developed to recode/rescale

raw data items and construct composite variables.4 Using the algorithm, we produced measures

of the following:

● Project-caused errors

● Household financial conditions (e.g., income and expenses)

● Demographics (e.g., household size, number of bedrooms, elderly household with

disabilities)

The Project Data. The PSQ file contained more than 200 raw data items, with many having a large

number of categories that describe project characteristics and practices.

3 Of the 538 sampled projects (or project-like entities, hereafter referred to as projects) for the PSQ,

33 projects failed to respond; consequently, 148 households under these projects did not have PSQ data

matched. A sensitivity analysis suggested that excluding these households may introduce bias to the

analysis. Although they differed in some household variables, no difference was found in the rent error

dependent variables. Due to the lack of difference in the dependent variables and due to the lack of

knowledge about the type of these missing projects, we did not impute these records.

4  Although, in previous years, we have used this algorithm to take the logarithm of the dollar value in

order to tighten variables with skewed distribution, this year we chose to use a Tobit regression model,

which effectively accounted for the skewed distribution of the gross error dollar amount and, therefore,

did not use the logarithmic variable.

HUDQC Final Report for FY 2015

F-3

Appendix F: Multivariate Analysis

C. Methodology

Model Specification and Estimation. We conducted a combined multilevel, multivariate analysis

to account for rent errors (gross rent error, subsidy overpayment, and subsidy underpayment) and

project-caused errors. We used a multivariate regression model with logistic and linear (Tobit)

regression outcomes to model both the probability and amount of the error. Modeling the probability

(subsidy underpayment and subsidy overpayment) and amount together allowed us to differentiate

the impact of the household and project characteristics and practices on both the direction and

amount (intensity) of the rent error. In addition, we used a Tobit modeling approach to estimate

the linear regression coefficients in order to account for the skewness of the data (i.e., a large

majority of HUD applications do not have any rent error). We considered variables from all four

factors: (1) project characteristics, (2) project practices, (3) project-caused errors, and (4) household

characteristics.

The full set of variables eligible for the model are presented in Exhibit F-1. We selected the final

variables for the model in two steps. First, we calculated individual models for each variable and

identified those that had a significant relationship with gross rent error, subsidy underpayment, or

subsidy overpayment at p < 0.10. Second, we entered all the resultant variables into the model and

used a backward selection algorithm to identify the final variables for the model.5 Only the variables

from step 1 were included in the backward algorithm. Any variable that was significant at p < 0.05

was included in the model.

To examine factors underlying project-caused errors, we conducted multivariate analyses using

project characteristics, project practices, and household characteristics to account for each measure

of project-caused errors. There were five project-caused errors that were included in the model:

overdue recertification error, allowance calculation error, income calculation error, other calculation

error, and transcription error. These project-caused errors were coded so that an error was indicated

by 1 and a 0 indicated no error and included in the model as intermediary variables (i.e., outcomes

of household characteristics and predictors of rent errors).

Unless otherwise noted, we conducted statistical analyses using MPLUS with underpayment subsidy

and overpayment subsidy as categorical outcomes and gross rent error as a censored outcome with

a floor effect. Additionally, we used a two-level complex analysis type with project characteristics

and practices identified as  between variables and household characteristics and project-caused errors

in the  within level of the model. Models were executed with the weight variable, cluster variable,

and stratification variable identified.

For  descriptive  statistics,  we  used  SAS  9.4,  specifically  PROC  SURVEYMEANS  and  PROC

SURVEYFREQ. All statistics presented here were generated with sample weights and replicate

weights, using the Jackknife procedure.

5  The backward algorithm introduces all variables into the model and test for significance. Variables with

the weakest non-significant relationship with the outcome measure (i.e., p value) are removed from the

model. After each variable is removed from the model, the algorithm reevaluates the significance of other

effects in the model. Those effects that are no longer significant are removed from the model one at a

time, in order of the weakest relationship.

F-4

August 31, 2016

D. Findings

To address the first research question of identifying predictor variables that accounted for rent

error and project-caused errors, we present bivariate tabulation, regression coefficients, and related

significance test statistics to establish whether or not an effect exists beyond chance (i.e., whether

the effect is statistically significant).

Gross Rent Error

We tabulate descriptive statistics for the eligible predictor variables separately by two groups of

households: those with and those without gross rent error. This approach offers a preliminary view

of the predictor variables differentiated by gross rent error. Exhibit F-1 presents statistics of the

variables6 by the indicator of subsidy underpayment (with or without an error of $5 or more). Exhibit

F-2 presents statistics of the variables by the indicator of overpayment subsidy (with or without an

error of $5 or more). Statistically significant differences at the p < 0.05 level are noted by an asterisk.

The proportion of Public Housing Authority (PHA)-administered Section 8 projects is higher for

households with subsidy underpayment errors, while the percentage of both PHA-administered

Section 8 and Owner-administered projects is higher for households with subsidy overpayment

errors. There are no project practices that are significant between households with rent error and

those without error (underpayment or overpayment). There are a number of significant differences

on both subsidy overpayment and underpayment for tenant characteristics and project-caused errors.

Exhibit F-1

Variables Evaluated For Rent Error Modeling: Households With and Without Underpayment Error

(Original Scales, Weighted)

Households Without Error (Underpayment)

Households With Error (Underpayment)

Standard  Lower

Upper

Standard  Lower  Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Project Characteristics

Public Housing

2,081

0.229

0.004

0.222

0.237

319

0.221

0.023

0.173

0.269

PHA-administered

Section 8

2,081

0.468

0.004

0.459

0.476 *

319

0.520

0.027

0.464

0.576

Owner-administered

2,081

0.303

0.003

0.296

0.310

319

0.259

0.020

0.217

0.302

Response Across

Project

2,081

0.948

0.013

0.922

0.975

319

0.932

0.020

0.890

0.974

Cases per Staff

(in 100s)

1,957

1.038

0.043

0.948

1.127 *

295

1.158

0.091

0.968

1.349

Cases per Certification

Staff (in 100s)

1,949

1.522

0.097

1.320

1.725

295

1.676

0.155

1.352

2.000

Cases per Experienced

Certification Staff

1,948

2.361

0.405

1.516

3.207

292

2.579

0.403

1.738

3.421

(in 100s)

6   For dummy variables (coded 1 for a “yes” and 0 for a “no” response), the means are equivalent to the

percentage of households that had a value of 1.

HUDQC Final Report for FY 2015

F-5

Appendix F: Multivariate Analysis

Households Without Error (Underpayment)

Households With Error (Underpayment)

Standard  Lower

Upper

Standard  Lower  Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Move-in Cases

(in 100s) per Staff

1,957

0.140

0.008

0.123

0.157

295

0.146

0.013

0.119

0.174

Member

Move-in Cases

(in 100s) per

Certification Staff

1,949

0.200

0.010

0.180

0.220

295

0.203

0.017

0.168

0.238

Member

Move-in Cases

(in 100s) per

Experienced

1,948

0.303

0.046

0.208

0.399

292

0.308

0.052

0.198

0.417

Certification Staff

Member

Percentage of

Experienced

1,949

85.124

1.731

81.513

88.735

295

84.313

2.733

78.612 90.015

Certification Staff

Certification Staff

Turnover Rate

1,957

1.684

0.447

0.751

2.616

295

1.964

0.835

0.223

3.706

Project Practices

Assigned Case by

Transaction Type

1,957

0.199

0.028

0.140

0.258

295

0.169

0.030

0.106

0.232

Assigned Case by

Activity

1,957

0.006

0.003

-0.001

0.013

295

0.008

0.006

-0.004

0.021

Assign Case Randomly 1,957

0.157

0.020

0.115

0.199

295

0.153

0.030

0.090

0.215

Assigned Case

Nonsystematically

1,957

0.380

0.034

0.309

0.452

295

0.356

0.046

0.261

0.452

Number of Activities

Using a Computer

1,957

7.799

0.084

7.624

7.974

295

7.814

0.106

7.593

8.035

Number of Activities

Using Software

1,957

5.253

0.051

5.146

5.360

295

5.212

0.093

5.018

5.406

Project Characteristics

Contracted Out to

Perform Certifications

1,957

0.116

0.019

0.077

0.154

295

0.123

0.032

0.056

0.190

Dedicated Training

Department or

Staff Trainer Within

1,957

0.589

0.034

0.518

0.659

295

0.638

0.043

0.547

0.728

Organization

Number Reported

Training Hours for New  1,957

63.290

8.602

45.346

81.233

295

61.920

13.067

34.663 89.177

Staff

Total Number of

Training Hours for New  1,957 195.381

25.604 141.973 248.790

295 195.127

29.602 133.377 256.876

Staff on Specific Topics

Count of Methods for

Training Used for New  1,957

34.122

10.192

12.861

55.383

295

33.940

12.427

8.019 59.862

Staff

1+ Web-Based Training

for New Staff

1,957

0.493

0.029

0.432

0.554

295

0.499

0.049

0.397

0.601

F-6

August 31, 2016

Households Without Error (Underpayment)

Households With Error (Underpayment)

Standard  Lower

Upper

Standard  Lower  Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Shadowing/ Mentoring

Training for New Staff

1,957

0.725

0.028

0.666

0.784

295

0.729

0.041

0.644

0.814

Enterprise Income

Verification (EIV)

1,957

0.642

0.028

0.583

0.701

295

0.657

0.044

0.566

0.748

Training for New Staff

1+ Calculation Training

for New Staff

1,957

0.627

0.031

0.562

0.691

295

0.636

0.045

0.541

0.730

Interviewing Tenants

Training for New Staff

1,957

0.587

0.035

0.515

0.660

295

0.590

0.051

0.485

0.696

Total Number of

Training Hours for

Experienced Staff on

1,957 123.507

22.021

77.572 169.442

295 133.718

31.845

67.292 200.145

Specific Topics

Count of Methods

for Training Used for

1,957

3.755

0.102

3.541

3.969

295

3.820

0.164

3.478

4.162

Experienced Staff

1+ Web-Based Training

for Experienced Staff

1,957

0.562

0.023

0.514

0.611

295

0.573

0.046

0.477

0.669

Shadowing/

Mentoring Training for  1,957

0.558

0.028

0.499

0.617

295

0.533

0.039

0.451

0.615

Experienced Staff

EIV Training for

Experienced Staff

1,957

0.830

0.016

0.797

0.864

295

0.826

0.028

0.767

0.885

1+ Calculation Training

for Experienced Staff

1,957

0.631

0.035

0.558

0.703

295

0.640

0.036

0.566

0.714

Interviewing

Tenants Training for

1,957

0.547

0.030

0.484

0.610

295

0.537

0.039

0.456

0.618

Experienced Staff

Staff With Goal-Setting

Process for Performing  1,957

0.712

0.028

0.653

0.771

295

0.697

0.040

0.614

0.781

Certifications

Dedicated QC Staff

1,957

0.736

0.027

0.679

0.792

295

0.772

0.030

0.708

0.835

Number of Errors

Tracked

1,957

5.244

0.308

4.601

5.887

295

5.424

0.361

4.672

6.177

Certification Review

Rate

1,951

39.248

2.456

34.125

44.371

293

40.033

3.322

33.103 46.963

Frequency Making

Errors

1,950

17.119

0.362

16.364

17.873

292

16.913

0.393

16.094 17.732

100% Certifications

Reviewed

1,957

0.162

0.026

0.107

0.218

295

0.150

0.031

0.086

0.214

Randomly Select for

Review

1,957

0.632

0.020

0.590

0.674

295

0.669

0.038

0.589

0.748

New/Error-Prone/

Household Factor

1,957

0.247

0.029

0.187

0.308

295

0.277

0.028

0.219

0.335

Select for Review

Review 5 Key Info

1,957

0.654

0.026

0.601

0.707

295

0.617

0.045

0.522

0.711

HUDQC Final Report for FY 2015

F-7

Appendix F: Multivariate Analysis

Households Without Error (Underpayment)

Households With Error (Underpayment)

Standard  Lower

Upper

Standard  Lower  Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Review in 30

Days Form HUD-

1,957

0.886

0.015

0.854

0.917

295

0.878

0.029

0.818

0.939

50058/50059 Submit

Review 3-12

Months Form HUD-

1,957

0.882

0.018

0.845

0.919

295

0.867

0.031

0.802

0.933

50058/50059 Submit

Feedback Specifics

and in Person

1,957

0.348

0.032

0.282

0.415

295

0.337

0.040

0.253

0.421

Feedback Monthly or

Shorter

1,957

0.619

0.033

0.551

0.688

295

0.620

0.042

0.532

0.708

Feedback Quarterly or

Longer

1,950

0.288

0.026

0.234

0.341

292

0.290

0.031

0.224

0.355

Review by Supervisor/

Leader

1,957

0.801

0.027

0.743

0.858

295

0.793

0.036

0.718

0.867

Review by Contract

Administrator

1,957

0.150

0.032

0.083

0.217 *

295

0.192

0.037

0.115

0.269

Review by Office of

Inspector General

1,957

0.086

0.018

0.049

0.123

295

0.088

0.021

0.045

0.132

(OIG) Auditor

Review Prior Form

HUD-50058/50059

1,957

0.732

0.028

0.674

0.790

295

0.692

0.045

0.599

0.785

Household Characteristics

Number of Household

Members

2,081

2.031

0.040

1.947

2.116 *

319

2.535

0.105

2.316

2.754

Total Annual Income in

thousands

2,081

12.932

0.385

12.129

13.736 *

319

19.014

0.713

17.526 20.502

Earned Income

593

16.905

0.877

15.074

18.735 *

166

19.681

1.089

17.409 21.953

Other Income

307

3.970

0.279

3.388

4.552 *

68

5.030

0.436

4.120

5.939

Public Assistance

Income

158

4.046

0.419

3.173

4.920

22

3.445

0.593

2.207

4.683

Pension Income

1,302

11.763

0.177

11.393

12.133 *

174

13.378

0.619

12.087 14.669

Medical Expense

589

1.223

0.099

1.016

1.430 *

93

1.694

0.252

1.168

2.219

Total Number of

Sources of Income/

2,081

1.291

0.019

1.251

1.332 *

319

1.565

0.050

1.460

1.669

Expenses

Total Number of

Allowances

2,081

1.147

0.022

1.102

1.192 *

319

1.224

0.043

1.135

1.312

Age of Head of

Household

2,081

53.152

0.619

51.861

54.443 *

319

50.170

1.115

47.843 52.496

Moving to Work

2,081

0.065

0.019

0.026

0.104 *

319

0.091

0.028

0.032

0.149

Elderly Household

2,081

0.599

0.021

0.557

0.642 *

319

0.444

0.030

0.381

0.507

Dependent in

Household

2,081

0.380

0.014

0.350

0.409 *

319

0.542

0.030

0.479

0.604

Minor in Household

2,081

0.286

0.014

0.258

0.315 *

319

0.393

0.035

0.321

0.465

Disabled in Household 2,081

0.484

0.015

0.453

0.515 *

319

0.397

0.025

0.345

0.450

Total Number of

Expenses

2,081

0.311

0.017

0.275

0.347

319

0.332

0.029

0.273

0.392

F-8

August 31, 2016

Households Without Error (Underpayment)

Households With Error (Underpayment)

Standard  Lower

Upper

Standard  Lower  Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Total Amount of

Allowances

2,081

0.978

0.038

0.898

1.058 *

319

1.379

0.126

1.116

1.643

Actual Rent Paid by

the Household

2,081

0.254

0.009

0.234

0.274 *

319

0.313

0.022

0.267

0.358

Project-Caused Errors

Percent of Items With

Transcription Errors

2,081

0.352

0.013

0.324

0.380 *

319

0.849

0.030

0.786

0.912

Overdue Recertification

Error

2,081

0.011

0.003

0.006

0.017

319

0.042

0.012

0.017

0.066

Allowance Calculation

Error

2,081

0.010

0.002

0.005

0.015 *

319

0.020

0.006

0.007

0.033

Consistency Error

2,081

0.208

0.014

0.178

0.238

319

0.222

0.027

0.166

0.278

Income Calculation

Error

2,081

0.022

0.006

0.011

0.034

319

0.029

0.009

0.010

0.048

Other Calculation Error 2,081

0.034

0.005

0.023

0.045 *

319

0.055

0.012

0.030

0.080

* The two groups differ significantly in the predictor variable (p < 0.05).

Source: HUDQC FY 2015 household-level data collection and PSQ

HUDQC Final Report for FY 2015

F-9

Appendix F: Multivariate Analysis

Exhibit F-2

Variables Evaluated For Rent Error Modeling: Households With and Without Overpayment Error

(Original Scales, Weighted)

Households Without Error (Overpayment)

Households with Error (Overpayment)

Standard   Lower   Upper

Standard   Lower   Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Project Characteristics

Public Housing

2,099

0.231

0.003

0.226

0.237

301

0.207

0.018

0.170 0.244

PHA-administered

Section 8

2,099

0.465

0.003

0.458

0.472 *

301

0.541

0.021

0.498 0.585

Owner-administered

2,099

0.304

0.003

0.298

0.310 *

301

0.252

0.018

0.213 0.290

Response Across Project 2,099

0.945

0.014

0.915

0.975

301

0.952

0.011

0.930 0.974

Cases per Staff

(in 100s)

1,969

1.022

0.040

0.939

1.105 *

283

1.267

0.105

1.047 1.486

Cases per Certification

Staff (in 100s)

1,962

1.481

0.089

1.295

1.667 *

282

1.958

0.202

1.537 2.380

Cases per Experienced

Certification Staff

1,958

2.275

0.353

1.539

3.011 *

282

3.168

0.731

1.644 4.693

(in 100s)

Move-in Cases

(in 100s) per Staff

1,969

0.139

0.008

0.123

0.155

283

0.152

0.016

0.118 0.186

Member

Move-in Cases

(in 100s) per Certification  1,962

0.197

0.009

0.178

0.216 *

282

0.225

0.022

0.179 0.270

Staff Member

Move-in Cases

(in 100s) per

Experienced Certification  1,958

0.295

0.040

0.212

0.378 *

282

0.365

0.089

0.179 0.550

Staff Member

Percent of Experienced

Certification Staff

1,962

85.051

1.813

81.269 88.833

282 84.782

2.297 79.990 89.573

Certification Staff

Turnover Rate

1,969

1.771

0.546

0.632

2.909

283

1.384

0.325

0.707 2.062

Project Practices

Assigned Case by

Transaction Type

1,969

0.195

0.028

0.137

0.252

283

0.198

0.042

0.109 0.286

Assigned Case by

Activity

1,969

0.007

0.004

-0.001

0.015

283

0.004

0.004

0.004 0.012

Assign Case Randomly

1,969

0.161

0.021

0.117

0.205

283

0.125

0.035

0.053 0.198

Assigned Case

Nonsystematically

1,969

0.374

0.034

0.304

0.444

283

0.402

0.043

0.312 0.491

Number of Activities

Using a Computer

1,969

7.799

0.079

7.635

7.963

283

7.817

0.132

7.541 8.092

Number of Activities

Using Software

1,969

5.232

0.056

5.115

5.350

283

5.352

0.074

5.197 5.507

F-10

August 31, 2016

Households Without Error (Overpayment)

Households with Error (Overpayment)

Standard   Lower

Upper

Standard   Lower   Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Project Characteristics

Contracted Out to

Perform Certifications

1,969

0.118

0.020

0.077

0.159

283

0.104

0.027

0.048

0.160

Dedicated Training

Department or

Staff Trainer Within

1,969

0.590

0.033

0.521

0.658

283

0.632

0.047

0.534

0.730

Organization

Number Reported

Training Hours for New  1,969

62.396

8.939

43.750

81.043

283

67.917

11.200 44.555 91.279

Staff

Total Number of

Training Hours for New  1,969 200.840

26.935

144.654 257.025 *

283 158.226

20.162 116.169 200.284

Staff on Specific Topics

Count of Methods for

Training Used for New

1,969

35.989

11.374

12.263

59.716 *

283

21.312

2.463 16.174 26.449

Staff

1+ Web-Based Training

for New Staff

1,969

0.490

0.030

0.428

0.553

283

0.521

0.047

0.423

0.618

Shadowing/ Mentoring

Training for New Staff

1,969

0.723

0.029

0.664

0.783

283

0.738

0.034

0.666

0.809

EIV Training for New

Staff

1,969

0.638

0.028

0.579

0.697

283

0.683

0.041

0.598

0.769

1+ Calculation Training

for New Staff

1,969

0.624

0.031

0.558

0.689

283

0.657

0.043

0.567

0.746

Interviewing Tenants

Training for New Staff

1,969

0.586

0.036

0.512

0.661

283

0.599

0.041

0.513

0.684

Total Number of

Training Hours for

Experienced Staff on

1,969 125.737

23.722

76.254 175.220

283 118.962

28.583 59.339 178.586

Specific Topics

Count of Methods

for Training Used for

1,969

3.762

0.104

3.545

3.979

283

3.775

0.161

3.440

4.110

Experienced Staff

1+ Web-Based Training

for Experienced Staff

1,969

0.567

0.025

0.515

0.619

283

0.541

0.036

0.465

0.617

Shadowing/

Mentoring Training for

1,969

0.560

0.029

0.500

0.621

283

0.517

0.037

0.441

0.593

Experienced Staff

EIV Training for

Experienced Staff

1,969

0.826

0.017

0.792

0.861

283

0.853

0.019

0.813

0.893

1+ Calculation Training

for Experienced Staff

1,969

0.629

0.033

0.560

0.697

283

0.656

0.043

0.566

0.745

Interviewing

Tenants Training for

1,969

0.547

0.029

0.486

0.609

283

0.533

0.043

0.444

0.623

Experienced Staff

Staff With Goal-Setting

Process for Performing  1,969

0.709

0.027

0.653

0.765

283

0.720

0.045

0.626

0.814

Certifications

HUDQC Final Report for FY 2015

F-11

Appendix F: Multivariate Analysis

Households Without Error (Overpayment)

Households with Error (Overpayment)

Standard   Lower

Upper

Standard   Lower   Upper

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

95%

Dedicated QC Staff

1,969

0.736

0.025

0.684

0.788

283

0.771

0.041

0.687

0.856

Number of Errors

Tracked

1,969

5.241

0.296

4.624

5.858

283

5.451

0.382

4.654

6.248

Certification Review

Rate

1,961

39.652

2.498

34.442

44.862

283

37.336

3.154 30.758 43.915

Frequency Making

Errors

1,960

16.906

0.338

16.201

17.611 *

282

18.351

0.600 17.099 19.603

100% Certifications

Reviewed

1,969

0.161

0.027

0.106

0.217

283

0.156

0.029

0.096

0.216

Random Select for

Review

1,969

0.634

0.021

0.591

0.676

283

0.662

0.030

0.600

0.723

New/Error-Prone/

Household Factor

1,969

0.247

0.027

0.191

0.304

283

0.278

0.036

0.202

0.353

Select for Review

Review 5 Key Info

1,969

0.651

0.026

0.596

0.706

283

0.635

0.038

0.556

0.715

Review in 30

Days Form HUD-

1,969

0.880

0.017

0.845

0.915 * 283

0.917

0.018

0.880

0.955

50058/50059 Submit

Review 3-12

Months Form HUD-

1,969

0.875

0.020

0.833

0.917 * 283

0.918

0.019

0.879

0.957

50058/50059 Submit

Feedback Specifics and

in Person

1,969

0.352

0.032

0.286

0.418

283

0.310

0.036

0.235

0.384

Feedback Monthly or

Shorter

1,969

0.616

0.032

0.549

0.683

283

0.643

0.041

0.557

0.729

Feedback Quarterly or

Longer

1,960

0.287

0.024

0.237

0.338

282

0.290

0.030

0.227

0.353

Review by Supervisor/

Leader

1,969

0.797

0.026

0.742

0.851

283

0.821

0.047

0.722

0.919

Review by Contract

Administrator

1,969

0.158

0.032

0.091

0.224

283

0.142

0.034

0.072

0.213

Review by OIG Auditor

1,969

0.088

0.018

0.050

0.126

283

0.074

0.024

0.024

0.123

Review Prior Form

HUD-50058/50059

1,969

0.730

0.028

0.670

0.789

283

0.709

0.041

0.623

0.795

F-12

August 31, 2016

Households Without Error (Overpayment

Households with Error (Overpayment)

Standard   Lower   Upper

Standard   Lower  Variable

Variable Label

n

Mean

Error

95%

95%

n

Mean

Error

95%

Label

Household Characteristics

Number of Household

Members

2,099

2.050

0.044

1.959

2.141 *

301

2.436

0.102

2.224

2.649

Total Annual Income in

Thousands

2,099

13.601

0.399

12.770 14.432

301 14.787

0.486 13.774

15.800

Earned Income

637

18.256

0.807

16.573 19.939 *

122 13.794

0.953 11.806

15.781

Other Income

324

4.233

0.262

3.687

4.779 *

51

3.867

0.397

3.040

4.694

Public Assistance

Income

154

3.889

0.415

3.022

4.756

26

4.461

0.906

2.571

6.351

Pension Income

1,295

11.808

0.231

11.325 12.291 *

181 12.886

0.483 11.878

13.894

Medical Expense

588

1.117

0.120

0.867

1.367 *

94

2.259

0.283

1.668

2.850

Total Number of

Sources of Income/

2,099

1.304

0.020

1.262

1.346 *

301

1.492

0.043

1.402

1.581

Expenses

Total Number of

Allowances

2,099

1.124

0.022

1.078

1.170 *

301

1.384

0.052

1.275

1.494

Age of Head of

Household

2,099

52.980

0.666

51.591 54.369 *

301 51.180

1.091 48.904

53.456

Moving to Work

2,099

0.063

0.018

0.026

0.100

301

0.104

0.041

0.019

0.190

Elderly Household

2,099

0.580

0.022

0.535

0.625

301

0.568

0.034

0.496

0.640

Dependent in

Household

2,099

0.383

0.015

0.352

0.413 *

301

0.530

0.034

0.459

0.600

Minor in Household

2,099

0.292

0.014

0.262

0.322 *

301

0.359

0.034

0.287

0.430

Disabled in Household 2,099

0.466

0.015

0.434

0.497

301

0.516

0.038

0.438

0.595

Total Number of

Expenses

2,099

0.306

0.018

0.269

0.343 *

301

0.369

0.032

0.302

0.435

Total Amount of

Allowances

2,099

0.963

0.036

0.888

1.038 *

301

1.501

0.111

1.269

1.733

Actual Rent Paid by

the Household

2,099

0.255

0.010

0.234

0.276 *

301

0.308

0.016

0.275

0.341

Project-Caused Errors

Percentage of Items

With Transcription

2,099

0.365

0.011

0.342

0.388 *

301

0.786

0.038

0.706

0.865

Errors

Overdue

Recertification Error

2,099

0.012

0.003

0.006

0.018

301

0.038

0.014

0.009

0.067

Allowance Calculation

Error

2,099

0.010

0.002

0.005

0.015 *

301

0.020

0.009

0.002

0.038

Consistency Error

2,099

0.209

0.015

0.178

0.239

301

0.218

0.032

0.151

0.284

Income Calculation

Error

2,099

0.022

0.005

0.011

0.032

301

0.034

0.013

0.007

0.060

Other Calculation Error 2,099

0.032

0.003

0.025

0.039 *

301

0.069

0.022

0.023

0.114

* The two groups differ significantly in the predictor variable ( P < 0.05).

Source: HUDQC FY 2015 household-level data collection and PSQ

HUDQC Final Report for FY 2015

F-13

Appendix F: Multivariate Analysis

Multiple Logistic Regression Model Elements. In the multiple logistic regression elements, the

regression coefficient estimates are typically evaluated in terms of odds ratios.7 The odds ratio

represents the odds that an event (e.g., gross rent error) will occur given a particular characteristic

(e.g., administrative error), relative to the odds of the event occurring in the absence of that

characteristic. The odds ratio is simply the probability of the event occurring divided by the

probability of the event not occurring:  OR = p/(1-p). For example, if the probability of a gross rent

error without an administrative error is 25 percent, the odds of the event are 25 percent/75 percent,

or 1 to 3. If an administrative error increases the probability of a gross rent error to 50 percent, the

odds are 1 to 1. The odds ratio is the odds with the administrative error relative to the odds without

an administrative error, or:

The odds ratio is estimated as the given predictor’s relationship with the rent error, net of other

predictor effects (hereafter, statements to interpret regression coefficient estimates are all qualified

such that the estimated effect exists while holding other effects equal).

Multiple Linear Regression Models Elements. In the multiple linear regression elements of the

rent error model, the regression coefficient estimate indicates the given predictor’s relationship with

the rent error, net of other predictor effects (hereafter, statements to interpret regression coefficient

estimates are all qualified such that the estimated effect exists while holding other effects equal).

The estimated intercept represented a reference point for interpreting estimates of predictor effects

on gross rent error from each model. For example, the intercept estimated for gross rent error in the

final model was -0.01, which, although below the threshold of 0, still indicates expected average

gross error of a “reference” group of households that had a zero value on each predictor variable in

the model. For binary-coded predictors such as program type (e.g., Public Housing), the zero value

represented the projects in programs other than the given one—in this case, PHA-administered

Section 8 or Owner-administered programs. For project-caused errors, the zero value indicates there

was not an instance of a particular error for that observation.

A coefficient estimate for a predictor, if statistically significant, represents the difference from the

“reference” value in gross rent error associated with this predictor. We focused on interpreting the

regression coefficients that were statistically significant (p ≤ 0.05), as they represented effects that

were unlikely to be due to chance.

Gross Error Element. Overall, the final model differed from previous years in that no project

characteristics were found to significantly impact gross rent error. Alternatively, project practices,

tenant  characteristics,  and  project-caused  errors  continued  to  have  significant  findings.  In  the

multilevel multivariate model, we found the following variables were significant predictors of the

amount of gross rent error (n = 2,157):

● Project characteristics: No project characteristics were significant as related to the amount

of gross rent error.

7  The estimated coefficients in the logistic model are the log odds ratios, log (OR). The odds ratios are

obtained as  eb, where  e is a constant of approximately 2.718 and  b is the estimated logistic regression coefficient.

F-14

August 31, 2016

● Project practices: A single project practice was associated with the amount of rent

error, when EIV training is conducted with new staff. However, the result suggests that

conducting the training increases the amount of gross rent error ($6.54). One reason for

this relationship is that staff may focus on the EIV as a resource and neglect to use other

sources of income verification.

● Tenant characteristics: The amount of the rent error increases for households as household

income increases ($3.74). Gross rent error also increases with the actual amount rent paid by

the  household ($0.15). Alternatively, increases in the amount of total allowances decreases

the amount of gross rent error ($3.07).

● Project errors: Two project errors were significant predictors: overdue recertification error

and transcription errors. Overdue recertification errors have the largest impact on the value

of the rent error, with a $45.48 increase. Transcription errors also have a large impact on

the amount of the error, with an increase in $16.62.

The model continued to suggest that the amount of gross rent error is tied to the household financial

measures, including annual income, rent, and total amount of allowances. While these metrics tend

to conform to the models from previous years, more sociodemographic tenant measures seem to

impact the amount of gross rent error indirectly through the project-caused transcription error. We

will explore these indirect relationships later in this section.

Figure F-1 presents the predictors found to be statistically significant in the gross rent error model.

HUDQC Final Report for FY 2015

F-15

Appendix F: Multivariate Analysis

Figure F-1

Statistically Significant Predictors of Gross Rent Error:

Multiple Regression Dollar Value Net Effects From the Final Model

*p < 0.05, **p < 0.01, *** p < 0.001 (test with the null hypothesis that a coefficient = 0; a significant result indicates that the corresponding predictor is associated with the gross rent error)

Estimate represents the unique effect of a given predictor, net of other effects in the model.

Significant effects based on design adjusted variance using delete-a-group Jackknife.

Source: HUDQC FY 2015 household-level data collection and PSQ

F-16

August 31, 2016

Overpayment and Underpayment

As described earlier, we incorporated these two subsidy errors into the final model to help determine

variables that impact the direction of gross rent error. As with the amount of gross rent error, we

initially included all tenant characteristics, project characteristics, practices, and errors as predictors

of the direction of rent error. Each of these rent errors were modeled using logistic regression

analyses that were elements of the iterative full model. Figures F-4a and F-4b present statistically

significant results.

An analysis of overpayment and underpayment rent errors produced additional information that may

improve program ability to deal with more specific rent errors. We estimated two equations with the

same four sets of predictors, rescaled into a logarithm, as those used in modeling gross rent error to

explain overpayment and underpayment. Figures F-4a and F-4b present the statistically significant

results of underpayment and overpayment respectively. Many of the effects, especially in regard to

subsidy underpayment, are similar to the amount of gross rent error, including transcription error,

overdue recertification error, total annual household income, and household rent. Alternatively, the

number of dependents in the household, number of allowances, and certified staff caseload also

increase the possibility of subsidy overpayment, while having a transcription error also increases

the likelihood of overpayment.

Figure F-2a and F-2b present the log odds of the statistically significant predictors of underpayment

and overpayment.

HUDQC Final Report for FY 2015

F-17

Appendix F: Multivariate Analysis

Figure F-2a

Statistically Significant Predictors of Underpayment Rent Error Indicators (1 = Error, 0 = No error):

Logistic Regression Odds Ratios From the Final Model

*p < 0.05, **p < 0.01, *** p < 0.001 (test with the null hypothesis that a coefficient = 0; a significant result indicates that the corresponding predictor is associated with the subsidy underpayment)

Estimate represents the unique effect of a given predictor, net of other effects in the model.

Significant effects based on design adjusted variance using delete-a-group Jackknife.

Source: HUDQC FY 2015 household-level data collection and PSQ

F-18

August 31, 2016

Figure F-2b

Statistically Significant Predictors of Overpayment Rent Error Indicators (1 = Error, 0 = No error):

Logistic Regression Odds Ratios From the Final Model

*p < 0.05, **p < 0.01, *** p < 0.001 (test with the null hypothesis that a coefficient = 0; a significant result indicates that the corresponding predictor is associated with the subsidy overpayment)

Estimate represents the unique effect of a given predictor, net of other effects in the model.

Significant effects based on design adjusted variance using delete-a-group Jackknife.

Source: HUDQC FY 2015 household-level data collection and PSQ

HUDQC Final Report for FY 2015

F-19

Appendix F: Multivariate Analysis

Underpayment. A number of predictors of household characteristics and project-caused errors were

associated with underpayment in patterns similar to those for gross error. We present the significant

effects for modeling the probability of an underpayment (n = 2,147):

● Project characteristics: There are no significant effects based on project characteristics.

This is consistent with the FY 2013 and FY 2014 analyses.

● Project practices: There  are  no  significant  effects  based  on  project  practices  differing

from the findings in FY 2014.

● Tenant characteristics: The odds of subsidy underpayment is 1.99 times higher when a

tenant is in the Moving to Work program. In addition, the odds of underpayment are also

increased by 1 percent for each $1,000 increase in the tenant’s total income. Alternatively,

we see decreases in the probability of underpayment by 7 percent with incremental increases

in actual rent paid by the household, and by 31 percent if there is an adult over 62 in the

household.

● Project errors:  Two  project  errors  were  significant  predictors:  transcription  error  and

overdue recertification error. Transcription error was the largest effect in the model with

an odds ratio of 9.74. An overdue recertification error increased the likelihood of a subsidy

underpayment by 4.14 times.

Overpayment.  In modeling overpayment error, there were fewer significant predictors than the

underpayment and gross error model. We present the significant effects for modeling the probability

of an underpayment and then the significant effects for modeling the value of the effect (n = 2,147):

●  Project characteristics: A higher number of cases per certified staff is associated with an

increase in the possibility of overpayment. An increase of 100 cases per staff increases the

possibility of overpayment by 10 percent.

●  Project practices: No project practices are significant predictors of the probability of a

subsidy overpayment.

●  Tenant characteristics: There is a 69 percent increase in the odds of a subsidy overpayment

for each increase in the number of tenant allowances. In addition, there is a 47 percent

increase in the likelihood of a subsidy overpayment if there are dependents in the household.

●  Project errors:  Transcription  error  is  the  only  significant  project  error  predicting  the

likelihood of subsidy overpayment. It had the largest effect on overpayment, increasing the

odds of overpayment 5.70 times.

While the probability of overpayment and underpayment subsidy only shared the transcription

project-caused  error,  in  both  cases  it  appeared  to  have  significant  impact  on  the  likelihood  of

subsidy error. While no household characteristics were identified as predictors of subsidy error,

this may be an effect of modeling household characteristics as a predictor of project-caused errors

as intermediate variables.

F-20

August 31, 2016

Project Error Models

In modeling gross rent errors, underpayments, and overpayments, we included five project-caused

errors:  overdue  recertification  error,  transcription  error,  consistency  error,  income  calculation

error, allowance calculation error, and other calculation error. Of these, we found only overdue

recertification error and transcription error to be significant predictors of the amount or direction

of gross rent error. As this analysis is focused on the impact of variables directly and indirectly

on gross rent error, we focus on the variables that were significantly related to these two project-

caused errors.

Overdue Recertification Error: While only a small number of cases reported overdue recertification

error, the impact of overdue recertifications was significant on the amount of gross rent error and

the likelihood of subsidy underpayment. Our model indicated that higher numbers of experienced

certified project staff increased the percent likelihood (0.1%) of an overdue recertification error.

While this finding may be counter-intuitive, it may suggest that although rare, larger projects may

run the risk of not getting certifications or recertifications completed on time.

Transcription Error: In contrast to overdue recertification error, transcription error was found

to  be  a  significant  predictor  of  both  the  amount  and  direction  (via  subsidy  underpayment  and

overpayment) of gross rent error. Additionally, our findings seem to suggest that the majority

of household indicators and the Owner-administrator project type identified in previous years as

significant predictors of gross rent error may actually effect rent error through transcription error.

Specifically, household member characteristics reduced the likelihood of a transcription error by

79 percent if there was at least one adult aged 62 or older, by 52 percent if there were dependents

in the household, and by 40 percent if household included a minor. Alternatively, we did see that

an increase in the age of the head of household increased the likelihood of a transcription error by

1 percent.

Alternatively, our findings showed that household financial variables were more complex in their

impact on transcription errors. While the greater increase in expenses (e.g., medical, childcare,

disability) a household had reduced the likelihood of a transcription error by approximately

33 percent, total annual income and the number of sources of income increased the likelihood of a

transcription error by 3 percent and 55 percent, respectively. In addition, we found that the number

of different allowances a household reported was the largest predictor of likelihood, with each

additional allowance increasing the likelihood of a transcription error by 4.24 times.

Finally, the results also depicted that Owner-administered projects were 21 percent less likely to

have a transcription error than other types of projects.

Indirect Effects. The combined model allowed us to determine if the household characteristics,

project practices and project characteristics that impact project-caused errors as proximal outcomes

also significantly impacted gross rent error measures through project-caused errors. We found that

all significant indicators of project-caused errors (previously described) also had significant indirect

effects on gross rent errors and subsidy overpayment and underpayment. These results confirm our

reasoning that household characteristics appear to act through project-caused errors and, therefore,

were not found as significant direct indicators (i.e., differing from previous year models).

Figure F-3 presents the log odds of the statistically significant predictors of transcription error.

HUDQC Final Report for FY 2015

F-21

Appendix F: Multivariate Analysis

Figure F-3

Statistically Significant Predictors of Transcription Error Indicators (1 = Error, 0 = No error):

Logistic Regression Odds Ratios From the Final Model

*p < 0.05, **p < 0.01, *** p < 0.001 (test with the null hypothesis that a coefficient = 0; a significant result indicates that the corresponding predictor is associated with having a transcription error)Estimate represents the unique effect of a given predictor, net of other effects in the model.

Significant effects based on design adjusted variance using delete-a-group Jackknife.

Source: HUDQC FY 2015 household-level data collection and PSQ

F-22

August 31, 2016

E. Summary of Findings

The FY 2015 HUDQC multivariate modeling modified the approach used in previous years by

creating a single combined model. The new approach continued to analyze variables representing

project characteristics, project practices, tenant characteristics, and project-caused errors as they

relate to gross rent error, subsidy underpayments, and subsidy overpayments. In addition, this year’s

approach continued to examine the relationship between household and project characteristics

and project-caused errors. However, instead of examining the relationships in separate models,

we included all the relationships in a single iterative model that allowed us to perform logistic

and linear modeling concurrently. We suggested that examining the full system of variables using

an integrated modeling approach would allow us to better understand the impact of the predictor

variables on project-caused errors and the amount and direction of gross rent error while accounting

for the covariance between the outcome variables. Key findings are highlighted as follows.

Rent Errors

Project characteristics were not generally found to be predictive of the probability of a rent error.

The only characteristic found significant was the cases per certified staff, which was positively

associated with the likelihood of subsidy overpayment (more cases per certified staff caused an

increasing likelihood of subsidy overpayment).

Of numerous  project practice indicators, only projects that conduct EIV training for new staff are

associated with an increase in the probability of rent error.

Project-caused errors represented the largest effects in measuring rent error. Two indicators

were most salient: (1) transcription errors and (2) overdue recertification errors (just 2 percent

of recertifications in FY 2015). Both predicted substantially higher rent error. With regards to

the amount of gross rent error, overdue recertification errors had the largest effect. However, for

underpayment and overpayment subsidy, transcription errors were more salient. The rent error model

suggests that reducing project-caused errors will reduce rent errors. The model elements focused

on predicting project-caused errors may demonstrate significant impacts on rent errors.

Household characteristics play a significant role in explaining project errors, and were, thereby,

also found to impact gross rent error. Both household financial conditions and demographics present

varying levels of challenge to transcription and overdue recertification processing. Over the years,

the basic patterns have been quite consistent and fit common sense (i.e., a household’s complex

financial situations raise the risk of errors in determining rent). Some key predictors are:

● Households with higher total income and fewer total allowances had higher amounts of

gross rent error.

● Households with higher total income and without an elderly member were more likely to

have subsidy underpayment; while those that had dependents were more likely to have

subsidy overpayment.

● Household membership, including an adult over age 62, minors, and at least one dependent,

decreased the likelihood of a transcription error.

HUDQC Final Report for FY 2015

F-23

Appendix F: Multivariate Analysis

● Complex  household  finances,  including  higher  total  income,  more  types  of  allowances,

and more sources of income, indicated a higher likelihood of transcription error; however,

fewer types of expenses decreased the possibility of a transcription error.

Implications for Program Improvement

We underscored a number of implications for HUD’s subsidy improper payment management.

First, the three housing programs were not substantially different in rent error after project and

household variables were considered. This finding suggests that targeting a particular program

type for strengthening financial integrity may not be as effective as targeting specific problems

underlying rent errors across all program types (e.g., reducing project-caused mistakes in (re)

certification processing).

Project-caused  errors  in  certification  processing  represent  the  largest  effects  with  rent  error,

particularly transcription errors. Such errors can lead to increased amounts of subsidy overpayment or

underpayment. Reducing project-caused errors should be a priority for reducing the rate of improper

payment. The significant project-caused variables are highly related to household characteristics

(sources of income/expenses, number of allowances, household with elderly members or other

dependents). Households with more complex financial situations are more susceptible to project

errors and, thus, more susceptible to rent errors.

Underpayment and overpayment relate to different issues, and may require different strategies to

remedy. The only common effect among the two is transcription error. This is the strongest predictor

of rent error probability for both the amount and direction (underpayment and overpayment) of rent

error. Total annual income, actual household-paid rent, and overdue recertification processing are

significant factors contributing to the amount of rent error and subsidy underpayment. In addition,

the total amount of allowances also significantly impacts the amount of rent error, while having an

adult over 62 in the household and being part of the Moving to Work program indicated potential

for subsidy underpayment. Alternatively, having dependents in the household and large numbers

of allowances increased the possibility of subsidy overpayment.

Future Research

The new approach evaluated the effects of both direction and amount of rent error by examining

both the logistic and linear relationships and the proximal (project-caused errors) and distal (gross

rent errors) concurrently within the model. While a number of results are consistent with prior

studies, the path through which these predictors impact the distal outcome of rent errors may have

emerged. Increasing our understanding of these effects on rent errors as well as their relationship

with other predictors (e.g., project-caused errors) will allow focused efforts to reduce errors. Our

findings have shown that a number of housing characteristics impact rent error through the project-

caused transcription error. Using these characteristics (e.g., large sources of income, large numbers

of allowances) to identify specific households that could be at risk for improper payment in advance

of  the  (re)certification process  could  allow  for  increased quality control  for  these  cases  or  the

development of specific training around these at-risk households.

F-24

August 31, 2016

The project practices that emerged from the modeling seem to be inconsistent in their relationship

with rent errors (e.g., increased training associated with increased rent error). Further investigating

these counterintuitive effects could help identify whether the effect is truly associated with rent error

or whether there are underlying factors that are confounded with rent error. This will improve clarity

on specific challenges facing certification staff that lead to processing errors and payment errors.

As stated in the FY 2014 report, historical analysis of the HUDQC data seems both feasible and

desirable. Thus far, the analytic task has focused solely on a single year. With more than a decade of

data accumulated by the annual surveys, a great deal of comparable measures of improper payment

errors, project background/activities, and household characteristics are available. The large amount

of data would allow in-depth analyses to describe the changing patterns of these important measures

and their relationships. In a broad chronological framework, historical data analysis may shed light

on how much progress HUD has made in dealing with housing subsidy improper payment, and how

such progress can be attributed to changes in housing program practices in relation to changes in

the sociodemographic characteristics of program participants. Information yielded from historical

analysis could be useful for HUD’s long-term planning and program development to improve

financial integrity.

HUDQC Final Report for FY 2015

F-25

Appendix F: Multivariate Analysis

References

Cohen, J. (1988).  Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ:

Erlbaum.

Harrell, F. E. (2001).  Regression modeling strategies. New York: Springer.

Hosmer, D. W. & Lemeshow, S. (2001).  Applied logistic regression, textbook and solutions manual.

New York: Wiley & Sons.

Raudenbush, S. W. & Bryk, A. S. (2002).  Hierarchical linear models: Applications and data

analysis methods (2nd ed.). Thousand Oak, CA: Sage Publishing.

F-26

August 31, 2016

Document Outline

- Executive Summary

- I. Introduction

- II. Methodology

- III. Study Objectives and Analytic Methods

- IV. Findings

- V. Recommendations

- Appendix A: Rent Calculations

- Appendix B: Weighting Procedure

- Appendix C: Source Tables

- Source Tables Based on Quality Control Data

- Source Tables Based on Tenant File Data

- Appendix D: Consistency and Calculation Errors

- Appendix E: Project Staff Questionnaire Descriptive Analysis

- Appendix F: Multivariate Analysis

- Exhibit ES-1 Frequency of Rent Error by Program Type

- Exhibit ES-2 Subsidy Dollar Error

- Exhibit ES-3 Estimates of Error in Program Administrator Income and Rent Determinations (in $1,000s)

- Exhibit ES-4 Comparative 2000 Through FY 2015 Gross Erroneous Payments*

- Exhibit ES-5 Rent Components Responsible for the Largest Dollar Error for Households With Rent Error

- Exhibit ES-6 Impact of Changes in the Error Threshold on Frequency and Estimates of Error (in $1,000s)

- Exhibit II-1 Change in Frame Population Totals Used to Pull the Study Sample Over Time

- Exhibit III-1 PHA-administered Section 8 Unit Size Standards

- Exhibit III-2 Summary of Study Objectives

- Exhibit IV-1 Percentage of Households With Proper Payments

- Exhibit IV-2 Percentage of Households With Error, Average Dollars in Error, and Dollar Error Rate for All Households With Error

- Exhibit IV-3a Underpayment Households: Percentage of Households and Average Monthly Dollar Amount of Error

- Exhibit IV-3b Overpayment Households: Percentage of Households and Average Monthly Dollar Amount of Error

- Exhibit IV-4 Gross and Net Dollar Rent Error (Monthly) for All Households

- Exhibit IV-5 Gross and Net Dollar Error Rates (Monthly) for All Households

- Exhibit IV-6 Certifications and Recertifications by Administration Type*

- Exhibit IV-7a Percentage of Newly Certified Households Meeting Certification Criteria

- Exhibit IV-7b Percentage of Newly Certified Households Meeting Certification Criteria by Program Type

- Exhibit IV-8 Average Monthly Underpayment and Overpayment: Dollar Amounts Averaged Across All Households

- Exhibit IV-9a Positive Subsidy Households (Tenant Underpayment) Percentage of Households and Average Monthly Dollar Amount of Error

- Exhibit IV-9b Negative Subsidy Households (Tenant Overpayment) Percentage of Households and Average Monthly Dollar Amount of Error

- Exhibit IV-10 Average Monthly Underpayment and Overpayment: Dollar Amount Averaged Across All Households

- Exhibit IV-11 Rent Components Responsible for the Average Annual Largest Dollar Error for Households With Rent Error

- Exhibit IV-12 Total and Annual Component Dollars in Error for Households With Rent Error

- Exhibit IV-13 Percentage of Households With Rent Component Error by Payment Type

- Exhibit IV-14 Percentage of Households With Elderly/Disabled Allowances and Dependent Allowances*

- Exhibit IV-15 Findings With and Without Information Obtained From Sources Other Than the Tenant File

- Exhibit IV-16 Percentage of Households With Calculation and Consistency Errors

- Exhibit IV-17 Form HUD-50058/50059 Rent Calculation Error Compared With QC Rent Error

- Exhibit IV-18 Percentage of Households Fully Verified by Either the PHA/Owner or ICF

- Exhibit IV-19 Verification of Form HUD-50058/50059 Rent Components by PHA/Owners**

- Exhibit IV-20 Verification of Form HUD-50058/50059 Rent Components by PHA/Owner Staff, by Form Type

- Exhibit IV-21 QC Error Households With Missing Verification in the Tenant File

- Exhibit IV-22a Form HUD-50058/50059 Administrative Error: Percentage of Households, Average Dollars in Error (Non-MTW Households)

- Exhibit IV-22b Form HUD-50058/50059 Administrative Error: Percentage of Households, Average Dollars in Error (Households With QC Rent Error)

- Exhibit IV-23 Administrative Error: Percentage of Households Average Dollars in Error for All Households

- Exhibit IV-24a Percentage of Households in Units With the Correct Number of Bedrooms (According to Study Guidelines)

- Exhibit IV-24b Percentage of All Households in FY 2014 by Number of Bedrooms and Number of Household Members*

- Exhibit IV-24c Percentage of All Households in FY 2015 by Number of Bedrooms and Number of Household Members*

- Exhibit IV-25 PHAs by Predominant Rent Reasonableness Method*

- Exhibit IV-26a Rent Reasonableness Documentation for New Admissions and Annual Recertifications

- Exhibit IV-26b Type of Rent Reasonableness Documentation for New Admissions and Annual Recertifications*

- Exhibit IV-27 Timing of Most Recent Rent Reasonableness Determination: New Admissions and Annual Recertifications*

- Exhibit IV-28a Types of Documents Used by PHAs to Identify Utilities and Calculate the Utility Allowance Value*

- Exhibit IV-28b Comparison of Utility Allowance on the Form HUD-50058 to the Utility Allowance Worksheet

- Exhibit IV-28c Availability of All Information to Enable QC Utility Allowance (UA) Calculation*

- Exhibit IV-28d QC Utility Allowance Compared to Form HUD-50058 Utility Allowance

- Exhibit IV-29a Number and Percentage of Households With Payment Standard Discrepancies

- Exhibit IV-29b Payment Standard Compared With the Fair Market Rent

- Exhibit IV-29c Details of Cases Falling Outside 90%–110% of the Fair Market Rent*

- Exhibit IV-29d Comparison of the FY 2014 to FY 2015 Payment Standard Analysis

- Exhibit IV-30a PIC/TRACS Data by Program Type and Average Gross Dollars in Error for All Households

- Exhibit IV-30b PIC/TRACS Data by Program Type and Average Gross Dollars in Error for Households in Error

- Exhibit IV-30c Average Gross Dollars in Error by Payment Type and PIC/TRACS Data

- Exhibit IV-30d Average Net and Gross Dollars in Error by Administration Type and PIC/TRACS Data for All Households

- Exhibit IV-30e Percentage of Matched and Non-Matched Dollar Amounts for Key Variables Matching Variables From the Form HUD-50058/50059 and PIC/TRACS

- Exhibit IV-31a Administrative Errors in the 20 Largest PHAs

- Exhibit IV-31b Dollar Rent Errors in the 20 Largest PHAs*

- Figure ES-1 Comparative 2000 through FY 2015 Gross Erroneous Payments Over Time

- Figure IV-1 Payment by Program Type

- Figure IV-2 Percentage of Cases by Case Type

- Figure IV-3 Example of the Impact of Component Errors

- Figure IV-4 PIC/TRACS Data Present by Program Type for All Households Over Time

- Figure IV-5 Conceptual Model Used in Multivariate Analysis

Table of Contents

Executive Summary

I. Introduction

II. Methodology

III. Study Objectives and Analytic Methods

IV. Findings

V. Recommendations

Appendix A: Rent Calculations

Appendix B: Weighting Procedure

Appendix C: Source Tables

Source Tables Based on Quality Control Data

Source Tables Based on Tenant File Data

Appendix D: Consistency and Calculation Errors

Appendix E: Project Staff Questionnaire Descriptive Analysis

Appendix F: Multivariate Analysis

Exhibit ES-1 Frequency of Rent Error by Program Type

Exhibit ES-2 Subsidy Dollar Error

Exhibit ES-3 Estimates of Error in Program Administrator Income and Rent Determinations (in $1,000s)

Exhibit ES-4 Comparative 2000 Through FY 2015 Gross Erroneous Payments*

Exhibit ES-5 Rent Components Responsible for the Largest Dollar Error for Households With Rent Error

Exhibit ES-6 Impact of Changes in the Error Threshold on Frequency and Estimates of Error (in $1,000s)

Exhibit II-1 Change in Frame Population Totals Used to Pull the Study Sample Over Time

Exhibit III-1 PHA-administered Section 8 Unit Size Standards

Exhibit III-2 Summary of Study Objectives

Exhibit IV-1 Percentage of Households With Proper Payments

Exhibit IV-2 Percentage of Households With Error, Average Dollars in Error, and Dollar Error Rate for All Households With Error

Exhibit IV-3a Underpayment Households: Percentage of Households and Average Monthly Dollar Amount of Error

Exhibit IV-3b Overpayment Households: Percentage of Households and Average Monthly Dollar Amount of Error

Exhibit IV-4 Gross and Net Dollar Rent Error (Monthly) for All Households

Exhibit IV-5 Gross and Net Dollar Error Rates (Monthly) for All Households

Exhibit IV-6 Certifications and Recertifications by Administration Type*

Exhibit IV-7a Percentage of Newly Certified Households Meeting Certification Criteria

Exhibit IV-7b Percentage of Newly Certified Households Meeting Certification Criteria by Program Type

Exhibit IV-8 Average Monthly Underpayment and Overpayment: Dollar Amounts Averaged Across All Households

Exhibit IV-9a Positive Subsidy Households (Tenant Underpayment) Percentage of Households and Average Monthly Dollar Amount of Error

Exhibit IV-9b Negative Subsidy Households (Tenant Overpayment) Percentage of Households and Average Monthly Dollar Amount of Error

Exhibit IV-10 Average Monthly Underpayment and Overpayment: Dollar Amount Averaged Across All Households

Exhibit IV-11 Rent Components Responsible for the Average Annual Largest Dollar Error for Households With Rent Error

Exhibit IV-12 Total and Annual Component Dollars in Error for Households With Rent Error

Exhibit IV-13 Percentage of Households With Rent Component Error by Payment Type

Exhibit IV-14 Percentage of Households With Elderly/Disabled Allowances and Dependent Allowances*

Exhibit IV-15 Findings With and Without Information Obtained From Sources Other Than the Tenant File

Exhibit IV-16 Percentage of Households With Calculation and Consistency Errors

Exhibit IV-17 Form HUD-50058/50059 Rent Calculation Error Compared With QC Rent Error

Exhibit IV-18 Percentage of Households Fully Verified by Either the PHA/Owner or ICF

Exhibit IV-19 Verification of Form HUD-50058/50059 Rent Components by PHA/Owners**

Exhibit IV-20 Verification of Form HUD-50058/50059 Rent Components by PHA/Owner Staff, by Form Type

Exhibit IV-21 QC Error Households With Missing Verification in the Tenant File

Exhibit IV-22a Form HUD-50058/50059 Administrative Error: Percentage of Households, Average Dollars in Error (Non-MTW Households)

Exhibit IV-22b Form HUD-50058/50059 Administrative Error: Percentage of Households, Average Dollars in Error (Households With QC Rent Error)

Exhibit IV-23 Administrative Error: Percentage of Households Average Dollars in Error for All Households

Exhibit IV-24a Percentage of Households in Units With the Correct Number of Bedrooms (According to Study Guidelines)

Exhibit IV-24b Percentage of All Households in FY 2014 by Number of Bedrooms and Number of Household Members*

Exhibit IV-24c Percentage of All Households in FY 2015 by Number of Bedrooms and Number of Household Members*

Exhibit IV-25 PHAs by Predominant Rent Reasonableness Method*

Exhibit IV-26a Rent Reasonableness Documentation for New Admissions and Annual Recertifications

Exhibit IV-26b Type of Rent Reasonableness Documentation for New Admissions and Annual Recertifications*

Exhibit IV-27 Timing of Most Recent Rent Reasonableness Determination: New Admissions and Annual Recertifications*

Exhibit IV-28a Types of Documents Used by PHAs to Identify Utilities and Calculate the Utility Allowance Value*

Exhibit IV-28b Comparison of Utility Allowance on the Form HUD-50058 to the Utility Allowance Worksheet

Exhibit IV-28c Availability of All Information to Enable QC Utility Allowance (UA) Calculation*

Exhibit IV-28d QC Utility Allowance Compared to Form HUD-50058 Utility Allowance

Exhibit IV-29a Number and Percentage of Households With Payment Standard Discrepancies

Exhibit IV-29b Payment Standard Compared With the Fair Market Rent

Exhibit IV-29c Details of Cases Falling Outside 90%–110% of the Fair Market Rent*

Exhibit IV-29d Comparison of the FY 2014 to FY 2015 Payment Standard Analysis

Exhibit IV-30a PIC/TRACS Data by Program Type and Average Gross Dollars in Error for All Households

Exhibit IV-30b PIC/TRACS Data by Program Type and Average Gross Dollars in Error for Households in Error

Exhibit IV-30c Average Gross Dollars in Error by Payment Type and PIC/TRACS Data

Exhibit IV-30d Average Net and Gross Dollars in Error by Administration Type and PIC/TRACS Data for All Households

Exhibit IV-30e Percentage of Matched and Non-Matched Dollar Amounts for Key Variables Matching Variables From the Form HUD-50058/50059 and PIC/TRACS

Exhibit IV-31a Administrative Errors in the 20 Largest PHAs

Exhibit IV-31b Dollar Rent Errors in the 20 Largest PHAs*

Figure ES-1 Comparative 2000 through FY 2015 Gross Erroneous Payments Over Time

Figure IV-1 Payment by Program Type

Figure IV-2 Percentage of Cases by Case Type

Figure IV-3 Example of the Impact of Component Errors

Figure IV-4 PIC/TRACS Data Present by Program Type for All Households Over Time

Figure IV-5 Conceptual Model Used in Multivariate Analysis