Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Quality Control for Rental Assistance Subsidy Determinations Final Report for FY 2007 U.S. Department of Housing and Urban Development Oﬃce of Policy Development and Research

Visit PD&R’s Web Site www.huduser.org to find this report and others sponsored by HUD’s Office of Policy Development and Research (PD&R). Other services of HUD USER, PD&R’s Research Information Service, include listservs; special interest reports, bimonthly publications (best practices, significant studies from other sources); access to public use databases; hotline 1-800-245-2691 for help accessing the information you need.

Quality Control for

Rental Assistance

Subsidies Determinations

Final Report for FY 2007 Prepared for: U.S. Department of Housing and Urban Development Office of Policy Development and Research Prepared by: Macro International Calverton, MD

October 15, 2008 Acknowledgments This report was prepared by Macro International of Calverton, MD and could not have been done without the deep expertise and hard work of the following: JoAnn Kuchak, Officer in Charge; Mary K. Dent Sistik, Project Manager; Sophia Zanakos, PhD., Deputy Project Manager; Analysts and Programmers Pedro Saavedra, Ph.D., Francine Barrington, Gary Huang, Ph.D., Michelle Wilson. Cecille Witt, and Martin Wulfe; Expert Consultants Judy Lemeshewsky, Judy Payne and Frank Wann; Survey Managers Jagruti Patel, Karen Smith and Laura Webb; Field Supervisors, Data Quality Specialists and Support Staff Meagan Canali, Brian Connolly, Ebele Ebonwu, Safal Khatri, Helene Mulligan, Diana Oresky, Wilma Samuels-Reeves, Rebecca Villagomez, Andrew Walton, and Caren Williams.

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

Table of Contents Executive Summary ................................................................................................................. ES-i

I. Introduction.....................................................................................................................I-1

A. Purpose of the Quality Control for Rental Assistance Subsidy

Determinations Study for FY2007..............................................................................I-1

B. Background of the Study ............................................................................................I-1

C. Organization of This Report .......................................................................................I-2

D. Definitions of Key Terms ...........................................................................................I-2

II. Methodology .................................................................................................................. II-1

A. HUD Requirements and Study Standards.................................... .............................II-1

B. The Sample....................................................................... .........................................II-1

C. Data Collection.................................................................. ...................................... .II-2

D. Field Data Collection Time Periods......................................................................... .II-5

E. Constructing the Analysis Files.................................................... ........................... .II-6

F. Rent Formulae.......................................................................................................... .II-7

G. Calculation of Rent Error......................................................................................... .II-8

H. Quality Control Rent................................................................................................ .II-8

I. HUD Requirements Complicating the Analysis....................... .............................. II-10

III. Study Objectives and Analytic Methods................................................................... III-1

IV. Findings.........................................................................................................................IV-1

A. Overview.................................................................................................................. IV-1

B. Rent Error................................................................................................................. IV-2

C. Sources of Error ..................................................................................................... IV-12

D. Errors Detected Using Information Obtained From Project Files ......................... IV-15

E. Occupancy Standards............................................................................................. IV-23

F. Project Staff Questionnaire Analysis ..................................................................... IV-25

G. The Relationship between Characteristics and Practices, and Error...................... IV-25

H. Rent Reasonableness.............................................................................................. IV-25

I. Utility Allowance Analysis.................................................................................... IV-31

J. Payment Standard Analysis ................................................................................... IV-34

K. PIC/TRACS Analysis ............................................................................................ IV-39

V. Recommendations……………………………………………………………………..V-1 A. Modifying the Quality Control Process…………………………………………….V-1 B. Policy Actions………………………………………………………………………V-3 iii

Table of Contents Appendices Appendix A: Rent Calculations Appendix B: Weighting Procedures Appendix C: Source Tables Appendix D: Consistency and Calculation Errors Appendix E: Project Staff Questionnaire Descriptive Analysis Appendix F: Multivariate Analysis List of Exhibits Exhibit ES-1 Frequency of Rent Error by Program Type ............................................... ES-iv

Exhibit ES-2 Subsidy Dollar Error ....................................................................................ES-v

Exhibit ES-3 Estimates of Error in Program Administrator Income and Rent Determinations (in 1,000’s)..........................................................ES-v Exhibit ES-4 Comparative 2000 and FY 2003 through FY 2007 Gross Erroneous Payments................................................................................... ES-vi Exhibit ES-5 Rent Components Responsible for the Largest Dollar Error for Households with Rent Error............................................................... ES-viii Exhibit III-1 PHA-Administered Section 8 Unit Size Standards...................................... III-4

Exhibit IV-1 Percent of Households Fully Verified by Either the PHA/Owner or Macro.................................................................................. IV-3 Exhibit IV-2 Percent of Households with Proper Payments ............................................. IV-4

Exhibit IV-3 Percent of Households with Error, Average Dollars in Error, and Dollar Error Rate for All Households with Error.................................. IV-5 Exhibit IV-4a Underpayment Households Percent of Households and Average Monthly Dollar Amount of Error ........................................... IV-5 Exhibit IV-4b Overpayment Households Percent of Households and Average Monthly Dollar Amount of Error ........................................... IV-6 Exhibit IV-5 Gross and Net Dollar Rent Error (Monthly) for All Households............................................................................................. IV-7 Exhibit IV-6 Gross and Net Dollar Error Rates (Monthly) for All Households............................................................................................. IV-7 Exhibit IV-7 Certifications and Recertifications by Administration Type ....................... IV-8

Exhibit IV-8a Percent of Newly Certified Households Meeting Certification Criteria .................................................................................... IV-9 Exhibit IV-8b Percent of Newly Certified Households Meeting Certification Criteria by Program Type ..................................................... IV-10 Exhibit IV-9 Average Monthly Underpayment and Overpayment Dollar Amount Averaged Across All Households..................................... IV-10 iv

Table of Contents Exhibit IV-10a Negative Subsidy Households (Tenant Overpayment)

Percent of Households and Average Monthly Dollar

Amount of Error ......................................................................................... IV-11

Exhibit IV-10b Positive Subsidy Households (Tenant Underpayment)

Percent of Households and Average Monthly Dollar

Amount of Error ......................................................................................... IV-11

Exhibit IV-11 Average Monthly Dollar Amounts of Error for Negative

(Tenant Overpayment) and Positive (Tenant Underpayment)

Subsidies Averaged Across All Households.............................................. IV-12

Exhibit IV-12 Rent Components Responsible for the Largest Dollar

Error for Households with Rent Error ....................................................... IV-13

Exhibit IV-13 Total and Largest Component Dollars in Error

for Households with Rent Error................................................................. IV-14

Exhibit IV-14 Rent Component Error by Payment Type for All Households .................. IV-14

Exhibit IV-15 Elderly/Disabled Allowances and Dependent Allowances........................ IV-15

Exhibit IV-16 Findings With and Without Information Obtained

from Sources Other Than the Tenant File.................................................. IV-16

Exhibit IV-17 Percentage of Households with Calculation

and Consistency Errors .............................................................................. IV-17

Exhibit IV-18 50058/50059 Rent Calculation Error Compared

with QC Rent Error........................................................... .........................IV-18 Exhibit IV-19 Verification of 50058/50059 Rent Components by

PHA/Owners.......................... .................................................................... IV-19

Exhibit IV-20 Verification of 50058/50059 Rent Components

by PHA/Owner Staff by Program .............................................................. IV-20

Exhibit IV-21 QC Error Households with Missing Verification

in the Tenant File ............................................................ ...........................IV-21 Exhibit IV-22 50058/50059 Administrative Error: Percent of

Households, Average Dollars in Error....................................................... IV-22

Exhibit IV-23 Administrative Error: Percent of Households

Average Dollars in Error For All Households ........................................... IV-23

Exhibit IV-24 Percentage of Households in Units with the Correct Number

of Bedrooms According to Study Guidelines ............................................ IV-24

Exhibit IV-24a Percentage of All Households in FY 2006 by Number of

Bedrooms and Number of Household Members ....................................... IV-24

Exhibit IV-24b Percentage of All Households in FY 2007 by Number of

Bedrooms and Number of Household Members ....................................... IV-25

Exhibit IV-25a PHAs by Predominant Rent Reasonableness Method (unweighted) ......... IV-30

v

Table of Contents Exhibit IV-25b PHAs by Predominant Rent Reasonableness Method

Combinations (unweighted)....................................................................... IV-31

Exhibit IV-26a Rent Reasonableness Documentation for New Admissions ...................... IV-32

Exhibit IV-26b Type of Rent Reasonableness Documentation for

New Admissions ........................................................................................ IV-32

Exhibit IV-27 Timing of Most Recent Rent Reasonableness

Determination—New Admissions ............................................................. IV-33

Exhibit IV-28a Rent Reasonableness Documentation for

Annual Recertifications ............................................................................. IV-33

Exhibit IV-28b Type of Rent Reasonableness Documentation for Annual

Recertifications Where Documentation of the Rent Reasonableness

Determination Was Found ......................................................................... IV-34

Exhibit IV-29 Timing of Most Recent Rent Reasonableness

Determination—Annual Recertifications .................................................. IV-34

Exhibit IV-30a Type of Document Used by the PHA to Calculate

the Utility Allowance Value ...................................................................... IV-35

Exhibit IV-30b Utility Allowance Worksheet (WS) vs. 50058 (AC) Utility Allowance

Comparison Findings ................................................................................. IV-36

Exhibit IV-30c Availability of All Information to Enable QC Utility

Allowance Calculation............................................................................... IV-37 Exhibit IV-30d QC vs. AC Utility Allowance Comparison Findings................................. IV-37

Exhibit IV-31 Number and Percent of Households with Payment

Standard Discrepancies.............................................................................. IV-40

Exhibit IV-32 Percentage of Households Meeting Payment Standard Requirements

When Comparing Payment Standard to Fair Market Rent ........................ IV-40

Exhibit IV-33 Details of Cases Falling Outside the 90-110% of FMR Band ................... IV-41

Exhibit IV-34 Comparison of the FY 2006 to FY 2007 Payment Standard

Analysis...................................................................................................... IV-42 List of Figures Figure IV-1 Payment by Program Type........................................................................... IV-6

Figure IV-2 Case Type..................................................................................................... IV-8

vi

Executive Summary The Department of Housing and Urban Development (HUD) Quality Control for Rental Assistance Subsidy Determinations studies provide national estimates of the extent, severity, costs, and sources of rent errors in tenant subsidies for the PHA-administered Public Housing, Section 8 Housing Choice Voucher, and Moderate Rehabilitation programs; and the owner-administered Section 8, and Section 202 and Section 811 programs with Project Rental Assistance Contracts (PRAC) or Project Assistance Contracts (PAC). These programs account for nearly all of HUD’s current housing assistance outlays administered by the Offices of Housing and Public and Indian Housing, as well as the large majority of units assisted by HUD. This study was designed to measure the extent of administrator income and rent determination error by housing providers. This study does not involve an audit of individual PHAs or projects; nor does it monitor the implementation of housing programs. Its focus is on identifying households where an error was made when calculating the amount of the household’s rent; and providing nationally representative findings related to those errors. The errors we evaluated in this study affect the rent contributions tenants should have been charged. The findings presented in this report are a result of data collected from February through June 2008 for actions taken by Public Housing Authority (PHA) and project staff during FY 2007 (October 2006 through September 2007). These findings show that the percent of errors, the average dollars in error, and the gross dollar error rate in the Public Housing, Section 8 Housing Choice Voucher, Moderate Rehabilitation, owner-administered Section 8, and Section 202 and Section 811 programs with PRAC or PAC tenant subsidies continues to remain stable when compared with results from previous studies. HUD’s rental housing assistance programs are administered on HUD’s behalf by third-party program administrators, including PHAs, public and private project owners, and contracted management agents. In the programs examined, eligible tenants are generally required to pay 30 percent of their income toward shelter costs (rent plus utilities), with HUD providing the balance of the rental payment. New program applicants are required to provide certain information on household characteristics, income, assets, and expenses that is used to determine what rent they should pay. Existing tenants are required to recertify this information annually and also, in some circumstances, when there are significant changes in household income or composition. Applicant or tenant failure to correctly report income may result in HUD’s over- or underpayment of housing assistance. The failure of the responsible program administrator to correctly interview the tenant or process, calculate, and bill the tenant’s rental assistance may also result in HUD’s over- or underpayment of housing assistance. In 2000, HUD began to establish a baseline error measurement to cover the three major types of rental housing assistance payment errors: 1) program administrator income and rent determination error, 2) intentional tenant misreporting of income, and 3) errors in program administrator billings for assistance payments. Six studies have been conducted to identify program administrator income and rent determination error. In addition to the 2000 study, studies were conducted in 2003/2004, 2004/2005, 2005/2006 and 2006/2007 covering (re)certifications conducted in FY 2003, FY 2004, FY 2005 FY 2006 and FY 2007 respectively. The study referenced in this report covers FY 2007, and is being used to update the FY 2006 measurement of errors in program administrator income and rent determinations. The tenant data collected for this study were also used to provide the sample for the income match to measure the extent of intentionally unreported tenant income. The findings from this income match study will be published as a separate report. The balance of this report relates solely to program administrator income and rent determination error. ES-i

Executive Summary For purposes of this study, “error” is defined as any rent calculation or eligibility determination that differs from what would have occurred if the PHA or other program administrator had followed all HUD income certification and rent calculation requirements during the income certification or annual recertification conducted in FY 2007. When appropriate, study findings are compared with findings from the previous studies. Financial Impact of Identifying Rent Error. Reduction in the rent error associated with the programs included in this study does not mean there will be an overall savings in the costs associated with administering these programs. Given there are large numbers of eligible households on waiting lists, if a household leaves the program because it is no longer eligible for a subsidy, another household will take its place. The replacement household may be entitled to a smaller or a larger subsidy than the household that left the program. Therefore, the most direct benefit of identifying households with rent error is making sure those households who are eligible for the program are receiving the correct subsidy, rather than reducing the amount of funds needed to administer the programs. The most appropriate use of this study is as a tool for strengthening HUD’s procedures for ensuring administrative compliance with regulations. The recommendations presented in this report may require greater rather than fewer resources to provide HUD, PHAs, and owners with the written policy guidelines, training, standardized forms, and on-going monitoring needed to assure the programs are administered correctly. HUD’s objective of providing the right subsidies to the right families is a worthy one that this study can assist in achieving. A. Methodology HUD Requirements and Study Standards. Using the Code of Federal Regulations and official HUD handbooks and notices, all HUD requirements relevant to the determination of rent were consolidated into a set of HUD requirements. Nationally recognized experts were involved in establishing and reviewing the standards used in this study. The Sample. A nationally representative sample of 600 projects in the United States and Puerto Rico was selected for this study. These projects were selected from the universe of the three program types covered by the study— ♦ Public Housing ♦ PHA-administered Section 8 (Vouchers and Moderate Rehabilitation) ♦ Owner-administered Section 8, Section 202 PRAC, Section 811 PRAC, Section 202/162 PAC A random sample of four households was selected for most projects, but more tenants were selected from unusually large projects. The final study data set includes responses from 2,404 households. The Data Collection Process. The data collection effort included creating and automating more than 30 data collection instruments, contacting and obtaining information from PHA/owner staff, hiring and training more than 60 field interviewers, and selecting the project and tenant sample. Field interviewers obtained data from tenant files, and interviewed tenants using computer ES-ii

Executive Summary assisted personal interviewing software developed for this study. The automated data collection process included built-in consistency and edit checks that prompted interviewers to probe inconsistent and anomalous responses. Collected data were electronically transferred daily to Macro headquarters for review. Requested third-party verifications related to income, assets and expenses were also processed at Macro headquarters. Calculation of Rent Error. A quality control (QC) rent was calculated for each household in the sample using the information reported by the PHA/project and household. Rent error was calculated by subtracting the QC rent from the actual tenant rent (the rent from HUD Forms 50058 or 50059 that had been calculated by the project staff). A discrepancy of $5 or less between the actual and QC rent was not counted as an error. This $5 differential was used to eliminate rounding differences and minor calculation discrepancies that have little effect on program-wide subsidy errors. B. Major Rent Error Findings National Rent Error Estimates. The analysis of the FY 2007 tenant files, tenant interview, and income verification data indicates that— ♦ 64 percent of all households paid the correct amount of rent within $5 (51 percent paid exactly the right amount) ♦ 18 percent of all households paid in excess of $5 less than they should have (with an average error of $58 per month) ♦ 18 percent of all households paid in excess of $5 more than they should (with an average error of $30 per month) Rent Error Estimates by Program Type. Both the owner-administered and PHA– administered Section 8 programs had an underpayment of rent rate of 19 percent compared to 16 percent in the Public Housing program. The rate of overpayment was highest in the PHA – administered Section 8 programs with 20 percent overpayments of rent compared to 17 percent of the owner-administered program, and 14 percent in the Public Housing program. The exhibit that follows summarizes this information. Exhibit ES-1 Frequency of Rent Error by Program Type Rent Underpayment (Subsidy Rent Overpayment (Subsidy Program Overpayment) Underpayment) Public Housing 16% 14% PHA-Administered Section 8 19% 20% Owner-Administered 19% 17% Total 18% 18% Dollar Error Effect of Rent Errors. All summary error estimates represent the summation of net case-level errors. That is, a case is determined to have a net overpayment error, no error, or a net underpayment error. Major findings were— ES-iii

Executive Summary ♦ ♦ Multiplying the $63 by ♦ gross The net The study found that the net 1 . This data responds to Exhibit ES-2

l i $57 $30

l lds $11 $5

1 Rent Underpayments of Approximately $524 Million Annually (down from $648 Million in FY 2006). For tenants who paid less monthly rent than they should pay (18.46 percent), the average monthly underpayment was $57. For purposes of generalization, total underpayment errors were spread across all households (including those with no error and overpayment error) to produce a program-wide average monthly underpayment error of $10.56 ($127 annually). Multiplying the $127 by the approximately 4.1 million units represented by the study sample results in an overall annual underpayment dollar error of approximately $524 million per year. Rent Overpayments of Approximately $260 Million Annually (down from $306 Million in FY 2006). For tenants who paid more monthly rent than they should pay (17.52 percent), the average monthly overpayment was $30. When this error was spread across all households, it produced an average monthly overpayment of $5.24 ($63 annually). the approximately 4.1 million assisted housing units represented by the study sample results in an overall annual overpayment dollar error of approximately $260 million per year. Aggregate Net Rent Error of $264 Million Annually. When combined, the average rent error per case is $16 ($11 + $5). Over- and underpayment errors partly offset each other. overall average monthly rent error is $6 ($11-$5). HUD subsidies for Public Housing and Section 8 programs equal the allowed expense level or payment standard minus the tenant rent, which means that rent errors have a dollar-for-dollar correspondence with subsidy payment errors, except in the Public Housing program in years in which it is not fully funded (in which case errors have slightly less than a dollar-for-dollar effect). subsidy cost of the under- and overpayments was approximately $264 million per year ($524 million - $260 million) Subsidy over- and underpayment dollars are summarized in Exhibit ES-2. study Objective 1 (identify the various types of errors and error rates and related estimated variances). Subsidy Dollar Error Subsidy Subsidy Type Dollar Error Overpayment Underpayment Average Month y Per Tenant Error for Households w th Errors (18.46% of cases) (17.52% of cases) Average Month y Per Tenant Error Across All Househo Total Annual Program Errors $524 million $260 million Total Annual Errors—95% Confidence Interval $370 - 677 million $215 – 305 million Exhibit ES-3 provides estimates of program administrator error by program type. This data responds to study Objectives 3 (estimate national-level net costs for total errors and major error The actual estimate of annual rent underpayments is $523.70 million. The actual estimate of annual rent overpayments is $259.79 million. Therefore the actual estimate of net rent error is $263.90 million ($523.70 - 259.79 = $263.91). ES-iv

Executive Summary types), 8 (provide information on the extent to which errors are concentrated in projects and programs), and 11 (estimate total positive and negative errors in terms of HUD subsidies). Exhibit ES-3 ) Net Estimates of Error in Program Administrator Income and Rent Determinations (in $1,000’s Administration Type Subsidy Overpayments Subsidy Underpayments Erroneous Payments Gross Erroneous Payments Public Housing $106,392 $42,972 $63,420 $149,364 PHA-Administered Section 8 $282,840 $152,172 $130,668 $435,012 Total PHA-Administered $389,232 $195,144 $194,088 $584,376 Owner-Administered $134,460 $64,644 $69,816 $199,104 Total $523,692 $259,788 $263,904 $783,480 95% Confidence Interval +/-$153,438 +/-$44,799 +/-$162,357 +/-$157,292 In response to study Objective 5 (determine whether error rates and error costs have statistically significant differences from program to program), multiple regression analyses with design effect adjustment were conducted to compare the three program types included in the study on mean gross dollar error, mean gross dollar error rate, mean net dollar error, and mean gross dollar error rate. No significant differences in net dollar error or net dollar error rate were found, and no significant differences between owner-administered and Public Housing households were found. However, PHA-administered Section 8 households have a greater gross dollar error than households in either of the other two programs, and this result is statistically significant at the .05 level (two-tailed) for both gross error rate and gross error cost. Comparison with Prior Studies. Five prior studies, the 2000 baseline, the FY 2003 study, the FY 2004 study, the FY 2005 and the FY 2006 study estimated erroneous payments attributed to program administrator rent calculation and processing errors, using the same methodology, sampling procedures, and sample sizes as this FY 2007 study. The 2000 “Quality Control for Rental Assistance Subsidy Determinations” study was published as a final report in June 2001. The FY 2003 final report—“Quality Control for Rental Assistance Subsidies Determinations”— was completed in August 2004. The FY 2004 final report was completed in July 2005. The FY 2005 final report was completed in October 2006 and the FY 2006 final report was completed in October 2007. While the FY 2003 and FY 2004 studies demonstrated significant reductions in erroneous payments attributed to program administrator income and rent determinations, the FY 2005 findings indicated a smaller reduction in the gross dollars in erroneous payments that did not represent a statistically significant decrease from FY 2004. The FY 2006 study indicated a small increase in the gross dollars in erroneous payments which also did not represent a statistically significant difference. The FY 2007 study, however, indicates the lowest level of gross dollars in erroneous payments in study history with significant reductions in PHA administered programs. Exhibit ES-4 presents a comparison of the gross erroneous payments for 2000, FY 2003, FY 2004, FY 2005, FY 2006 and FY 2007. ES-v

Executive Summary Exhibit ES-4 Comparative 2000 through FY 2007 Gross Erroneous Payments* Percent Reduction in 2007 Gross 2006 Gross 2005 Gross 2004 Gross 2003 Gross 2000 Gross Gross Erroneous Erroneous Erroneous Erroneous Erroneous Erroneous Erroneous Payments Payments Payments Payments Payments Payments Payments from Administration Type (in $1000’s) (in $1,000’s) (in $1,000’s) (in $1,000’s) (in $1,000’s) (in $1,000’s) 2000 to 2007 Public Housing $149,364 $172,824 $220,464 $242,076 $316,116 $602,556 75.21% PHA-Administered Section 8 $435,012 $520,020 $456,240 $521,220 $730,956 $1,096,524 60.33% Total PHA-Administered $584,376 $692,844 $676,704 $763,292 $1,047,072 $1,699,092 65.61% Owner-Administered $199,104 $261,324 $248,580 $224,460 $368,796 $539,160 63.07% Total $783,480 $954,168 $925,232^ $987,744^ $1,415,844^ $2,238,252^ 65.00% +/-$157,292 +/-$192,000 +/- $164,000 (+/-$131,000) (+/-$163,000) (+/-$275,000) * Gross Rent Error is the sum of the absolute value of positive and negative rent error.

^ Numbers do not add exactly due to rounding.

C. Sources of Errors Rent errors are often a result of a mix of different types of errors. This study also examined administrative and component errors. For purposes of this study, administrative errors are analyzed separately from specific component errors. Administrative errors are errors that result from administrative mistakes. They consist of— ♦ Consistency errors—errors in logical conformity between elements within the 50058 or 50059 Forms ♦ Calculation errors—arithmetic errors within subsections of the 50058 or 50059 Forms ♦ Transcription errors—errors in transferring information from documentation in the tenant file to the 50058 or 50059 Forms ♦ Failure to conduct a recertification in a timely manner ♦ Failure to verify information Component errors are related to the income and expense components used to calculate rent. The income components are employment income, Social Security benefits and pensions, public assistance, other income, and asset income. The expense/allowance components are elderly/disabled allowance, dependent allowance, medical allowance, child care allowance, and disability allowance. Component errors often occur when project staff do not conduct a thorough tenant interview or do not verify the information obtained during the interview. However, component error may also occur when the tenant supplies incorrect information, either intentionally or unintentionally. The discussion below responds to study Objectives 2 (identify the dollar costs of the various types of errors), and 6 (determine the apparent cause of significant rent errors). ES-vi

Executive Summary Administrative Errors. The two most common administrative errors are calculation errors and transcription errors. The HUD PIC and TRACS data systems check the rent calculations on Forms 50058 and 50059. For tenants for whom data are submitted (and corrected if required), these systems virtually eliminate rent determination calculation errors for the items included on the forms. However, not all cases are reported and some cases returned to program sponsors for correction are ignored or are changed in HUD systems but not actually implemented. PIC/TRACS data system matches were made (to respond to Objective 14) for the 2404 households in the study. Ninety-five percent of these households (97 percent of owner- administered households, and 94 percent of PHA-administered households) were found in the PIC/TRACS databases. Interestingly enough, there was very little difference in the percent of households with rent error for households for which PIC/TRACS data were or were not available. However, the average gross dollars in error were higher for households where PIC/TRACS data were absent. Verification Errors. The percentage of income and expense items verified by PHA/owner staff in FY 2007 showed little change compared to FY 2006. Income items were verified at least 75 percent of the time (compared to 74 percent in FY 2006). While in FY 2005, the percentage of items verified remained similar to the percentage verified in FY 2004, there was a downward trend in FY 2006. In FY 2007 the percentage of items verified increased for most rent components. Failure to use verified income and expense amounts continues to be a problem. The percent of items where the verified amounts matched the amount reported on the 50058 and 50059 Forms increased slightly for three rent components, pensions, public assistance and medical expenses. The degree to which programs correctly incorporate verified data for earned income is unchanged in FY 2007 (68 percent in both FY 2007 and FY 2006). Obtaining income verification is often difficult. Even when repeated requests are made, employers sometimes do not respond to requests for verification. Some program sponsors do a much better job than others in achieving third-party compliance with written verification. The QC study shows that it is reasonable to expect all program sponsors to have as high a success rate as the current high performers. The study also shows that there is significant room for improvement in using the verification data obtained, which are often collected consistent with procedures but then filed and never used. Overdue Recertifications. HUD requires that every household be recertified annually. Recertifications for 2 percent of the households were overdue, compared to 3 percent in FY 2006. Component Errors. Incorrect income and allowance amounts were by far the most significant sources of error in determining rents. Less than one percent of households with rent errors did not have an income or expense component error. Earned income (24 percent), pension income (21 percent), and medical allowances (23 percent) continued to have the greatest percent of households in error. The following exhibit shows the frequency of the most serious component errors and the average dollar amount for each type. The Percentage of Households represents the households with any rent component error where the specified rent component was responsible for the largest error. The Average Dollar Amount represents the average dollar amount for the specified rent component for households where the specified component was responsible for the largest error. For comparison purposes, findings from FY 2006 are provided in parentheses. ES-vii

Executive Summary Note that both the percentage of households with component errors and the average dollar amount of error have tended to decrease. Exhibit ES-5 Rent Components Responsible for the Largest Dollar Error for Households with Rent Error (FY 2006 Findings Are Provided in Parentheses) Percentage of Average Dollar Rent Component Households Amount Earned Income 24% (26%) $2892 ($4,544) Pensions 21% (25%) $2065 ($2,246) Other Income 11% (10%) $2437 ($2,488) Public Assistance 6% (9%) $2492 ($1,823) Asset Income 4% ( 2%) $1502 ($1,733) Medical Allowance 23% (17%) $972 ($1,099 ) Child Care Allowance 4% (4%) $2259 ($2,128) Dependent Allowance 3% (3%) $622 ($703) Elderly/Disabled Allowance 3% (2%) $400 ($400) No Rent Component Error 1% (1%) $0 Total 100% $1957 ($2,513)* * The sum of the dollars associated with the largest component in error divided by the number of households with error. D. Additional Findings Eligibility of Newly Certified Households. A separate analysis of newly certified households (16 percent) was conducted to determine if these households were eligible for HUD housing assistance. Ninety one percent of these households met all the eligibility criteria (the same as in FY 2006). There were no newly certified households in the sample who were not income- eligible on the basis of the QC income determination. Two percent of the newly certified households failed to document Social Security numbers (or certify non-assignment of a number) for one or more family members (at least 6 years of age), and 14 percent lacked the signed consent forms needed to authorize verification of income and assets (for each member of the household at least 18 years of age). Only 2 percent lacked the signed declaration forms or evidence accepted as proof of citizenship. These findings respond to study Objective 9 (estimate the percentage of newly certified tenants who were incorrectly determined eligible for program admission. Occupancy Standards. Study Objective 7 asks for the extent to which households are overhoused relative to HUD’s occupancy standards. Fifteen percent of all households occupied a unit with too many or too few bedrooms in FY 2007, according to the guidelines used for this study. This number is up slightly from FY 2006, where fourteen percent of all households occupied a unit with an incorrect number of bedrooms ES-viii

Executive Summary Rent Reasonableness. Study Objective 10 asks for the extent to which Section 8 Voucher rent comparability (reasonableness) determinations are found in the tenant file, and the method used to support the determinations. Seventy-four percent of new admission files contained rent reasonableness documents, as did 66 percent of the files for households for which data were collected for an annual recertification. However, the absence of documentation does not necessarily indicate a determination was not completed; only that it was not properly documented. Information was also collected at the PHA level to understand the method used to determine rent reasonableness. About 91 percent of the PHAs in the study used unit-to-unit rent comparison, unit-to-market rent comparison, or a point system when determining if the rent was reasonable. For the remaining 9 percent there was either no information available, the PHA used some other method of determining rent reasonableness, or the units were subject to rent control. Utility Allowances. For PHA-administered Section 8 Voucher households, the utility allowances found on the 50058 forms were compared to the utility allowance worksheets found in the tenant file, and to the utility allowance values calculated using the utility allowance schedules provided by the PHAs. For the first comparison, 90 percent of the utility allowance values matched. For the second comparison, 88 percent of the values matched. However, the fact that the values did not match, does not necessarily mean the utility allowance found on the 50058 form was incorrect. For at least 5 percent of the households, the information needed to determine the accuracy of the utility allowance value was not available. Payment Standards. A special analysis was conducted to determine if the correct payment standards were used for Section 8 Voucher households. The payment standard found on the 50058 form was compared to the payment standard schedules provided by the PHA, and to the Fair Market Rent (FMR) for the appropriate geographical area. For the first comparison, 92 percent of the payment standards matched. For the second comparison, 96 percent of the payment standards found on the 50058 form fell within the 90 to 110 percent FMR band. As with the utility allowance analysis, the information needed to conduct the analysis was not always available. Therefore, the fact that the payment standards did not match does not necessarily mean the incorrect payment standard was used when calculating the amount of the tenant rent. 50058/50059 Rent Calculation Error. The tenant rent was calculated using only data on the 50058/50059 to determine the relationship between errors detected using the 50058/50059 forms and total rent errors found in the study (in response to study Objective 4). When using only the 50058/50059 data to calculate rent, errors were found in 7 percent of the households. This is clearly different then the QC error calculation where errors were found in 36 percent of the households. In addition, error was found in both the 50058/50059 and QC calculation in only 3 percent of the households. Automated Rent Calculation Systems. Study Objective 12 asks whether error rates in projects that use an automated rent calculation system differ from errors in those that do not. We did not find a difference between PHA/projects that use automated rent calculation systems and those that do not. This is not surprising because nearly 97 percent of all projects use an automated rent calculation system. Tenant Characteristics, and Project Characteristics and Practices. In response to study Objective 8 (provide information on the extent to which errors are concentrated in projects and programs), data were collected from PHA/project staff via a structured mail survey. Multivariate ES-ix

Executive Summary analyses were conducted to explore whether project characteristics or practices contributed to administrative or rent errors. It was found that projects that retained experienced certification staff (over 5 years) had lower average rent error. This finding is compatible with what was found in the FY 2006 analysis where staff educational requirements were related to lower rent error, namely, staff quality is a critical factor in reducing errors. In response to study Objective 13 (determine whether other tenant or project characteristics on which data are available are correlated with high or low error rates), additional multivariate analyses were conducted. The analysis found that projects whose survey respondents reported perceived error causes tended to have a lower chance of certain types of staff error. It is arguable that staff’s ability to report difficulties causing mistakes is equivalent to knowledge of the problems and hence a potential way to deal with them. However, far more information is needed to learn why and how project staff perceive error. E. 2000–2007 Progress In response to the findings and recommendations of the 2000 Assisted Housing Quality Control Study, HUD initiated a series of aggressive actions to address the causes of erroneous assistance payments, including extensive onsite monitoring. Actions taken by HUD included the following— ♦ A Rental Housing Integrity Improvement Program (RHIIP) committee headed by the Office of the Chief Financial Officer with representatives from the other affected Offices was formed to coordinate and monitor corrective actions. The committee meets to review progress, and identify and resolve impediments to progress in reducing errors. ♦ The Offices of Housing and Public and Indian Housing developed and issued new handbooks and instructional material that detailed all current HUD program requirements and standardized them to the extent possible without regulatory or statutory change. These handbooks cover nearly all aspects of occupancy policy, from the point of tenant application for admission and rent calculations through ongoing occupancy to lease termination. For Public Housing, the issuance of a Public Housing Occupancy Guidebook represented the first such effort in more than 20 years, and provided a defined methodology for calculating a number of complex requirements (e.g., the Earned Income Disallowance). ♦ The Offices of Housing and Public and Indian Housing substantially increased training efforts, and have held a number of national and regional training sessions. This contrasts with a less activist role in the 1980’s and 1990’s. ♦ The Offices of Housing and Public and Indian Housing initiated comprehensive, large-scale, and onsite occupancy and management reviews, which also represented a major procedural change from the previous two decades for most HUD offices— • The Office of Housing primarily used new agreements with Contract Administrators, which are usually State agencies, to perform this function. Contract Administrators provide technical support in adhering to HUD program requirements and routinely perform detailed monitoring on agency compliance. ES-x

Executive Summary • The Office of Public and Indian Housing initiated a system of Rental Integrity Monitoring (RIM) reviews to detect and reduce errors in income and rent calculations at targeted PHAs, reduce rent under- and/or overpayments by residents, and ensure that HUD’s limited housing resources were being used to serve eligible families in a fair and equitable manner as intended by Congress. ♦ HUD initiated a legislative change that gives it access to the Department of Health and Human Service’s New Hires income and wage database for income matching purposes. It will use these data to compare tenant-reported income with state wage data to better ensure that the right subsidy payments are made to the right households in accordance with program statutory and regulatory requirements. This legislation was passed in late 2003 and required implementation of agreements and data systems. HUD also negotiated agreements with some states to obtain access to the same information. Some local agencies have already initiated income-matching systems, and it seems that this has made some contribution to error reductions. HUD’s performance goals, which were developed in consultation with the Office of Management and Budget, called for reducing the 2000 benchmark assisted housing error levels by 50 percent by the end of 2005. The study of program administrator error for FY 2007 shows that HUD exceeded this goal. It should be noted, however, that the reduction of errors and improper payments is unlikely to have an equivalent effect on budget outlays. HUD’s experience indicates that its program integrity improvement efforts are likely to result in some higher income tenants leaving assisted housing and being replaced with lower income tenants requiring increased outlays. Nevertheless, HUD’s goal remains to ensure that the right benefits go to the right people. F. Recommendations The progress when comparing the 2000 findings to the FY 2007 results is impressive. However, the percent of errors has remained stable since the FY 2004 study and the average dollars in error and the gross dollar error rate have only decreased slightly. On the basis of the current study’s results, the following approaches to further reduce program administrator income and rent determination error rates are recommended: ♦ HUD should continue its plans to use the Department of Health and Human Service’s New Hires income matching database. However, access to the New Hires income matching database by itself will not result in a reduction in error. PHA/project staff must use this information to assist them in resolving discrepancies between the database and the tenant’s declaration. ♦ HUD should continue to provide PHAs and owners with the forms, training, and other tools required to determine rent correctly. Changes in policy should be reported to PHAs and owners in a timely fashion with the guidance needed to implement those changes in an accurate manner. ♦ HUD should continue to implement its onsite monitoring program, and PHAs and owners should be held accountable for implementing HUD regulations and calculating rent accurately. ES-xi

Executive Summary ♦ Federal laws, regulations, and HUD requirements should be simplified to the extent possible. ♦ HUD should consider implementing policy that allows reexaminations, for selected populations, to be completed less often than annually. In addition, the quality control studies could be modified to supplement the findings from this study and identify options for reducing error in the future. The following are possible methods to achieve this goal: ♦ Collect more information regarding PHA/project policies and practices. Each PHA establishes its own policies, procedures, and forms for collecting the information that is ultimately used to calculate tenant rent. The differentiation in these practices may have some (possibly major) impact on the rent error, yet the analysis of the project practices and characteristics collected in the Project Staff Questionnaire designed for this study do not demonstrate the expected impact. Focus groups and cognitive interviewing could be used to identify additional PHA/project level factors that may impact error. This additional information could be used to revise the Project Staff Questionnaire to include questions focused on the specific practices expected to influence errors. ♦ Gather information to document the outcome of the HUD quality control studies. Overall, the HUDQC studies indicate that both the percent of errors and dollars associated with those errors have decreased in the last seven years. However, there is no information on changes in tenant behavior related to the identification and reduction of error. To really understand the overall impact of the quality control studies on subsidy funding, additional information is needed regarding both the tenants receiving the subsidies and the PHA/projects administering the housing benefits. ♦ Expand contractor access to verification obtained through Social Security Administration and National Directory of New Hires data. Despite increasing rates of third-party verification, a large proportion of tenant income and expenses are not being verified. This is especially important given the study results indicate a significant relationship between third-party verification of certain types of income and rent errors. Expanded access to Federal databases would allow the contractor to investigate discrepancies in the information obtained from multiple sources. ♦ Continue to investigate PIC/TRACS data for sampling and other purposes. Ideally PIC/TRACS data would be used to select the quality control sample, and provide the actual data used by the PHA/project staff when calculating rent (in place of abstracting 50058/50059 Form data from the tenant file). However, to do this the data must be available for the specific period of time covered by the study. ♦ Continue to expand existing computer systems and processes that further automate data collection, processing, and reporting functions. Expanding and investing in better automated systems will yield large dividends in terms of costs, time required to collect and process data, as well as the breadth, depth, and quality of data. ES-xii

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Executive Summary ♦ Continue the HUD quality control studies as a regular, ongoing effort to monitor and manage HUD rent determination processes. Ongoing evaluation of the subsidy programs administered by HUD is essential to the management of those programs. Although the primary goal of these studies is to measure rent errors, the studies also give HUD the opportunity to learn more about alternatives to reducing rent errors, and better management of current and changing conditions at PHAs/projects. ES-xiii

I. Introduction A. Purpose of the Quality Control for Rental Assistance Subsidies Determinations Study for FY 2006 The purpose of this study is to provide national estimates of rent subsidy errors for the U.S. Department of Housing and Urban Development’s (HUD’s) Public and Indian Housing (PIH)-administered Public Housing (Public Housing), PIH-administered Section 8 Housing Choice Voucher and Moderate Rehabilitation programs (PHA-administered Section 8); and Housing-administered Section 8, Section 202 PRAC, Section 811 PRAC, and Section 202/162 PAC programs (owner-administered). Rent subsidy errors occur during the tenant certification and annual recertification processes, and this study examines the extent, costs, and sources of these subsidy errors. 1 For the purpose of this study, “error” is defined as any rent calculation or eligibility determination that differs from what would have occurred if the PHA/owner had followed all of HUD’s income certification and rent calculation requirements. This study focuses on (re)certifications conducted during Federal FY 2007. HUD identified 14 study objectives related to types of errors and cost issues; this report addresses each of these objectives. The analysis also identifies errors in assigning appropriate size units to households and certain procedural errors in the eligibility and rent determination process. In addition, a special analysis was conducted of Utility Allowances, Payment Standards and Rent Reasonableness practices used by the PHAs administering the Section 8 voucher program. B. Background of the Study This study is the seventh in a series of studies designed to identify current HUD eligibility, income, and rent determination regulations, translate these regulations into survey instruments, develop an error detection system, and provide nationally representative estimates of rent subsidy errors. In the past three studies, an additional income match of Social Security benefit data was conducted. The results of previous studies were published as follows: ♦ The final report for the first study, conducted by Macro International Inc., (Macro), and KRA Corporation (KRA) was published in April 1996 (data were collected in 1992). ♦ The final report for the second study, conducted by Macro 2 , was published in June 2001 (data were collected in 2000). ♦ The final report for the third study, also conducted by Macro and which covered the first half of FY 2003, was published in April 2004. Following the collection of data for the second half of FY 2003 a follow-up report was written and published in August 2004. 1 PHAs and owners of HUD-assisted housing are required to make an initial determination of eligibility (a “certification”) and thereafter an annual recertification of each household’s rent (a “recertification”). In this report, the term (re)certification refers to certifications and annual recertifications. Interim recertifications were not included in this study. 2 From May, 1999 through December, 2006 Macro International was a wholly owned subsidiary of Opinion Research Corporation (ORC) and conducted business under the name ORC Macro. I-1

I. Introduction ♦ The final report for the fourth study, conducted by Macro was published in July, 2005 (data were collected in 2004). ♦ The final report for the fifth study, conducted by Macro was published in October, 2006 (data were collected in 2006). ♦ The final report of the sixth study, conducted by Macro was published in October, 2007 (data were collected in 2007). Work on the current project began in May 2007. Tasks completed before data collection included designing the research and survey methodology, compiling HUD’s regulations for the programs included in the study (Public Housing, PHA-administered Section 8, and owner- administered), and automating the data collection process. Data were collected from a nationally representative sample of HUD-assisted housing projects and project residents whose (re)certifications were conducted from November 2006 through October 2007. C. Organization of This Report This report is organized as follows: ♦ Section I: Introduction ♦ Section II: Methodology ♦ Section III: Study Objectives and Analytic Methods ♦ Section IV: Findings ♦ Section V: Recommendations ♦ Appendices A. Rent Calculations B. Weighting Procedures C. Source Tables D. Consistency and Calculation Errors E. Project Staff Questionnaire Analysis

F. Multivariate Analysis

D. Definitions of Key Terms Definitions of key terms used throughout this report are listed below: Actual Rent—the tenant rent from the 50058 or 50059 Form. I-2

I. Introduction Administration Type—PHA or owner. Abstract Month—the month in which the data collection process for any given household was

initiated.

Calculation errors—arithmetic errors within subsections of the 50058 or 50059 Form.

Case Type—certification, recertification, and overdue recertification.

Component errors—the income and expense components used to calculate rent. The income

components are employment income, Social Security and pensions, public assistance, other

income, and asset income. The expense/allowance components are elderly/disabled allowance,

dependent allowance, medical expenses, child care expenses, and disability expenses.

Consistency errors—errors in logical conformity between elements within the 50058 or 50059

Form.

Dollar Rent Error—is calculated at the household level by subtracting the household’s QC

Rent from the Actual Rent.

Error Rate—the sum of the dollar amount of Gross Rent Error divided by the sum of the dollar

amount of the QC Rent.

Gross Rent Error—the sum of the absolute values of under- and overpayments.

Largest Component Dollar Error—the annual dollar amount of error in the component with

the largest error.

Net Rent Error—the arithmetic sum of over- and underpayments.

(Rent) Overpayment—results when the household paid more than it should have paid; HUD’s

contribution was less than it should have been.

Payment Type—underpayment, proper payment, and overpayment.

Program Type—Public Housing, Section 8 Housing Choice Voucher, Section 8 Moderate

Rehabilitation, Section 8 project-based, Section 202 PRAC, Section 811 PRAC, and Section

202/162 PAC.

Quality Control Month—the month in which the PHA/owner completed the rent calculation.

Quality Control (QC) Rent—calculated by Macro using the tenant file, household interview

and verification data.

Rent Component—the five sources of income (earned, pensions, public assistance, other

income, and assets) and the five types of deductions (medical, child care, and disability

assistance expenses, dependent allowance, and elderly/disabled allowance).

Rent Error—the difference between the monthly Actual Rent and the monthly QC Rent.

I-3

I. Introduction Total Component Dollars in Error—the absolute sum (i.e., the sum of the positive and negative amounts, ignoring the plus or minus signs) of all individual income and expense component errors. These errors are combined to provide an overall Total Dollars in Error and are presented as an annual amount. Transcription errors—errors in transferring information from documentation in the tenant file to the 50058 or 50059 Form. (Rent) Underpayment—results when the household paid less than it should have paid; HUD’s contribution was higher than it should have been. I-4

II. Methodology A. HUD Requirements and Study Standards Using the Code of Federal Regulations and official HUD handbooks and notices, all HUD rules relevant to the determination of rent were consolidated into a set of HUD requirements. These requirements were used to create a uniform set of rules that could identify errors in eligibility determination, rent calculation, and unit assignment for the housing programs in the study. In general this uniform set of rules, known as the standards, follows the official HUD requirements. However, for some complex requirements, standardized procedures had to be developed so the data could be collected in a uniform manner. A complete list of standards used in this study can be found in the Data Collection Standards for the FY2007 HUDQC Study, Quality Control for Rental Assistance Subsidies Studies: 2007. 1 B. The Sample The initial sampling design called for a nationally representative sample of 600 projects with four households randomly selected from each project, or 2,400 households. Projects were selected with probabilities proportional to size (PPS), but projects whose size exceeded the sampling interval were selected for eight, twelve, or more households in the project, and were counted as more than one project for purposes of determining the sample size. The sampling design required approximately equal allocations for the three assisted program types: Public Housing, PHA-administered Section 8 (Vouchers and Moderate Rehabilitation), and owner- administered (Section 8, Section 202 PRAC/PAC, and Section 811 PRAC/PAC). PHAs that participated in the Move to Work block grant demonstration program through Public Housing or Section 8 Vouchers were removed from the project-level sample. Because some large projects were selected multiple times, the study sample included 549 distinct projects in 59 geographic areas across the United States and Puerto Rico. We sampled 200 projects from each major program type. In addition, data were collected for four households in one additional owner- administered project. This additional project was added to the sample to ensure, that given any unexpected circumstances, the sample would included a minimum of 2400 households. Since no unexpected issues were encountered, the final data set includes responses from 2,404 households in the 549 projects. The tenant sample was selected from all households that were receiving assistance in Federal FY 2007. A random sample of four households was selected from most projects. An equal number of potential “replacement” households were identified as potential substitutes when selected households did not meet the study requirements or were unavailable to be interviewed. However, as noted above, some large projects had additional households. For example, the Columbus Metropolitan Housing Authority Section 8 Voucher program had a household sample size of 16, and eleven other Housing Authorities’ Section 8 Voucher programs had household sample sizes of 12, including those of New York City and Los Angeles. For additional information on the sampling procedures, see the Sampling Report: FY2007, Quality Control for Rental Assistance Subsidies. 2 1 Macro unpublished report to HUD dated July 27, 2007. 2 Macro unpublished report to HUD dated July 2, 2007. II-1

II. Methodology C. Data Collection This study used a multi-stage data collection process to obtain all required information. Mail surveys provided project-level information from PHA/project staff. Tenant-level information was obtained by field interviewers who abstracted data from the household file, interviewed the tenant, and requested verification for income, expense, and household composition items from third parties. 3 Tenant income, expense, allowance, and third-party verification information were collected using HUD-sanctioned data collection procedures. Macro field interviewers strictly adhered to these procedures to avoid misclassifying errors caused by PHAs/projects that did not follow HUD requirements. The initial collection of project level data began in November 2007. Field data collection began in February 2008 and ended in June 2008. Because PHA/projects have varying practices, data collection forms and guidelines for data collection were designed to be flexible enough to obtain data from circumstances as found in the PHA/project. The major tasks accomplished during data collection and the forms used to accomplish them are discussed below. Creating the Data Collection Instruments. More than 35 data collection forms were used for this study to collect data on both the project and tenant levels. These forms were similar to those used for the previous data collection efforts, though modifications were made to many forms to improve the data collection process. Project-level forms were used to gather information to facilitate data collection, collect data elements necessary to calculate Quality Control (QC) rent, and gather information about certification and recertification practices. The tenant-level data collection forms were created to collect data and determine whether: 1) there were errors in the eligibility determination, 2) the household rent was calculated correctly, and 3) units were correctly assigned according to the study standards. Each form was created by a survey research specialist and reviewed by a HUD policy expert. The Office of Management and Budget (OMB) approved all data collection forms. Automating the Data Collection Process. This study used an enhanced version of the data collection system used in previous studies. While project-level data were collected on paper and the data entered upon receipt at Macro, data from tenant files were entered directly into laptop computers, and a computer-assisted personal interviewing (CAPI) system was used to interview tenants. This system, referred to as the HUDQC Data Collection Software (HDCS) system, was developed by a special team of Macro survey specialists and computer systems experts. 4 As sections of the instruments were collected by field interviewers, the HDCS system compared the data with a range of acceptable responses and data previously entered, allowing data entry errors to be corrected in the field. The system required that the data be collected in the correct order, and that all the appropriate skip patterns be followed. The automated system also alerted the field interviewer if key pieces of information used to calculate rent were missing and needed to be located and documented. This structured, automated process greatly reduced the need to edit, code, and clean the data after data collection 3 Verification is a process of obtaining information about income or expenses from a third party who can attest to the accuracy of the information provided by the household. HUD requires that most information provided by the household be verified by a third party or substantiated from documents (e.g., print-outs from EIV system). 4 The base of HDCS is the CSPRO software system used to collect demographic and health information in many countries, in conjunction with the U.S. Agency for International Development (USAID). II-2

II. Methodology was completed. HDCS data were transferred to Macro electronically on a daily basis. The incoming data were reviewed in an ongoing quality control process. This continual review of data during data collection ensured the accuracy of the data and permitted headquarters staff to resolve issues or request further clarifying documents while the field interviewers were still in the field. Contacting the PHA/Project. PHA/project contact names were obtained from HUD headquarters staff. Letters were sent to PHA/project staff advising them of the study and requesting their participation. Prior to field interviewer training and data collection, each project in the study was sent a form requesting background information essential to the data collection process and specific data used in the calculation of QC rent. The rent calculation information requested varied by program but included such items as passbook rate, utility allowance schedules, payment standards, minimum rent and flat rent. PHA/project staff verified the project type and size, and the location of project offices and files. Projects were also requested to indicate if the selected project had been designated a “special demonstration project” by HUD. If a project answered in the affirmative to this question, the status was confirmed and the project was replaced in the study. Public Housing projects were also requested to identify any income exclusions that had been adopted in addition to those specified by HUD. The data requested from the PHA/project were essential in preparation for interviewers to begin the process of collecting data and for the calculation of the QC rent. For these reasons, a 100 percent response rate to our request for information was necessary. Rigorous strategies were employed to ensure compliance and completeness of requested information prior to field data collection. As the data collection in the field began, a second mail survey was sent to a PHA/project staff person knowledgeable about certification and recertification procedures. This survey requested information about local policies and procedures that might help explain the rent error findings. Questions included staff training practices, verification procedures, workload of staff who conduct certifications and recertifications, quality control practices used to review the work of this staff, and, for PHAs, optional questions regarding their policies on interim reviews. Hiring and Training Field Interviewers. Fifty-nine field interviewers were hired to complete the field data collection. Each field interviewer was assigned a group of projects. Field interviewers typically lived in the same general area as the projects selected for the study. Ten- day training sessions were held for 34 field interviewers who had not worked in the FY 2006 study, and a three-day training was conducted for 25 interviewers who had completed the FY 2006 study. The ten-day training covered: ♦ Project background ♦ HUD programs and requirements ♦ Survey procedures ♦ Automated data collection ♦ Administrative procedures The three day training covered a review of the background and procedures and focused particularly on changes implemented for the 2007 study. II-3

II. Methodology Abstracting from Tenant Files. At certification and recertification, PHAs/projects must complete a HUD Form 50058 for each household in Public Housing and PHA-administered Section 8 programs. A HUD Form 50059 is required for all other programs in the study. Data from the HUD Forms 50058/50059 (50058/50059 Form) were entered directly into the HUD Data Collection Software (HDCS) on each field interviewer’s laptop computer. As the data were entered, the system identified potential data entry errors, such as incorrect codes or numbers, on the basis of internal calculations and consistency checks. If key data used in the rent calculation formula were missing from the 50058/50059 Form, the system alerted the interviewer and the interviewer obtained the information from another document in the tenant file or project office. These electronic checking procedures enabled field interviewers to make immediate corrections and updates. HDCS was designed to collect data in the same formats as the official 50058 and 50059 Forms published by HUD. New York City Public Housing Authority uses a format for the 50058 that differs from this standard format. However, due to the large number of NYC Public Housing cases in the study, copies of the corresponding PIC 50058 forms for these cases were requested and used for data collection when available. 5 In other projects where the 50058 forms differed from the official HUD format, paper crosswalks were developed by Macro. Specially trained Macro staff examined the data elements on the atypical form and developed a plan that illustrated which fields corresponded to the standard 50058 form reflected in HDCS. There were no crosswalks needed for the 50059 Form, since the newest format seems to be widely in use. A paper crosswalk for the 50058 form was developed for less than one percent of projects in the study. In addition to the data collected from the 50058/50059 Form, field interviewers collected data from the tenant files to document the determination of tenant eligibility and the calculation of rent. A series of Documentation Forms were created for this purpose. The Documentation Form data were entered directly into the HDCS system. The Documentation Form module also collected information indicating whether the income, asset, household composition, or expense information used by the PHA/owner was verified. HDCS compared data from the 50058/50059 Form with that entered into the Documentation Forms module and alerted the field interviewer to possible data entry errors so that data could be reviewed and any necessary corrections made immediately, while the file documents were easily accessible. During the Documentation Form data entry phase, documents from the file were photocopied when appropriate and sent to Macro weekly. Always copied were the 50058/50059 Forms, any earned income documentation, utility allowance calculation worksheets, and the most recent 9886/9887 Tenant Consent form from the file. Field interviewers were also required to photocopy file documents that provided information that was missing from the 50058/50059 Form, if that information was necessary to calculate QC rent (i.e., number of bedrooms), and any Earned Income Disregard documentation in the file, as well as documents to support Flat Rent selection. The photocopies were used to insure the accuracy of QC rent. Interviewing Tenants. An adult household member (preferably the head of the household) was interviewed in person using CAPI for this study. Interview questions focused on family 5 This was the first study where obtaining copies of the standard 50058 form for NYC Public Housing cases was applied universally. This improvement to the study process enhanced the ability to collect accurate information in a timely manner. II-4

II. Methodology composition, sources and amounts of income, assets, and applicable expenses. Data were collected for the same point in time as when the (re)certification was conducted. HDCS compared data from the 50058/50059 Form with that entered during the interview to alert the interviewer to possible errors. Requesting Verification from Third-Party Sources. When there was no evidence in the tenant file that the PHA/owner verified the information used for calculating rent, or the existing verification information did not meet requirements agreed to for this study, 6 Macro requested verification from the appropriate third-party sources. Verification was also requested from third parties when household interviews resulted in the identification of sources of income that were not shown in the tenant files. Tenants signed release forms during the household interview so that third-party verification of income and expenses could be obtained. In addition, release form cover letters were also signed by all adult members of the household to ensure that the third parties would be satisfied with the validity of the requests for verification. Third-parties completed the forms and returned them to Macro. Matching Social Security Data. Sample household members were matched with Social Security Administration (SSA) files by HUD. Using the output from this match, the Social Security and SSI benefit, and Medicare premium data for all household members were identified. These data were considered third-party verification during the final QC rent determination. D. Field Data Collection Time Periods Data were collected in the field between January 2008 and June 2008 for the certification or annual recertification that occurred during FY 2007 (October 2006 through September 2007) 7 . Field interviewers collected data related to actions that may have occurred up to 20 months prior to the file abstraction and household interview. One of the challenges of collecting data to document actions taken in the past is developing methodologies to ensure data are collected for the situation that existed at the selected point in time. For the respondent in the household interview, recalling details of life situations at a past point in time presents difficulties. This may be complicated by the fact that some respondents in this population may have unstable situations resulting from inconsistent income or changing numbers of household members. In light of this, strategies were developed to ensure consistent and accurate collection of data across program types, projects, and households in the study. Two of the strategies developed that were of primary importance to the data collection are described in this section. Quality Control Month. The month for which data were collected is referred to as the Quality Control Month (QCM). This month represents the date the rent calculation for the certification or annual recertification (conducted in FY 2007) was completed. For most households in the owner-administered programs, the QCM is the month in which the project manager (or other authorized housing project staff member) signed the 50059 Form, certifying that the information contained on the form was correct. The rent calculation date on the 50058 form was the “date modified” printed on the form. If these pieces of information were not available on the 6 For purposes of this study, verification was acceptable if it was in writing, received from the third party, and dated 60 days before or 30 days after the (re)certification was completed. 7 To account for delays between the time the work is completed by the PHA/project staff and the effective date of the (re)certification, actions effective in October 2007were included in the FY 2007 study. II-5

II. Methodology 50058/50059 Form, the field interviewer used other documentation in the tenant file to determine when the action was taken. After the QCM was established, the data from the 50058/50059 Form corresponding to the QCM was entered into HDCS. The data from the documents used by the project staff to verify information on the 50058/50059 Form in the QCM were also entered in a separate HDCS module. The household interview was conducted with frequent reminders to the respondent that questions were being asked as of the QCM. Note: If the recertification was overdue by more than 12 months, the QCM was moved forward in 12-month intervals to a point in time within FY 2007. In this situation, during the household interview, the respondent was questioned about circumstances for the month in which the recertification would have been completed had the housing project staff completed it on time. In rare situations, when the rent was calculated after the effective date of the action (because of retroactive adjustments) the QCM is the earlier of the two dates—the rent calculation or the effective date of the action. Third-Party Verification Rules. Occasionally the verifications found in the file for household composition, income, asset, and expense items were different than those required by HUD. In addition, files were likely to contain verification documents other than those intended to support the (re)certification corresponding to the QCM. To ensure that the data from the right documents (those that had been gathered to verify the information on the 50058/50059 Form being reviewed) were entered in to HDCS, and to apply rules fairly and consistently across all households in the study, a set of rules defining acceptable verification were developed. For purposes of this study, verification was considered acceptable if it was in writing, was received from a third party, and was dated 60 days before or 30 days after the date the (re)certification was completed. Field interviewers were given detailed instructions on the various types of documents they were likely to find in the file and how to classify them. The date and type of verification for each household, income, and expense item was entered in to HDCS during file abstraction. The HDCS system informed the interviewer if any items did not meet the verification requirements of the study. For the items that did not meet the requirements, the field interviewer requested written verification from the appropriate third party. E. Constructing the Analysis Files The initial database consisted of five separate files that included abstracted 50058 and 50059 Forms, tenant file information from the Documentation Form module, information from the household interview, and the third-party release forms. Data fields were at both the member and household levels, with income and expense items in hourly, weekly, monthly, or annual amounts. Macro constructed an analysis file that annualized all income and expense data at the household level. For some items, such as stable income from Social Security, this calculation was relatively easy. For other items, such as sporadic employment or medical expenses, annualizing income or deductions was more complicated. A unique linking variable was created to compare information abstracted from the 50058/50059 Form and other file documentation with information obtained in the household interview and received from third-party verification. This variable specifically identified the income/asset/expense and household member to which it belonged. II-6

II. Methodology For the calculation of rent error, the final analysis files contained income and expense/allowance data aggregated at the household level in annual amounts. Rent data were in monthly amounts. Separate files were created for the analysis of issues such as verification, internal 50058/50059 Form errors, and occupancy standards. F. Rent Formulae HUD uses specific formulae for determining tenant rents for each of its programs. The formula for determining the Total Tenant Payment (TTP) is the same for all programs except Sections 202 PRAC, 811 PRAC, and 202/162 PAC. The TTP is the greater of: 1) 30 percent of a household’s adjusted monthly income, which is one-twelfth of the total of all household members’ earned and unearned income (other than those amounts specifically excluded by HUD or PHA policy), less allowances for elderly/disabled households and for household dependents, and deductions for disability, medical, and child care expenses. 2) 10 percent of a household’s gross monthly income with no allowances or expense deductions. 3) The welfare rent in as-paid states (New York was the only as-paid state in this study). 4) The minimum rent ($25 for owner-administered projects, or an amount established by the PHA, not to exceed $50). The formula for determining the TTP for the Sections 202 PRAC, 811 PRAC, and 202/162 PAC programs includes steps (1) through (3) above, but there is no minimum rent requirement for these programs. There are five different rent calculations used to calculate the actual amount of the household’s rent depending on the program type. For the Section 8 Voucher program, household-specific characteristics also affect the calculation. These five rent calculations include: ♦ Public Housing ♦ Section 8 Project-Based (including Moderate Rehabilitation), Sections 202 PRAC, 811 PRAC, and Section 202/162 PAC ♦ Section 8 Vouchers ♦ Section 8 Enhanced Vouchers (there were no Enhanced Voucher households in the study) ♦ Manufactured Home Space Rental for Section 8 Vouchers (there were no households in the study sample that met this criterion) The household rent was calculated after data from all sources were collected. When calculating rent, a cap was placed on the maximum amount of rent the tenant was required to pay. For all Section 8 programs, this is the Gross Rent. In the Public Housing program, this is the Flat Rent. If the Flat Rent was not available, the Ceiling Rent was used to cap the rent. The rent is not capped for the Section 202 PRAC or Section 811 PRAC programs. II-7

II. Methodology Additional rent calculations were necessary for households with ineligible noncitizens. Determining the correct rent for these households is a multi-part process that first determines whether the household is entitled to continued assistance, or temporary deferral of termination of assistance, and then prorating the rent if appropriate. Two proration formulae were used—one for Public Housing and one for all Section 8 programs. The algorithms for the rent calculation formulae can be found in Appendix A. G. Calculation of Rent Error The monthly rent algorithms used by Macro to calculate the national estimates of error are the following: ♦ Actual Rent: The monthly rent indicated on the 50058/50059 Form. If this item was missing on the 50058/50059 Form, the Actual Rent was taken from another official document in the file. 8 ♦ Quality Control Rent: The monthly rent calculated by Macro using all of the verified household information. 8 Rent error was calculated by subtracting the QC Rent from the Actual Rent. A discrepancy of $5 or less between the monthly Actual and QC Rent was not considered to be an error. The $5 window was used to allow for minor calculation and rounding errors, and to focus the data analysis on major sources of error. H. Quality Control Rent Macro calculated QC Rents using the best available information. Every effort was made to use data that would have been available to the PHA/project when determining which data to use in the QC rent calculation. Each income and expense item was processed individually. For each item, Macro first used available verification from the project files. If acceptable verification was not available from the tenant file, verification was requested from an appropriate third party (see Section II-D for a discussion of acceptable verification). If the verification was not returned by the third party and the tenant file did not include verification, information obtained during the household interview was used. The following special procedures were followed when calculating the QC Rent as appropriate: ♦ Income that started after the QCM was not counted when calculating the QC Rent. ♦ Income that ended after the QCM was counted for the full year unless it was clear that the PHA/owner knew that this income was going to end. 8 Rent Roll data was not used as a substitute for Actual Rent because a previous study found that the Rent Roll sometimes included amounts to make up for previous unpaid rent, fines, or damages, etc. 8 Attempts were made to verify items that were not verified by PHA/owner staff; however, verification was not always obtained. If verification was not available, other information from the tenant file or information obtained during the household interview was used to calculate the QC rent. When calculating QC rents, codes were assigned to indicate which rents were based on verified information and those for which the income/expense information was only partially or not verified. II-8

II. Methodology ♦ Earned income bonuses were not counted unless it was clear that the bonus was paid on a regular basis. ♦ Temporary Assistance to Needy Families (TANF) and Other Welfare income were treated as the same source of income so that income listed as TANF on one form (e.g., the household questionnaire), and Other Welfare on another form (e.g., the Documentation Forms) would not be counted twice. ♦ Welfare (TANF and Other Welfare) income, Child Support income, and Child Care expenses were treated at the household level instead of the member level so that the same source of income associated with one member (e.g., the head of household) on one form, and another member (e.g., a child) on another form would not be counted twice. ♦ Disability status identified in the Social Security match data for household members receiving Social Security or Supplemental Security Income (SSI) benefits was used to determine the disability status for the recipient of the Social Security benefit. ♦ Passbook rates (for determining the imputed income from assets) for PHA-administered programs were taken from the project-level information provided by PHA/owner staff. The passbook rate for owner-administered programs is 2 percent. ♦ For new certifications, the low and very low income limits were obtained from HUD’s Web site. ♦ When determining the prorated rent for Public Housing households with ineligible noncitizens, if the Maximum Rent was not present on the 50058 form, the Fair Market Rent (FMR) was used instead of the 95th percentile of Gross Rent because the 95th percentile of Gross Rent was not available. ♦ The values from the 50058 form were used for minimum rent, gross rent, payment standard, and flat rent unless the value was missing, in which case the missing value was taken from the PHA/project-level information provided by PHA staff. ♦ The values from the 50059 form were used for gross rent and contract rent unless the value was missing, in which case the missing value was taken from the project-level information provided by owner staff. ♦ Welfare rent for the State of New York was taken from the project-level information provided by PHA staff. ♦ A separate verification code was used to identify verification obtained from the Enterprise Income Verification (EIV) system. When Social Security, SSI, or Black Lung benefits were verified with EIV, the verification was considered third party in writing. If EIV information was in the file for earned income or unemployment benefits, the dates associated with the form were examined to determine if the PHA/project staff had access to the EIV information at the time of the (re)certification. Copies of EIV (as well as other types of verification of earned income found in the tenant file) were sent to Macro headquarters and reviewed by data quality specialists to prevent mistakes in calculating the QC earned income value. II-9

II. Methodology I. HUD Requirements Complicating the Analysis Several HUD requirements affected the data collection methodology and subsequent analysis. As noted in Section II-A, relevant HUD requirements were incorporated in the study standards used to determine error. All data collection procedures and analyses were developed on the basis of these study standards. Though most standards were easily implemented, several were more problematic and they complicated the data collection or analysis, as discussed below. Anticipated Income. The amount of rent a household will pay is determined on the basis of anticipated household income and deductions for the 12 months following (re)certification. For households with a stable income source like Social Security or steady employment, annual income estimates for the next 12 months are relatively accurate. However, many assisted households have members with sporadic employment or members who move in and out of the household. Also, certain expenses such as medical expenses (for elderly/disabled households) and child care costs may be very difficult to anticipate. Determining whether such income and expense amounts were figured correctly at the time of recertification is very difficult when data are collected after the changes occurred. Every effort was made to treat questionable income or expenses in the same manner as PHA/project staff treated them. Several of the special procedures described in Section II-H were created for this purpose. Third-Party Verification. HUD regulations require that the information supplied by residents at (re)certification be verified by third parties (e.g., employers, the Social Security Administration, banks, medical personnel). Field interviewers obtained release forms from the households when evidence of verification was not present in the tenant’s file and they then requested verification from the appropriate third parties. However, some third parties did not respond, others returned information for incorrect time periods, others required payment for the information requested, and other problems were encountered in obtaining the correct verification. Follow-up requests for missing verification were not made in all cases due to time constraints. Macro and HUD established a set of verification rules to determine whether an item was verified. Section II-D shows the rules used to determine if verification was acceptable and for each matched item used in the rent calculation. Verification rates for different rent components are in Tables 1a–1f (in Appendix C) and Exhibit IV-1 in Section IV-B. Earned Income Disregard. The regulations governing the Public Housing and the Section 8 Voucher programs require PHAs to exclude a portion of earned income for households meeting certain criteria. Only participants in these programs—not applicants entering the programs—are eligible for this income exclusion. To identify households eligible for the earned income disregard, tenants were asked about training and self-sufficiency programs during the household interview. Fifty-seven household members were identified as possibly being entitled to an earned income disregard. For these household members, we examined the tenant file information on the 50058 and the Documentation Forms. We compared the QC calculated earned income exclusion (using the household questionnaire information) with the earned income used by the PHA when calculating the total annual income. When determining whether a household member was entitled to an II-10

II. Methodology earned income disregard because of unemployment, we reviewed income match data available from the NDNH. In 36 (of the 57) cases, the PHA/project did not give an earned income disregard. In 33 of these cases, the QC calculated earned income disregard also indicated that the disregard was not applicable. In 21 (of the 57) cases the PHA/project did give an earned income disregard. In 17 of these cases, the QC calculated earned income disregard also indicated that the disregard was applicable. When both the PHA and the QC calculation indicated an EID was appropriate, the actual amount of the EID matched in 13 of the 17 cases. These differences in the amount of the earned income disregard were considered as errors in this study. In previous studies no error was attributed to differences in the EID calculations. Training Programs. The regulations governing all housing programs included in this study require PHA/owners to exclude all amounts received under training programs funded by HUD, as well as the incremental earnings and benefits resulting to any family member from participation in qualifying State or local employment training programs. To identify households eligible for the training program exclusions, the field interviewers documented training program information found in the tenant file and provided during the tenant interview. Between documentation in the tenant file and information from the household interview, eight household members had indications of involvement in training programs. Only two of these eight were found to be eligible for a training program income exclusion. In the first case, the income exclusion was applied by both the PHA and during the QC process. In the second of the two cases the income exclusion was not given by the PHA. Permissible Deductions. Public Housing programs may adopt deductions from annual income in addition to HUD’s required deductions. To make sure that the appropriate additional permissible deductions were taken into consideration when determining the adjusted annual income, we looked at two sources. First, we looked at items 8b through 8e on the 50058 form where the type and amount of permissible deductions were recorded. Second, we asked a question in the Project Specific Information request to identify additional exclusions adopted by the Public Housing PHAs. We found that many PHAs use the Permissible Deduction section (items 8b through 8e) of the 50058 form to record all kinds of information that have nothing to do with permissible deductions. Therefore, we had to rely on the Project Specific Information request to determine whether the items listed on the 50058 form were in fact additional permissible deductions. On the basis of the information obtained through the Project Specific Information requests and the 50058 forms, twelve households representing six PHAs were entitled to permissible deductions. In six cases the percent of FICA tax (7.65%) was deducted from gross earned income, and in two cases the percent of FICA tax (7.65%) plus the federal income tax were deducted from the gross earned income. In one case there was a $1000 flat deduction from the gross earned income; in one case child support payments were deducted from the gross earned income, and in one case $1300 for transportation costs was subtracted from earned income. In one other case thirty percent of the earned income was deducted from the gross earned income. The permissible deduction applied for QC purposes was exactly the same as the permissible deduction allowed by the PHA. Flat Rent. Households that elected to pay a flat rent rather than an income-based rent were included in the study. For these households there is no rent error. The QC rent is the same as the Flat Rent used by the PHA. There are 71 flat rent cases in the study sample. It should be noted II-11

II. Methodology that determining if a household is paying the flat rent is not always easy because of contradicting data within the 50058 form. For most cases, items 2a-Flat Rent Annual Update, and 10u-Type of Rent Selected could be used to identify whether the household is paying the flat rent instead of income-based rent. However, if these two items contradicted one another, notations from other documents in the file were taken into consideration. Ineligible Noncitizens. HUD regulations require that rent be prorated for households with ineligible noncitizens unless the household meets certain criteria that allow continuation of full assistance. Macro reviewed all households with ineligible noncitizens to ensure that the rent was calculated correctly. No households with ineligible noncitizens were entitled to continuation of full assistance. Less than one percent of the households in the study included an ineligible noncitizen. Reduced or Terminated TANF Benefits. The regulations governing Public Housing and PHA- administered Section 8 programs included in the study require using the amount of the TANF benefit before reduction or termination, resulting from fraud or failure to cooperate with the welfare family self sufficiency program. To identify households with reduced or terminated TANF benefits, tenants were asked during the household interview about previous receipt of TANF and whether their TANF benefits were reduced during the household interview. If the TANF benefits were reduced or terminated due to fraud or failure to comply with the welfare family self sufficiency requirements, the value of the TANF benefit before the reduction or termination was used in the QC Rent calculation. 9 The TANF benefits in 51 households were reviewed and in all cases the PHA/project was accounting for TANF correctly. Students. The regulations governing PHA-administered programs included in the study require that students age 18 or over but under age 24 meet certain criteria. If these criteria are not met, the student’s parent’s income must be included when determining if the student meets the program’s financial requirements. For households with students, field interviewers documented student enrollment and member characteristics found in the tenant file and provided during the tenant interview. These households were reviewed to determine if the student met the special student criteria as defined by HUD regulations. Fifteen cases were reviewed and all were correctly receiving housing assistance. 9 The value of this reduced or terminated TANF is offset by the amount of additional income the family received that started after the time the sanction was imposed. II-12

III. Study Objectives and Analytic Methods This section presents the 14 study objectives and a brief description of the methodology used to meet them. 1 At the end of this section, Exhibit III-2 presents a chart summarizing the objectives and providing information on where each objective is addressed within the report. Objective 1: Identify the various types of rent errors, rent error rates, and calculate their variance estimates. The types of errors and error rates in the 2000 through FY 2006 studies are replicated in the FY 2007 analyses. These errors include percent of households paying correct and incorrect rent, dollar error amounts, and dollar error rates. Variance estimates (standard errors) are provided for selected error rates. Errors are determined by recalculating the tenant rent on the basis of verified QC information and subtracting this amount from the tenant rent indicated on the 50058/50059 Form (Actual Rent). The following three types of dollar rent error estimates were calculated: Dollar Rent Error—The difference between the monthly Actual Rent and the monthly QC Rent (i.e., Actual Rent minus QC Rent). A household rent is found to be in error if the difference between the Actual Rent and QC Rent is greater than $5, while “proper” rent payments reflect differences of $5 or less. Rates of exactly matching Actual and QC rents (within $1) are also presented. Simple percentages of the number of households paying the proper and exact rents are reported, as well as the percentage of households in error per program, the average gross dollars in error, and the percentage of rent dollars in error. For households who were ineligible when initially certified, the QC Rent is the flat rent for Public Housing households, or the Housing Assistance Payment (HAP) for Section 8 programs. The dollar error is this amount minus the Actual Rent. Total Component Dollars in Error—The absolute sum (i.e., the sum of the positive and negative amounts, ignoring the plus or minus signs) of all individual income and expense component errors. These errors are combined to provide an overall Total Dollars in Error and are presented as annual amounts 2 . A dollar amount of rent overpayment and underpayment was calculated for each component with identified error; however, some of these errors were overlapping or offsetting. For example, earned income may have been underreported while— perhaps because of a calculation error—Supplemental Security Income may have been overstated. The net difference could be zero, or a positive or negative amount. Largest Component Dollar Error—The annual dollar amount of error for the income or expense components with the largest error. Income and expense components include the five sources of income (earned, pension, public assistance, other income, and assets) and the five types of deductions (medical, child care, and disability assistance expenses, dependent 1 See Analysis Plan for the FY2007 HUDQC Study, Quality Control for Rental Assistance Subsidy Determinations, an unpublished Macro report to HUD, dated August 20, 2007 for a more detailed description of the methodology. 2 Because dollar component errors (CE) are reported on an annual basis while dollar rent errors (RE) are reported on a monthly basis, and rents are generally set at 30 percent of adjusted income, component errors are usually 40 times the corresponding rent error (.30 * CE = 12 * RE, or CE = (12/.30) * RE = (120/3) x RE = 40 * RE). III-1

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

III. Study Objectives and Analytic Methods allowance, and elderly/disabled allowance). If the component with the largest error is earned income, the largest dollar error would reflect the difference between the earned income used by the PHA/project, and the earned income used in the QC rent calculation. The dollar error rate is used for other error calculations, including the National Rent Error Rate and Net and Gross Error Rates. The latter error calculations link errors in the rent determination process to dollar error rates, sparking new oversight practices to better manage HUD subsidies. Objective 2: Identify the dollar costs of the various types of errors. Five types of administrative errors are linked to rent errors. Data obtained directly from the 50058/50059 Form as well as project and tenant information from the tenant file are used to identify and measure each of the following error types: ♦ Calculation errors ♦ Consistency errors ♦ Transcription errors ♦ Incorrect determination of allowances and income sources ♦ Overdue recertifications Calculation errors are detected by recalculating section subtotals and the final rent based on the exact information on the 50058/50059 Form. The tenant rent is calculated using the detailed information on the 50058/50059 Form and compared to the actual tenant rent on the 50058/50059 Form. If the two rents differ, there is a calculation error. Consistency errors are determined when there is a lack of logical conformity between elements within the 50058/50059 Form. For example, the Effective Date of Action must be on or after the Date of Admission. Elderly status information must be consistent with information about the age of the head of household or spouse. Transcription errors are detected by comparing 50058/50059 Form data with information in the tenant file. If the 50058/50059 Form data for a specific income or expense item does not match the tenant file data, a transcription error exists. Incorrect determination of allowances and income sources are identified by taking tenant file information and comparing it with the 50058/50059 Form data. Allowance errors are detected by calculating the allowances based on the tenant file information and comparing this QC allowance with the Actual Allowance on the 50058/50059 Form. Similarly, income is calculated based on the types and amounts of income reported in the tenant file. The improper application of allowances and incorrect calculation of income are a subset of transcription errors. Overdue Recertifications produce rent errors because rents are based on out-of-date information. For households with overdue recertifications, the QC information is based on the month the recertification should have been completed rather than when it was completed. III-2

III. Study Objectives and Analytic Methods Objective 3: Estimate the national-level costs for total error and major error types. This analysis includes determining the National Rent Error Rate, the numbers and proportions of households found to be in error, and the dollar amount of rent error and the proportion of total dollars found to be in error. Sample data are weighted to provide national estimates. Objective 4: Determine the relationship between errors detectable using the HUD 50058 and HUD 50059 Forms and total errors found in the study. As discussed under Objective 2, calculation and consistency errors identify mistakes made by the housing project staff. Under Objective 4, households with calculation and consistency errors are compared to households with QC errors to determine if error found within the 50058/50059 Form can be used to predict QC error. Objective 5: Determine whether error rates and error costs have statistically significant differences from program to program. This analysis presents differences in error rates by program type. Data are provided for three program groups: Public Housing, PHA-administered Section 8 (Section 8 Vouchers and Moderate Rehabilitation programs), and owner-administered (Section 8, Section 202 PRAC, Section 811 PRAC, and Section 202/162 PAC). The gross and net error rates are provided for each of these program types. The gross error rate is the sum dollar amount of gross error divided by the sum dollar amount of QC Rent, and the net error rate is the sum dollar amount of net error divided again by the sum dollar amount of QC Rent. Objective 6: Determine the apparent cause of significant rent errors, either on a sample or a comprehensive basis, to provide HUD with information on whether the error was caused primarily by the tenant or by program sponsor staff. As was done in the previous studies, we provide descriptive information on the sources of discrepancies between housing file information and verified information, and describe the incidence of administrative errors and their impacts. We also examine whether failure to verify sources of income and expenses contributes to QC error. Multivariate analyses using administrative errors and income components as independent variables are performed to identify how these errors affect the QC Dollar Rent Error. Objective 7: Determine the extent to which households are overhoused relative to HUD's occupancy standards. This objective addresses whether households reside in units with the correct number of bedrooms. Generally acceptable HUD guidelines specifying the appropriate size unit for assisted households are shown in Exhibit III-1. 3 For most programs, the rules are not based solely on household size and allow discretion on the part of the project staff. All programs allow exceptions to these rules. This study replicates the 3 Local projects have discretion in determining unit size, and may determine unit size differently than shown. III-3

III. Study Objectives and Analytic Methods analyses in the previous studies that identified bedroom size and program, and the proportion of households in compliance with and in violation of occupancy standards according to the guidelines in the table below. Exhibit III-1 PHA-Administered Section 8 Unit Size Standards Number of Bedrooms Number of Persons in Household Minimum Maximum 0 1 1 1 1 2 2 2 4 3 3 6 4 4 8 5 5 10 Objective 8: Provide information on the extent to which errors are concentrated in projects and programs. Further descriptive analyses are conducted to examine whether errors are concentrated within or are randomly distributed across PHAs/projects. Multivariate analyses are conducted with the tenant as the unit of analysis. Tenant and PHA/project characteristics were analyzed as independent variables predicting error rates. This analysis identified how each of these variables contributes to rent error. The results will help guide HUD’s management of error rates and elaborate relationships between management practices and project/tenant characteristics that affect error rates. Objective 9: Identify the percentage of newly certified tenants who were incorrectly determined eligible for program admission. Incorrect initial eligibility determinations create long-term problems for assisted-housing programs. Newly certified households are reviewed to determine whether they met the eligibility requirements for assisted housing. Five eligibility requirements reviewed at initial certification are not a part of the recertification process (and thus not confirmed on an ongoing basis): definition of family, citizenship, verification of Social Security numbers, signing consent forms, and low and very low income limits. This study did not investigate the definition of family because it is determined by the PHA or owner. Therefore, findings are provided on four of the five initial certification criteria. This study also did not include suitability factors that PHA/owners may use in selecting tenants—factors such as tenant histories, histories of drug use or criminal activity. III-4

III. Study Objectives and Analytic Methods Objective 10: Determine the extent to which Section 8 voucher rent comparability determinations are found in the tenant file, and indicate the method used to support the determination. Determine whether voucher payment standards are within 90-110 percent of fair market rents, and determine whether the correct utility allowances are being used in Section 8 voucher households. To comply with the rent reasonableness requirement, housing authorities must determine that Section 8 voucher rents are reasonable in comparison with rents for similar housing in the private, unassisted market. Using information collected from tenant files, we estimated the proportion of Section 8 voucher recipients with comparable documentation. For those with documentation, we classified the type of evidence cited in the tenant file documentation (e.g., no evidence, cited market estimates for comparable units, or the rents of one or more units considered to be comparable). We present weighted proportions of voucher recipients with rent comparability data. Additionally, payment standard data from the 50058 Form are compared with FMR data to identify the households whose payment standards fall outside the 90–110 percent FMR band. Utility allowance schedules are likewise matched to tenant files to evaluate the issues associated with independently evaluating utility allowances as a potential component of rent error. Objective 11: Estimate the total positive and negative errors in terms of HUD subsidies. Proper payments are those in which the Actual Rent equals the QC Rent. Errors can be either overpayments (Actual Rent greater than QC Rent) or tenant underpayments (Actual Rent less than QC Rent). Overpayment error rates were calculated by dividing the total amount of overpayment by the total QC Rent; underpayment error rates were calculated similarly by dividing the total amount of underpayments by the total QC Rent. Objective 12: Determine the extent to which error rates in projects that use an automated rent calculation system differ from errors in those that do not. We investigated the relationship between using an automated rent calculation system and project-level gross error rate using an Analysis of Variance. We also examined whether gross rent error differed significantly by computer use between programs. Objective 13: Determine whether other tenant or project characteristics on which data are available are correlated with higher or lower error rates. To respond to this objective, we use multivariate analysis to conduct more detailed analyses of differences among PHA/projects and provide HUD with more information for identifying projects and tenants likely to exhibit high error rates. III-5

III. Study Objectives and Analytic Methods Objective 14: Determine whether cases for which 50058/50059 Form data had been submitted to HUD were more or less likely to have errors than those for which data had not been submitted. The QC sample was matched to the TRACS/PIC data. Analysis was conducted to compare the average dollars in error for households included in TRACS/PIC with those that are not. III-6

III. Study Objectives and Analytic Methods Exhibit III-2 Summary of Study Objectives Objective: Where Objective is Addressed Exec. Summary Section IV 1 Identify the various types of rent errors, rent error rates, and calculate their variance estimates • Dollar Rent Error • Total Component Dollars in Error • Largest Component Dollar Error p. iv – vii Exhibits 2 & 5 p. 4 -7; Exhibits 3 - 5 p. 13–14; Exhibits 13 - 14 2 Identify the dollar costs of the various types of errors. • Calculation errors • Consistency errors • Transcription errors • Incorrect determination of allowances and income sources • Overdue recertifications p. vi - viii p. 21 -23; Exhibits 22 - 23 p. 13 – 14; Exhibits 12 - 13 p. 10; Exhibit 9 3 Estimate the national-level costs for total error and major error types. p. v; Exhibit 3 p. 5 - 7; Exhibits 3 - 6 4 Determine the relationship between errors detectable using the HUD 50058 and HUD 50059 Forms and total errors found in the study. p. x p. 17 – 18; Exhibit 18 5 Determine whether error rates and error costs have statistically significant differences from program to program. p. v p. 5; Exhibit IV-3 6 Determine the apparent cause of significant rent errors, either on a sample or a comprehensive basis, to provide HUD with information on whether the error was caused primarily by the tenant or by program sponsor staff. p. vi - viii p. 12 – 23; Exhibits 12 - 23 7 Determine the extent to which households are over-housed relative to HUD's occupancy standards. p. ix p. 23 – 25; Exhibits 24 8 Provide information on the extent to which errors are concentrated in projects and programs. p. x p. 28 - 29 9 Identify the percentage of newly certified tenants who were incorrectly determined eligible for program admission. p. ix p. 9 – 10; Exhibits 8 10 For Section 8 voucher households, determine: • the extent to which rent comparability determinations are found in the tenant file, and indicate the method used to support the determination. • whether payment standards are within 90-110 percent of fair market rents • whether the correct utility allowances are being used. p. ix p. 29 – 42; Exhibits 25 - 34 11 Estimate the total positive and negative errors in terms of HUD subsidies p. v; Exhibit 3 p. 10 – 12; Exhibits 10 - 11 12 Determine the extent to which error rates in projects that use an automated rent calculation system differ from errors in those that do not. p. x p. 27 13 Determine whether other tenant or project characteristics on which data are available are correlated with higher or lower error rates. p. x p. 28 - 29 14 Determine whether cases for which 50058/50059 Form data had been submitted to HUD were more or less likely to have errors than those for which data had not been submitted. p. vii p. 42 – 45 Exhibits 35 - 37 III-7

IV. Findings A. Overview Analyses were conducted using weighted sample data for 2,404 households. 1 Data are presented by the three program types that were the basis for the sampling design—Public and Indian Housing (PIH)-administered (Public Housing), PIH-administered Section 8 Housing Choice Voucher, and Moderate Rehabilitation programs (PHA-administered Section 8); and Housing- administered Section 8, Section 202 PRAC, Section 811 PRAC, and Section 202/162 PAC programs (owner-administered). Each of the major study findings, the reasons for the errors, and other background information concerning these errors are discussed below. In many of the exhibits throughout the report, the data collected during the current study (referred to as the FY 2007 data) are compared with the data collected in a previous study. The data were collected and the analysis was completed for this earlier study (referred to as the FY 2006 data) in 2007. This discussion is divided into ten parts: the errors in the rent amount based on the QC data (rent error), the errors in sources of income and expenses (component errors), the errors found using only project file data (administrative error), occupancy standards, analysis of the responses received from PHA/project staff regarding PHA/project practices (based on the Project Staff Questionnaire), multivariate analysis, findings related to rent reasonableness determinations, utility allowance analysis, payment standard analysis, and comparisons with PIC/TRACS data. The first three parts present different types of error. Rent error is error that results in an actual dollar error. A dollar error means the household paid too much rent (an overpayment) or the household paid less rent than it should have paid (an underpayment). Component errors are the income and expense components used to calculate rent. The income components are employment income, Social Security and pensions, public assistance, other income, and asset income. The expense/allowance components are elderly/disabled allowance, dependent allowance, medical expenses, child care expenses, and disability expenses. Administrative Errors are errors that result from administrative mistakes. They consist of the following: ♦ Consistency errors—errors in logical conformity between elements within the 50058 or 50059 Form ♦ Calculation errors—arithmetic errors within subsections of the 50058 or 50059 Form ♦ Transcription errors—errors in transferring information from documentation in the tenant file to the 50058 or 50059 Form. ♦ Failure to conduct a recertification in a timely manner ♦ Failure to verify information. 1 Appendix B presents the procedure used in weighting the data. IV-1

IV. Findings Component and administrative errors may or may not result in rent errors. Administrative errors tell us at what point during the rent determination process an error occurred, while the component errors tell us which income or expense caused the error. Data supporting the discussion are presented in the source tables found in Appendix C. B. Rent Error Overview. Rent errors were identified by subtracting the QC Rent from the Actual Rent. 2 The QC Rent was calculated using third-party verification whenever possible. If third-party verification was not available, information from the Documentation Forms or Household Questionnaire was used. The Actual Rent is the Tenant Rent from the 50058/50059 Form. As noted above, a household was considered to be correct (proper payment) if the QC Rent and the Actual Rent matched within $5. All exhibits included in this report (except IV-2) and all tables in Appendix C define households whose Actual and QC Rents matched within $5 as proper payments, except for the supplemental tables (designated by the letter “S”), which are based on exact matches between these two rents. Definitions of Rent Errors. Dollar error can be determined by comparing the rent the household should have paid with what it was paying, or by identifying the percentage of the Federal subsidy that was paid in error. In this study, error was determined by the first method. The rent errors presented throughout this report were calculated in the following manner: ♦ Dollar Rent Error was calculated at the household level by subtracting the household’s QC Rent from the Actual Rent. Note that these are monthly rents. A negative number indicates an underpayment, meaning the household paid less than it should have paid, and that HUD’s contribution was higher than it should have been. A positive number indicates a household overpayment, meaning HUD’s contribution was less than it should have been. ♦ Gross Rent Error is the absolute value (i.e., the sum of the absolute value of positive and negative Rent Error) of the Dollar Rent Error for the sample as a whole or a specified group of households. The Gross Rent Error functions simply as a measure of the magnitude of the errors. The dollar amounts presented in the tables are Gross Rent Error values, unless otherwise indicated. ♦ Net Rent Error is the arithmetic value (i.e., the sum of the negative and positive values of over- and underpayments) of the rent error. ♦ Error Rate is calculated by dividing the sum of the Gross Rent Error by the sum of the QC Rent, for the entire sample or a specified group of households. 2 Rent error is determined on the basis of Tenant Rent, not TTP. Error based on TTP may differ from Tenant Rent because of the program specific rent formulas applied when calculating Tenant Rent. These rent formulas are listed in Section II-F and presented in detail in Appendix A. IV-2

IV. Findings Financial Impact of Identifying Rent Error. Reduction in the rent error associated with the programs included in this study does not mean there will be an overall savings in the costs associated with administering these programs. Given there are large numbers of eligible households on waiting lists, if a household leaves the program because it is no longer eligible for a subsidy, another household will take its place. The replacement household may be entitled to a smaller or a larger subsidy than the household that left the program. Therefore, the most direct benefit of identifying households with rent error is making sure those households who are eligible for the program are receiving the correct subsidy, rather than reducing the amount of funds needed to administer the programs. The most appropriate use of this study is as a tool for strengthening HUD’s procedures for ensuring administrative compliance with regulations. The recommendations presented in this report may require greater rather than fewer resources to provide HUD, PHAs, and owners with the written policy guidelines and training, standardized forms, and on-going monitoring needed to assure the programs are administered correctly. Verification Used in Determining the QC Rent. As indicated above, a set of rules was established for third-party verification (see Section II-D). If an income or expense component was used for a rent calculation and was not verified by the PHA/owner, Macro staff sought third- party verification. However, Macro verification could not be obtained for all PHA/owner unverified items despite considerable effort and expense 3 . Exhibit IV-1 shows the percentage of each rent component that was verified by either the PHA/owner or Macro. Exhibit IV-1 Percent of Households Fully Verified by Either the PHA/Owner or Macro Third-Party Verbal or In-Writing, Documentation, or EIV Third-Party In-writing Rent Component 2006 2007 2006 2007 Earned Income 89% 90% 73% 76% Pensions 99% 98% 91% 90% Public Assistance 86% 94% 65% 73% Other Income 83% 81% 65% 58% Asset Income 86% 86% 66% 65% Child Care Expense 75% 70% 68% 59% Medical Expense 74% 75% 54% 48%

Source: Tables 1a and 1b, Appendix C

The first two columns present the percentage of rent components that were verified with third- party in-writing, third-party verbal, documentation 4 or Enterprise Income Verification (EIV). The remaining two columns present the percentage of rent components that were verified with the more stringent verification requirements for this study (i.e., third-party in-writing). As the 3 If third-party verification was not available, documentation from the tenant file was used to calculate the QC rent.

If neither third party verification nor file documentation was available, information collected during the household

interview was used to calculate the QC rent.

4 Documentation means documents submitted by the family such as pay stubs or bank statements, or a statement in

the file indicating the project staff viewed an acceptable verification (but there was no copy in the file).

IV-3

IV. Findings exhibit indicates, while primarily the percentage of households where the rent component was fully verified remained the same, for the Public Assistance rent component, the percentage of households where the rent component was fully verified increased. It should be noted that since the sample size for Disability Expenses is so small, the findings are not reliable national estimates and not included in Exhibit IV-1. Tables C-1c, C-1d, and C-1e in Appendix C provide additional verification information by rent component. They present the number of households for which the income or expense component was not verified (i.e., no component items verified), partially verified (i.e., some component items verified), or fully verified (i.e., all component items verified). Table C-1c includes items that were verified verbally by a third party. Table C-1d provides data for items verified by file documentation, and Table C-1e provides data for items verified through the EIV system. Proper Payments. Exhibit IV-2 shows the percentage of households with proper payments by program, for households where the Actual and QC Rents matched within $5 and where the Actual and QC Rents matched exactly. At (re)certification, the rent was calculated correctly (within $5) in 64 percent of the households, the same percentage as in FY 2006. There was an exact match of rent payment in 51 percent of households in FY 2007, again the same as FY 2006. Exhibit IV-2 Percent of Households with Proper Payments Percent of Households Standard Percent of Households Standard Administration Type Within $5 Error Matched Exactly Error 2005 2006 2007 2007 2005 2006 2007 2007 Public Housing 66% 65% 69% 1.8% 53% 54% 57% 2.2% PHA-Administered Section 8 63% 61% 62% 2.3% 51% 49% 50% 2.2% Total PHA-Administered 64% 62% 64% 1.6% 51% 51% 53% 1.8% Owner-Administered 63% 66% 64% 1.9% 46% 53% 48% 2.4% Total 64% 64% 64% 1.4% 50% 51% 51% 1.8%

Source: Table 2 and 2S, Appendix C

Households with QC Rent Error. Exhibit IV-3 shows the percentage of households in error, the average dollar amount in error, and error rate by program. Thirty-six percent of the households have a rent error greater than $5, compared to 37 percent in FY 2006. The average gross dollars in error, calculated by dividing the sum of the dollar amount of gross error (i.e., the sum of the absolute values of under- and overpayments) by the total number of households is $16 in FY 2007 lower than the $19 average gross dollar error in FY 2006. The gross dollar error rate, calculated by dividing the sum of the dollar amount of Gross Rent Error by the sum of the dollar amount of the QC Rent, was 7 percent in FY 2007 compared to 10 percent in FY 2006. IV-4

IV. Findings Exhibit IV-3 Percent of Households with Error, Average Dollars in Error, and Dollar Error Rate for All Households with Error Percent of Dollars Gross Dollar Error Error in Error Rate 35% 31% $15 $13 7% 6% 39% 39% $23 $20 11% 9% Total PHA-Administered 38% 36% $21 $17 10% 8% i 34% 36% $17 $13 9% 6% 37% 36% $19 $16 10% 7% and IV-4b show the This finding is the same as in FY Exhibit IV-4a Percent of In Error ( ) 17% 19% 16% $59 $54 $57 $10 $10 $9 20% 22% 19% $70 $73 $67 $14 $16 $13 Total PHA-Administered 19% 21% 18% $67 $67 $64 $13 $14 $12 i 18% 16% 19% $55 $68 $44 $10 $11 $8 19% 19% 18% $63 $67 $57 $12 $13 $11 Households with Average Gross Administration Type 2006 2007 2006 2007 2006 2007 Public Housing PHA-Administered Section 8 Owner-Admin stered Total Source: Table 3, Appendix C Underpayment and Overpayment Households. Exhibits IV-4a percentage of households and average dollar amount of error for all households when errors of $5 or less are excluded from calculations. Exhibit IV-4a and IV-4b present the error for underpayment and overpayment households, respectively. Nineteen percent of all households paid in excess of $5 less than they should have in FY 2007. 2006 and FY 2005. For the FY 2007 households, the average monthly payment error was $57, lower than the mean of $67 in FY 2006 and the mean of $63 in FY 2005. Underpayment Households Percent of Households and Average Monthly Dollar Amount of Error Average Dollar Amount of Error Households For Underpayment Households Administration Type with errors > $5 For All Households 2005 2006 2007 2005 2006 2007 2005 2006 2007 Public Housing PHA-Administered Section 8 Owner-Admin stered Total Source: Table 2 and 4, Appendix C As shown in Exhibit IV-4b, 18 percent of all households paid in excess of $5 more than they should have in FY 2007, compared to 17 percent in FY 2006, and 18 percent in FY 2005. The average monthly overpayment for households with overpayment error was $30 in FY 2007, down from $36 in FY 2006 and $39 in FY 2005. IV-5

IV. Findings Exhibit IV-4b Overpayment Households Percent of Households and Average Monthly Dollar Amount of Error ( ) Average Dollar Amount of Error Percent of Households

Administration Type In Error

For Overpayment Households with errors > $5 For All Households 2005 2006 2007 2005 2006 2007 2005 2006 2007 Public Housing 17% 15% 14% $53 $31 $26 $9 $5 $4 PHA-Administered Section 8 17% 18% 20% $38 $42 $35 $7 $7 $7 Total PHA-Administered 17% 17% 18% $43 $39 $32 $7 $6 $6 Owner-Administered 19% 17% 17% $31 $31 $24 $6 $5 $4 Total 18% 17% 18% $39 $36 $30 $7 $6 $5

Source: Table 3 and 4. Appendix C

Figure IV-1 shows the percentage of underpayments, proper payments, and overpayments by program type. Programs were grouped into three categories—Public Housing, PHA- administered Section 8, and owner-administered. Note that the majority of cases fall in the proper payment category for all program types. As indicated above, a household was considered to be correct (proper payment) if the Actual Rent and the QC Rent matched within $5. Figure IV-1: Payment by Program Type

20

10

0 70

60

50

40

30

- P er cen t o f C as es Public Housing PHA- Ow ne r Underpayment Proper Payment Overpayment Adm inistered Adm inistered Section 8 Program Type Gross and Net Dollars in Error. Exhibit IV-5 presents the gross and net average dollars in error and their associated standard error. To obtain the Gross and Net Rent Error, the dollar amount of overpayments is added to the dollar amount of underpayments, first using the absolute values for gross error, and then the arithmetic values for the net error. The net error measures the dollar cost of the errors and is -$5 (indicating a tenant underpayment) for FY 2007; the average gross dollar error is $16 for FY 2007 and represents the dollars associated with the errors (the magnitude of the errors). IV-6

IV. Findings Exhibit IV-5

Gross and Net Dollar Rent Error (Monthly) for All Households

Gross Rent Error Net Rent Error

Average Dollars Average Dollars

in Error Standard Error in Error Standard Error

Administration Type 2006 2007 2006 2007 2006 2007 2006 2007

Public Housing $15 $13 $1.54 $1.89 -$6 -$6 $1.94 $2.04 PHA-Administered Section 8 $23 $20 $2.85 $2.51 -$8 -$6 $2.86 $2.04 Total PHA-Administered $21 $17 $1.88 $1.86 -$8 -$6 $1.97 $1.72 Owner-Administered $17 $13 $2.69 $1.71 -$6 -$4 $2.71 $1.82 l $19 $16 $1.79 $1.52 -$7 -$5 $1.79 $1.57 Error Rates by Program. Exhibit IV-6 7.4% 5.6% -2.8% -2.4% PHA-Admini 11.2% 9.0% -4.1% -2.7% Total PHA-Administered 9.9% 7.8% -3.6% -2.6% i 8.7% 5.9% -2.9% -2.1% 9.6% 7.2% -3.4% -2.4% Tota Source: Table 5, Appendix C * Difference at significance p < .05 Differences in error rates by program were investigated and the results are summarized in Exhibit IV-6. Differences include Gross Error Rate, which is the sum dollar amount of gross error divided by the sum dollar amount of QC Rent, and the Net Error Rate, which is the sum dollar amount of net error divided again by the sum dollar amount of QC Rent. The Gross Error Rate is slightly higher for PHA-administered Section 8 programs than for either Public Housing or owner-administered programs. Both the Gross Error Rate and the Net Error Rate for all programs are lower in FY 2007 than in FY 2006. Gross and Net Dollar Error Rates (Monthly) for All Households Error Rates Gross Error Rate Net Error Rate Administration Type 2006 2007 2006 2007 Public Housing stered Section 8 Owner-Admin stered Total Source: Table 5, Appendix C Certifications/Recertifications. The sample households included both certifications (i.e., newly admitted households) and recertifications. Certifications were analyzed to determine if these households were eligible for HUD housing assistance and recertifications were analyzed to determine if they were overdue. Figure IV-2 presents the breakdown of cases by case type— certifications, recertifications, and overdue recertifications. IV-7

IV. Findings Figure IV-2: Case Type Overdue Recertifications 2% Certifications 16% Recertifications

83%

Source: Table 6, Appendix C Exhibit IV-7 shows the breakdown of the percentage of certifications, recertifications not overdue, and recertifications overdue, by program type. The exhibit indicates that in FY 2007 16 percent of the households were certifications, and 2 percent of the households were overdue recertifications. The findings indicate that there was no change in the percentage of certifications from FY 2006, but a slight decrease in the percentage of overdue certifications (from 3% to 2%). Exhibit IV-7 Certifications and Recertifications by Administration Type By Year* Certifications Timely Recertifications Overdue Recertifications Row Total Administration Type 2006 2007 2006 2007 2006 2007 Public Housing 14% 16% 81% 82% 5% 3% 100% PHA-Administered Section 8 15% 15% 81% 83% 4% 2% 100% Total PHA-Administered 15% 15% 81% 83% 4% 2% 100% i 17% 16% 82% 84% 1% 16% 16% 81% 83% 3% 2% Owner-Admin stered 100% Total 100 Source: Table 6, Appendix C *Rounding error may result in totals not equal to 100%. Certifications. Exhibit IV-8a presents a summary of the findings related to eligibility criteria and Exhibit IV-8b shows the percentage of newly certified households meeting the certification criteria by program type. The reviewed criteria included citizenship, Social Security number, signing the appropriate consent form, and qualifying as low income or very low income households. However, only those households that do not meet the appropriate low or very low income limit are ineligible for assistance. One hundred percent of the households (according to the QC Rent calculation) fell within the low-income limit for total gross income. IV-8

IV. Findings A household met the citizenship or Social Security number criteria if there was evidence in the tenant file that the citizenship or Social Security number was verified. The data indicate that a citizenship code (indicating whether each household member was a citizen, eligible noncitizen, or ineligible noncitizen) and a Social Security number was available (from either the tenant file or the household interview) for each household member. According to the citizenship codes, only two percent of the households had at least one household member for whom there was no verification of citizenship. This finding was the same in FY 2007 as in the FY 2006 study. To meet the citizenship verification requirement, the file must have contained (for each household member) a signed declaration of U.S. citizenship or eligible immigration status; proof of age documentation; an INS card; or INS system verification of citizenship status, or documentation that the member was in process for verification or an INS hearing. Two percent of the households had at least one member age six or over for whom there was no verification of their Social Security number. To meet the Social Security number verification requirements the file must have contained (for each household member six years of age or older) a copy of the Social Security card, or statement from the Social Security Administration verifying the Social Security number or a certification indicating the member does not have a Social Security number. In 96 percent of the households, there was a signed consent form, dated within 15 months of the QCM (the date for which data were collected), for all members age 18 or over. Note that not meeting the Social Security number, citizenship, and consent form criteria may not mean the household was not eligible for assistance; rather, the project did not follow the HUD requirements in documenting the information. Exhibit IV-8a Percent of Newly Certified Households Meeting Certification Criteria Certification Criteria Met Criterion 2006 2007 Citizenship 98% 98% Social Security Number 97% 98% Consent Form 95% 96% Low and Very Low Income 100% 100% Meets All Eligibility Criteria 91% 91% Source: Table 7, Appendix C IV-9

IV. Findings Exhibit IV-8b Certification Criteria 99% 99% 94% 95% 99% 98% 98% 94% 97% Percent of Newly Certified Households Meeting Certification Criteria by Program Type Percent of Households Meeting the Criteria PHA-Administered Public Housing Section 8 Owner-Administered Citizenship Social Security Number Consent Form 99% Meets All Eligi 92% 92% 88% recertification. ). The result is an Exhibit IV-9 Low and Very Low Income 100% 100% bility Criteria Source: Table 7b, Appendix C Underpayments and Overpayments for Certifications, Recertifications, and Overdue Recertifications. Exhibit IV-9 presents a summary of the households with overpayments and underpayments by the type of case—certification, timely recertification, and overdue The Average Dollar Amounts are based on the sum of the dollar amounts for payment errors (either underpayment or overpayment) for the type of household (certification, overdue recertification, or timely recertification) divided by the number of households with that payment type (for whom a QC Rent could be calculated For example, the sum of the dollar amounts for new certifications with monthly underpayments ($6M) was divided by the total number of certifications for whom QC Rent could be calculated (.64M). underpayment average dollar amount of $9. The data indicate that the amount of underpayment and overpayment dollar error in new certifications in FY 2007 is about the same as the amount for recertifications. As might be expected, there is a very large difference in the underpayment error for overdue and timely recertifications ($53 and $10, respectively). $11 $9 $6 $6 $13 $10 $6 $5 $34 $53 $20 $6 $13 $11 $6 $5 Subsidies. For For Public Housing, the Average Monthly Underpayment and Overpayment Dollar Amount Averaged Across All Households Household Type Underpayment Average Dollar Amount Overpayment Average Dollar Amount 2006 2007 2006 2007 Certifications Timely Recertifications Overdue Recertifications Total Source: Table 8, Appendix C The actual cost of errors to HUD is expressed in terms of subsidy payments. purposes of this study, HUD subsidies for the Section 8 voucher program equal the lower of the Gross Rent or the applicable payment standard minus the Tenant Share. subsidy is the applicable payment standard minus the TTP, and for Housing programs, the subsidy is the Gross Rent minus the TTP. The subsidy is correct if the Actual Rent equals the IV-10

IV. Findings QC Rent (within $5). A negative subsidy error occurs when the tenant pays too much rent (QC Rent < Actual Rent). A positive subsidy error occurs when the tenant pays too little rent (QC Rent > Actual Rent). These subsidy errors by program type are summarized in Exhibit IV-10a and 10b, below. The subsidy errors by certification status are summarized in Exhibit IV-11. Exhibit IV-10a

Negative Subsidy Households (Tenant Overpayment)

Percent of Households and Average Monthly Dollar Amount of Error

Average Dollar Amount of Error Percent of

Households in

Administration Type Error

For Negative Subsidy Households (with errors > $5) For All Households 2006 2007 2006 2007 2006 2007 Public Housing 15% 14% $31 $26 $5 $4 PHA-Administered Section 8 18% 20% $42 $35 $7 $7 Total PHA-Administered 17% 18% $39 $32 $6 $6 Owner-Administered 17% 17% $31 $24 $5 $4 Total 17% 18% $36 $30 $6 $5 Source: Tables 3 and 4. Appendix C Note: Table results replicate Exhibit IV-4b for the convenience of the reader. Exhibit IV-10b

Positive Subsidy Households (Tenant Underpayment)

Percent of Households and Average Monthly Dollar Amount of Error

Percent of Error ( ) 19% 16% $54 $57 $10 $9 22% 19% $73 $67 $16 $13 Total PHA-Administered 21% 18% $67 $64 $14 $12 i 16% 19% $68 $44 $11 $8 19% 18% $67 $57 $13 $11 Average Dollar Amount of Error Households in For Positive Subsidy Households Administration Type with errors > $5 For All Households 2006 2007 2006 2007 2006 2007 Public Housing PHA-Administered Section 8 Owner-Admin stered Total Source: Tables 3 and 4, Appendix C Note: Table results replicate Exhibit IV-4a for the convenience of the reader. IV-11

IV. Findings Exhibit IV-11 Average Monthly Dollar Amounts of Error for Negative (Tenant Overpayment) and Positive (Tenant Underpayment) Subsidies Averaged Across All Households Household Type Negative Subsidy Average Dollar Amount of Error Positive Subsidy Average Dollar Amount of Error 2006 2007 2006 2007 Certifications $6 $6 $11 $9 Timely Recertifications $6 $5 $13 $10 Overdue Recertifications $20 $6 $34 $53 Total $6 $5 $13 $11 Source: Table 8, Appendix C

Note: Table results replicate Exhibit IV-9 for the convenience of the reader.

C. Sources of Error Additional analyses examined which income and expense components contributed the most to rent error. It should be noted that the component dollar amounts are annual income and expense dollars, rather than the monthly figures used to present rent error data, and that rents are generally computed at 30 percent of adjusted income. Therefore, every $100 of income or expense error generally translates into $2.50 of rent error. In addition, the sum of the component errors is greater than net rent errors because of off-setting errors. For example, the household presented in the chart below has earned income and child care costs with errors in both components. The total component error is $1000 ($800 + $200); however, the adjusted net income error (the amount used to determine the household’s rent) is only $600. Example: Component File Data QC Data Dollar Error Earned Income $2,200 $3,000 $800 Child Care Expense $400 $600 $200 Adjusted Income $1,800 $2,400 $600 Exhibit IV-12 presents each income and expense component included in the rent calculation and the percent of the households in error 5 where this component contributed the most to the gross error. The exhibit indicates that the largest average dollar error continues to be in earned income, with an average error of $2,887, in the 24 percent of households in error where earned income is the largest component error. Medical expense was the next most frequent component, 23 percent of the errors with an average dollar error of $972. Pensions were a component of error 22 percent of the time, with an average associated dollar error of $2,075. Other income was the largest component of error in 11 percent of households in error, with the associated average dollar amount being $2,437. 5 The denominator in the percentage is the number of households with any component error, which was 36 percent of total households in FY 2007. IV-12

IV. Findings Between FY 2005 and FY 2006, average dollar error amounts had increased for some income Rent components and decreased for others. In FY 2007, average dollar error amounts have shown modest decreases in all components except childcare and public assistance. The largest component increase was in public assistance which went up by over $600 in FY 2007 to an average dollar error amount of $2,492 compared to $1,823 in FY 2006. The largest component decrease was seen in earned income. In the 24 percent of households where earned income was the largest component in error, the average dollar amount of error was $2,887 in FY 2007 compared to $4,544 in FY 2006, a decrease of over $1600. Exhibit IV-12 Rent Components Responsible for the Largest Dollar Error for Households with Rent Error Rent Component Percent of Households in Error Average Dollar Amount 2006 2007 2006 2007 Earned Income 26% 24% $4,544 $2,887 Other Income 10% 11% $2,488 $2,437 Pensions 25% 21% $2,246 $2,075 Asset Income 2% 4% $1,733 $1,502 Public Assistance 9% 6% $1,823 $2,492 Child Care Allowance 4% 4% $2,128 $2,259 Medical Allowance 17% 23% $1,099 $972 Dependent Allowance 3% 3% $703 $622 Elderly/Disabled Allowance 2% 3% $400 $400 No Rent Component Error 1% 1% $0 $0 Total 100%* 100%* $2,513 $1957 Source: Table 9, Appendix C *Numbers do not add up to 100% due to rounding. Note that for some households the rent error is not caused by one of the ten components listed. Rather, it is caused by other arithmetic errors or using the wrong rent calculation formula. The percent of households in error stayed the same or changed slightly for most Rent components, with the largest increase in percent of households with medical allowances, from 17 percent in FY 2006 to 23 percent in FY 2007. Total and Largest Component Dollar Error by Program Type. Exhibit IV-13 shows the dollar amounts associated with the total dollars in error (the sum of the absolute value of errors in all Rent components) and the largest dollars in error (the largest error attributable to a specific source for each household), by program type. There were notable decreases in Average Total Dollars in Error for all program types from FY 2006 to FY 2007, with Owner-administered Housing showing the largest decrease of $929. There were also notable decreases in the Average Largest Dollars in Error for all program types in FY 2007. On average, the total for all programs decreased by $556 between FY 2006 and FY 2007. IV-13

-- -- -- -- -- -- -- IV. Findings Exhibit IV-13 Total and Largest Component Dollars in Error for Households with Rent Error Dollars in Error Dollars in Error Administration Type Average Total Average Largest 2006 2007 2006 2007 Public Housing $2,839 $2,126 $2,216 $1,778 PHA-Administered Section 8 $3,344 $2,688 $2,752 $2,281 Total PHA-Administered $3,187 $2,525 $2,586 $2,135 Owner-Administered $2,836 $1,907 $2,340 $1,583 Total $3,083 $2,326 $2,513 $1,957 Source: Table 10, Appendix C QC Rent Components by Payment Type and Administration Type. Exhibit IV-14 shows the percentage of the total number of households with (and without) component error by component type and payment type. For example, six percent of all households with underpayment rent error had errors in earned income, five percent of households with proper payment had errors in earned income and five percent of households with overpayment rent had errors in earned income. It also shows this information for PHA- and owner-administered households. Exhibit IV-14 Rent Component Error by Payment Type for All Households 7% 4% 6% 6% 3% 5% 6% 3% 5% 7% 8% 7% 7% 11% 9% 5% 8% 6% 2% 1% 1% 2% 1% 2% 1% 1% 1% 4% 3% 4% 4% 2% 3% 3% 1% 2% 2% 7% 4% 5% 7% 6% 2% 4% 3% 1% 1% 1% 1% 1% 2% 1% 2% 1% 1% 1% 2% 1% 2% 1% 1% 1% 2% 1% 1% <1% <1% <1% Medical 4% 10% 6% 4% 12% 7% 6% 11% 8% 43% 39% 42% Underpayment Proper Payment Overpayment Rent Component PHA Owner Total PHA Owner Total PHA Owner Total Earned Income Pensions Public Assistance Other Income Asset Income Dependent Allowance <1% Elderly/Disabled Allowance <1% <1% <1% Child Care Allowance <1% <1% <1% Disability Allowance Allowance No Rent Component Error <1% <1% <1% <1% <1% Source: Table 11, Appendix C Exhibit IV-14 reflects component errors in proper payment households when the component dollar error results in a tenant payment error of $5 or less. The exhibit indicates that medical expenses is the rent component that has the highest percentage of error (14 percent = 6 percent underpayment + 8 percent overpayment), followed by pension income (13 percent) and earned income (11percent). The components with the highest error remain the same. IV-14

-- -- -- -- IV. Findings Allowances. Elderly/disabled and dependent allowances were examined to determine whether these allowances were being applied correctly. 6 The findings are summarized in Exhibit IV-15. The exhibit shows the percentage of elderly/disabled and nonelderly/disabled households for which allowances were correctly or incorrectly applied. Elderly/disabled allowances were incorrectly used in five percent of the households in FY 2007. Five percent of the elderly/disabled households received an incorrect allowance, while less than one percent of non- elderly/disabled households received an allowance. The exhibit also shows the percentage of households with and without dependents for which a dependent allowance was correctly or incorrectly applied. The dependent allowances were incorrect in eight percent of the households. In less than one percent of the households, a dependent allowance was given to a household that did not have dependents. For the remainder of the households with dependents in error (8 percent), either a dependent allowance was not given when it should have been or the wrong allowance amount was given. Exhibit IV-15 Elderly/Disabled Allowances and Dependent Allowances Elderly Allowance Non-Elderly/ Elderly/ Disabled Disabled Dependent Allowance Households Households All Without With All Allowance Households Households Households Dependents Dependents Households No Allowance 100% 42% 100% 56% Incorrect Allowance <1% 5% 3% <1% 8% 4% Correct Allowance 95% 55% 92% 40% lTota 100% 100% 100% 100% 100% 100% Source: Tables 12a and 12b, Appendix C D. Errors Detected Using Information Obtained From Project Files To respond to HUD’s interest in understanding the cause of errors, tenant rent was recalculated using only income and expense items documented in the tenant file. The source of information used for this analysis only included items that were clearly documented in the tenant file in a location other than the 50058/50059 form worksheet. If an item was recorded on the 50058/50059 Form worksheet but not documented elsewhere in the tenant file, it was not included when the tenant file tenant rent was calculated for this analysis. Therefore, it is possible that some of the discrepancies identified between 50058/50059 rents and rents calculated solely based on file data were not, in fact, due to incorrect determinations but rather due to program sponsor failure to maintain information supporting income or expense items. The outcome is that relying solely on information in tenant files may result in misstating the basis for the program sponsor income and rent determination and could lead to a determination that an error existed when the determination was actually correct. The fact remains that, even if 6 Households with an elderly or disabled head or spouse are entitled to one $400 allowance (i.e., deduction from gross annual income) in calculating rent. Households are entitled to a $480 allowance for each dependent (defined as children under 18, full-time students, and disabled members other than the head or spouse). IV-15

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

IV. Findings a program sponsor made the correct income determination, failure to document the determination is and should be treated as a serious administrative problem. Also, in practice, it appears that these types of discrepancies are often suggestive of subsidy determination errors even if they cannot be assumed to prove the existence of such errors. The findings from this analysis were compared to the quality control findings where tenant rent was calculated based on all the information collected during the study (including household interview data, and verification obtained by Macro through third party sources). Exhibit IV-16 shows the percent of households in error and the average dollar error with and without income and expense items identified during the household interview and verified by Macro through third party sources. The data indicate that the income and expense items documented in the tenant file identify just over half of the cases with tenant underpayments (subsidy overpayments) and subsidy underpayments (tenant overpayments). The data regarding average dollar error indicate the tenant file more closely predicts the subsidy overpayments, but overestimates subsidy underpayments. Exhibit IV-16 Findings With and Without Information Obtained from Sources Other Than the Tenant File Percent of Households in Error Average Dollar Error Error Source Subsidy Subsidy Subsidy Subsidy Overpayment Underpayment Overpayment Underpayment Error Based on All Income and 18% $57 $30 Expense Items Identified during 18%

the Study

Error Without Income and 11% $56 $70 Expense Items Identified during 11%

the Household Interview

Source: QC Tables 2 and 4, and Tenant File Table 2 and 4, Appendix C Analysis of the errors on the 50058/50059 Form examined whether the errors identified using the 50058/50059 Form as a sole source of information are representative of the total errors in the program. The analyses focused on calculation and consistency errors: Calculation error was identified from income, expenses, and allowances used to calculate the rent amount and recorded on the 50058/50059 Form. This calculation did not take into account whether dollar amounts were verified or whether the recertification was conducted on time. This analysis identified errors due to arithmetic mistakes, the incorrect use of a formula, and items that were not completed but should have been. This analysis did not identify households where items were recorded in the wrong place on the 50058/50059 Form, although improper use of a field on the 50058/50059 Form can result in a calculation error. Table C-13 in Appendix C presents the number of households with 50058/50059 Form that contained calculation errors by the rent component contributing to the error. The items considered when determining calculation error, are listed in Appendix D. IV-16

IV. Findings Consistency errors were based on the logical conformity of elements in the 50058/50059 Form. For example, the effective date of action must be on or after the date of admission, elderly status information should be consistent with household head and spouse ages, and number of dependents should not exceed the number of household members. Table C-14 in Appendix C shows the number of households with consistency errors on the 50058/50059 Form, summarized by form subsections. Appendix D lists the data items by subsection that were included in this analysis. Exhibit IV-17 shows the percentage of households with calculation and consistency errors by 50058/50059 Form subsections. It is important to emphasize that the 50058 form is formatted differently and has more line items of information than the 50059 Form. Consequently, the number and types of calculation and consistency errors on the forms differ, and findings from the two forms are not directly comparable. In addition, the Office of Housing implemented a new version of the 50059 form in FY 2006. The large number of calculation errors (particularly on the 50058 forms) may be a contributing factor to QC errors, though a calculation or consistency error does not necessarily lead to a rent error. The PHA/owner may make an error when completing one section of the form, and still calculate the rent correctly. Exhibit IV-17 Percentage of Households with Calculation and Consistency Errors Percentage of Households 50058/50059 Item Calculation Errors Consistency Errors 50058 50059 Total 50058 50059 Total General Information n/a n/a n/a 2% 5% 3% Household Composition 8% 3% 6% 6% 3% 5% Net Family Assets and Income 8% 6% 7% 10% <1% 7% Allowances and Adjusted Income 43% 5% 31% 10% 1% 7% Family Rent and Subsidy Information 10% 3% 8% 3% 2% 3% Source: Tables 13 and 14, Appendix C Comparison of 50058/50059 Errors to QC Error. A comparison was made between the rent calculation errors on the 50058/50059 Form and errors identified through the QC Rent calculation process. The purpose of this comparison was to determine if errors identified using only the 50058/50059 Form data could predict the rent errors found in a QC review. When using only the 50058/50059 Form data to calculate the Actual Rent, errors were found in 7 percent of the households in FY 2007, a small increase from FY 2006’s figure of 5 percent. The QC error calculation found errors in 36 percent of the households in FY 2007, essentially the same as FY 2006’s 37 percent. The results are quite different from the individual and joint comparison methods. Error was found in both the 50058/50059 Form calculation and QC rent calculation in only 3 percent of the households. In 40 percent of the households, rent calculation error was found in either the 50058/50059 Form or the QC rent calculation, but not in both. This emphasizes that data from the 50058/50059 Form alone cannot accurately identify rent error. Exhibit IV-18 summarizes these results for FY 2006 and FY 2007. IV-17

IV. Findings Exhibit IV-18 50058/50059 Rent Calculation Error Compared with QC Rent Error Percentage of Percentage of Households Households Rent Calculation Correct Incorrect 2006 2007 2006 2007 Using Information on the 50058/50059 Form 95% 93% 5% 7% According to the QC Rent Calculation 64% 64% 37% 36% Both 50058/50059 Form Calculation and QC Rent Calculation 61% 60% 3% 3% Verification errors were identified by whether an item was verified by the project and, if it was, whether the correct information was transferred to the 50058/50059 Form. An error occurs when the verified amount obtained by the project is not recorded properly on the 50058/50059 Form (and, presumably, not used in the rent calculation). When determining whether a verified income or expense item matched the amount used on the 50058/50059 Form, we assumed a variance of $100 to accommodate potential rounding errors when annualizing data. Table C-15a in Appendix C shows the number of households where verification (of any type) was not obtained, where it was obtained but did not match the amount used on the 50058/50059 Form, and where the verified amount did match the 50058/50059 Form. Table C-15b provides the same information but only includes the number of households where verification was obtained from third parties in-writing (as required by the study). Tables C-15e and C-15f provide the same data by program type. Exhibit IV-19 summarizes the findings in Table C-15a. The percentage of items verified by the PHA/owner increased in six of the seven Rent components between FY 2006 and FY 2007 with the largest percentage increase of verification seen in Public Assistance (77 percent verified in FY 2006 compared to 89 percent in FY 2007), Pensions (91 percent verified in FY 2006 compared to 96 percent in FY 2007), and Medical Expenses (89 percent verified in FY 2006 compared to 93 percent in FY 2007). The percentage of items where the verification matched within $100 tended to be slightly higher in FY 2007 compared to FY 2006, with only Other Income and Child Care Expense verification showing decreases in percent of cases matching the 50058/50059 within $100. In FY 2007, the number of households where verification was not obtained by the PHA/owner decreased by a few percent in most Rent components. For example, Public Assistance showed the largest decrease (23 percent in FY 2006 compared to 11 percent in FY 2007). Pension Income verification showed the next highest decrease (9 percent in FY 2006 compared to 4 percent FY 2007), followed by Medical Expenses (11 percent in FY 2006 compared to 7 percent in FY 2007). IV-18

IV. Findings Exhibit IV-19 Verification of 50058/50059 Rent Components by PHA/Owners No Project Item Verified by Verification Matched Rent Component Verification Project 50058/50059 Within $100 2006 2007 2006 2007 2006 2007 Earned Income 12%

Pensions 9%

Public Assistance 23%

Other Income 26%

Asset Income 9%

Child Care Expense 12%

Medical Expense 11%

10% 88% 90% 68% 68% 4% 91% 96% 78% 84% 11% 77% 89% 65% 75% 25% 74% 75% 63% 61% 7% 91% 93% 84% 84% 12% 88% 88% 71% 67% 7% 89% 93% 67% 75% Source: Table 15a, Appendix C Exhibit IV-20 shows verification results by program type, again showing the verification rate for each rent component and the proportion that matched within $100 of the 50058/50059 Form amounts. When comparing the FY 2007 results to the FY 2006 findings, the following changes are of note: ♦ In the Public Housing program, there were increases in the verification rate for five out of the seven Rent components in FY 2007 when compared to FY 2006 with the largest rise occurring in public assistance verification (89 percent in FY 2007 compared to 75 percent in FY 2006). Percentage verified for medical expenses increased slightly (92 percent in FY 2007 compared to 90 percent in FY 2006). Verification rate for earned income increased from 85 percent in FY 2006 to 91 percent in FY 2007. Pension income was verified more often in FY 2007, 95 percent of the time compared to 88 percent in FY 2006. There was an increase in the percentage of verification rates in child care expenses to 81 percent in FY 2007 compared to 74 percent in FY 2006. The percentage to which Rent components declined were in asset income (85 percent in FY 2007 compared to 94 percent in FY 2006), and other income (65 percent in FY 2007 compared to 70 percent in FY 2006). ♦ In the PHA-administered Section 8 programs, there were increases in percentages of all Rent components verified comparing FY 2007 and FY 2006 The greatest gains were seen in public assistance, from 78 percent verified in FY 2006 to 89 percent verified in FY 2007, asset income, with 95 percent verified in FY 2007 compared to 89 percent verified in FY 2006, and medical expense from 74 percent verified in FY 2007 compared to 90 percent in FY 2006. Not only was there an increase in percentages verified, but verifications were used more often in most cases in FY 2007 than in FY 2006. One exception was a drop in child care expense verifications which fell from 90 percent verified in FY 2006 to 89 percent in FY 2007. In most cases verification was used more often when comparing FY 2007 to FY 2006. The most notable increase was in medical expense verifications which increased from 65 percent in FY 2006 to 79 percent in FY 2007. Other income and child care expenses were the only two components that verification was used less often, 63 percent in FY 2007 compared IV-19

IV. Findings with 65 percent in FY 2006, and 68 percent in FY 2006 compared to 66 percent in FY 2007, respectively. ♦ In the owner-administered programs, increases in percentages verified were seen in all Rent components in FY 2007, except for child care expenses which remained at 92 percent. Public assistance verifications increased the most, 79 percent in FY 2006 compared to 90 percent in FY 2007 and asset income showed the next greatest increase, 96 percent in FY 2007 compared to 91 percent in FY 2006. However, the percentage of verifications actually used in rent calculation increased slightly for all components, except earned income and child care expenses, with the greatest increase in verification use occurring in pension income from 81 percent matching in FY 2006 to 86 percent matching in FY 2006. Earned income was verified 93 percent of the time in FY 2006 compared to 90 percent in FY 2005, however verification use decreased by 4 percentage points in FY 2007, 70 percent compared to 74 percent in FY 2006. Exhibit IV-20 Verification of 50058/50059 Rent Components by PHA/Owner Staff by Program* PHA-Administered Public Housing Section 8 Owner-Administered Rent Component Verified Matched** Verified Matched** Verified Matched** Earned Income 91% (85%) 62% (64%) 89% (88%) 70% (67%) 93% (90%) 70% (74%) Pensions 95% (88%) 83% (71%) 94% (90%) 82% (80%) 97% (93%) 86% (81%) Public Assistance 89% (75%) 69% (65%) 89% (78%) 74% (60%) 90% (79%) 82% (76%) Other Income 65% (70%) 53% (54%) 78% (75%) 63% (65%) 76% (75%) 64% (63%) Asset Income 85% (94%) 77% (86%) 95% (89%) 87% (81%) 96% (91%) 86% (85%) Child Care Expense 81% (74%) 66% (59%) 89% (90%) 66% (68%) 92% (92%) 69% (81%) Medical Expense 92% (90%) 74% (62%) 96% (90%) 79% (65%) 92% (88%) 74% (69%) Source: Table 15g, Appendix C * Findings from FY 2006 are in parentheses. ** Matched within $100 Tenant File Verification Compared with QC Error. Errors identified through the QC process were investigated to determine whether they were associated with sources of income and expenses. Exhibit IV-21 presents the percentage of households with QC error for which verification was missing in the tenant file. Each error is presented by rent component. The data indicate that missing verification does have a major impact on error. Verification for most Rent components was missing in at least 71 percent of all households with QC error. Missing verification for Public Assistance was an exception with 59 percent of households with missing Public Assistance verification also having QC errors. Data from PHA-administered programs show that 13 percent had a QC error in earned income with 74 percent of the cases in error missing verification for earned income. There were minor changes in these findings when compared to 2006 data. These changes were most evident in the following rent component: Public assistance across all program types showed a significant decrease in errors with missing verification, (PHA-administered, 73 percent in FY 2006 compared to 59 percent in FY 2007, owner-administered, 96 percent in FY 2006 compared to 85 percent in FY 2007). However, asset income showed an increase in missing verification, (73 percent in FY 2006 compared to 81 percent in FY 2007). Owner-administered programs showed significant changes in two of the IV-20

-- -- -- -- -- -- IV. Findings rent component, child care expenses decreased significantly, (82 percent in FY 2006 compared to 71 percent in FY 2007). However, there was a large increase in missing verifications between FY 2006 and FY 2007 for medical expense from 80 percent to 89 percent. Exhibit IV-21 QC Error QC Error 14% 13% 68% 74% 15% 12% 86% 85% 5% 3% 73% 59% 7% 7% 80% 75% 5% 5% 73% 81% 3% 2% 81% 82% <1% 1% Medical 10% 11% 90% 90% 64% 66% 8% 6% 69% 71% 15% 15% 84% 81% 3% 2% 96% 85% 6% 4% 83% 81% 6% 11% 81% 77% 2% 2% 82% 71% 0% 0% 15% 20% 80% 89% 68% 65% QC Error Households with Missing Verification in the Tenant File 50058 Households with Households with Households with QC Errors and Missing Verification Households with QC Errors and Missing Verification Rent Component 2006 2007 2006 2007 2006 2007 2006 2007 Earned Income Pensions Public Assistance Other Income Asset Income Child Care Expense Disability Expense 100% 100% Expense No Component Error 50059 Source: Tables 16a and 16b, Appendix C Summary of 50058/50059 Form Errors. Exhibit IV-22 provides a summary of the errors identified from the 50058/50059 Form. These include consistency errors, calculation errors, and overdue recertifications. The exhibit shows the percentage of households in error, the average dollar error, and the standard errors for both households with recalculated 50058/50059 Form error (error determined using only the 50058/50059 Form), and households with QC Rent error. This information is provided for households with error for each error type. Beginning with the FY 2005 study, transcription error for any household was added to this exhibit and the data that was described as an unduplicated count of 50058/50059 Form error has been revised to an unduplicated count of any type of administrative error. The exhibit shows that most individual types of 50058/50059 Form errors are not closely associated with QC rent error. However, 50058/50059 Forms with transcription error are associated with QC rent error in 62 percent of households and any type of administrative error (transcription, consistency, calculation, or overdue recertifications) are associated with QC Rent Error in 72 percent of the households. When the findings in this exhibit are compared with the FY 2006 findings, there are significant decreases in percentage of households with QC rent for overdue recertifications and households with other calculation error. There were also significant decreases in the households for recalculated 50058/50059 error, households with transcription error (53% in FY 2006 compared to 40% in FY 2007), households with allowance calculation error (12% in FY 2006 compared to 5% in FY 2007), households with other calculation error (19% in FY 2006 compared to 12% in FY 2007), and overdue recertifications (8% in FY 2006 compared to 2% in FY 2007). IV-21

IV. Findings In addition, both the average dollar error for households with QC rent error and the average dollar error for recalculated 50058/50059 error show some declines relative to FY 2006 figures. This is true for households with recalculated 50058/50059 error, with income calculation ($189 in FY 2006 and $94 in FY 2007. There were no significant increases in average dollar error amounts or percent of households in error. To understand the reason for the change in the average dollar error for households with recalculated 50058/50059 error, it is important to review how this number is calculated. It is the average dollar rent error for all cases (based on recalculated 50058/50059 Form rent error—not QC rent error) that have error in the category identified in the row header. So, for example, although the average rent error dollars for households with income calculation errors is $94, because many of these cases have a large rent error (which may have nothing to do with the income) and the number of cases with income calculation error is small (4 percent of households in error), the average dollar error is large. Exhibit IV-22 50058/50059 Administrative Error: Percent of Households, Average Dollars in Error Households with Recalculated Households with 50058/9 Error QC Rent Error Error Type Based Percent of Standard Average Standard Percent of Standard Average Standard on 50058/50059 Households Error of Dollar Error of Households Error of Dollar Error of Recalculation in Error Percent Error Mean in Error Percent Error Mean Households with Transcription error 40% 4.5% $24 $8.75 62% 3.0% $48 $3.99 Households with Consistency Error 32% 4.9% $33 $9.54 23% 2.3% $43 $6.20 Households with Allowance 1.9% $88 $20.23 5% 1.0% $43 $14.16 Calculation Error 5% Households with Income 1.7% $94 $32.93 4% 0.8% $34 $10.94 Calculation Error 4% Households with Other Calculation Error 12% 4.2% $44 $18.94 8% 1.3% $68 $10.39 Overdue Recertifications 2% 1.3% $78 $93.84 3% 0.8% $78 $23.95 Unduplicated Count, Any Type of Administrative Error 59% 4.3% $24 $7.30 72% 2.3% $46 $3.62 l $19 $5.52 $44 $3.12Total Househo ds 100% 100% Source: Table 17, Appendix C Summary of Administrative Errors. As outlined in the study objectives, calculation errors, consistency errors, transcription errors, failure to recertify on time, and failure to apply allowances appropriately produce administrative errors. Exhibit IV-23 shows the Gross and Net IV-22

IV. Findings Rent Errors for households with each type of administrative error. Starting in FY 2005, two major changes were made to this exhibit. First, the category of consistency errors was added to illustrate inconsistencies found within the 50058/50059 Form. Second, the findings are based on QC error rather than recalculated 50058/50059 error. Percent of households in error remained stable when compared to FY 2006 for all error types. Gross average dollars in error for consistency errors decreased from $25 in FY 2006 to $17 in FY 2007; otherwise the gross average amounts by error type are comparable to FY 2006. Net average dollars in error remained comparable for income calculation errors and any administrative errors; decreased for consistency errors and allowance calculation errors; and increased for transcription errors and overdue recertification errors. Exhibit IV-23 Administrative Error: Percent of Households, Average Dollars in Error for All Households Gross Rent Error Net Rent Error Percent of Average Standard Average Standard Households Dollars Error of Dollars Error of Error Type in Error in Error Mean in Error Mean Transcription Errors 38% $28 $2.65 -$10 $3.25 Consistency Errors 20% $17 $3.09 -$3 $3.05 Calculation Errors—Allowances 3% $27 $9.15 -$0 $10.12 Calculation Errors—Income 3% $16 $4.66 -$4 $6.96 Calculation Errors—Other 6% $34 $5.66 -$15 $5.22 Overdue Recertifications 2% $59 $21.36 -$47 $21.22 Any Administrative Errors 51% $24 $2.33 -$8 $2.78 l $16 $1.63 -$5 $1.68 Tota 100% Source: Table 18, Appendix C E. Occupancy Standards Exhibit IV-24 presents a summary of the analysis that determined whether households are assigned units with the correct number of bedrooms. It shows the percentage of households by actual number of bedrooms and correct number of bedrooms according to the guidelines used in the study. Note that the guidelines used in this study are generally acceptable HUD guidelines. All programs allow exceptions to HUD’s rules. The Section 8 Voucher program sometimes allows households to rent units with fewer or more bedrooms than specified by the guidelines. Fifteen percent of all households occupied a unit with too many or too few bedrooms in FY 2007, according to the guidelines used for this study. This number is up slightly from FY 2005, where fourteen percent of all households occupied a unit with an incorrect number of bedrooms. Seventeen percent of Public Housing households, seven percent of households, and nineteen percent of Housing Choice voucher program households were over- or under-housed in FY 2007. IV-23

-- -- -- -- -- -- -- -- -- -- -- -- -- IV. Findings Exhibit IV-24 Percentage of Households in Units with the Correct Number of Bedrooms According to Study Guidelines Owner- PHA-Administered Administered Total Number of Public Housing HCVP

Bedrooms

2006 2007 2006 2007 2006 2007 2006 2007

0 98% 96% 77% 90% 98% 100% 94% 96% 1 100% 99% 100% 100% 99% 100% 99% 100% 2 80% 71% 70% 74% 85% 75% 77% 74% 3 83% 81% 85% 77% 84% 93% 84% 81% 4 55% 55% 45% 58% 53% 44% 48% 56% 5+ 35% 19% 34% 58% -- -- 31% 43% All Units 87% 83% 81% 81% 93% 93% 86% 85% The shaded cells indicate the percentage of households that fall within study guidelines. Number of Source: Table 19, Appendix C Exhibits IV-24a and IV-24b show the percentage of households that met these guidelines for each bedroom size for FY 2006 and FY 2007, respectively. Exhibit IV-24a Percentage of All Households in FY 2006 by Number of Bedrooms and Number of Household Members FY 2006 Number of Household Members Bedrooms 1 2 3 4 5 6 7 8+ 0 94% 3% 3% 1 91% 9% 1% <1% 2 22% 47% 23% 7% 1% 3 5% 10% 37% 31% 13% 4% 1% 4 1% 5% 17% 24% 19% 19% 8% 6% 5 4% 10% 14% 16% 10% 16% 31% <1% Source: Table 19a, Appendix C IV-24

-- -- -- -- -- -- -- -- -- -- -- -- -- -- IV. Findings Exhibit IV-24b Percentage of All Households in FY 2007 by Number of Bedrooms and Number of Household Members FY 2007 Number of Number of Household Members Bedrooms 1 2 3 4 5 6 7 8+ 0 96% 4% 1 90% 10% <1% <1% 2 25% 45% 20% 8% 1% 3 6% 11% 29% 30% 17% 5% 2% 1% 4 1% 2% 6% 32% 27% 12% 13% 5% 5 8% 6% 10% 16% 5% 9% 11% F. Project Staff Questionnaire Analysis The purpose of the Project Staff Questionnaire (PSQ) was to obtain information on project and PHA practices and procedures, in order to better understand how work is carried out in projects and PHAs, and to identify difficulties and potential areas for improvement. The executive directors or managers of the PHA/projects 7 in the FY 2007 study were surveyed, using a self- administered, paper questionnaire that examined in detail such topics as the number and type of PHA/project staff, training received by staff on how to conduct (re)certifications, communicating information about changes in HUD policies to the staff, quality control monitoring of work done by (re)certification staff, methods of obtaining household information, automation use when processing (re)certifications, various verification procedures employed in the process of (re)certifications, and difficulties in verifying tenants’ information. The results were analyzed separately for three major program types: Public Housing, PHA-administered Section 8, and owner-administered. A brief summary of the key findings from this analysis are presented below. A more detailed summary of the PSQ information is found in Appendix E. ♦ Number and Type of Staff. Overall, PHA/projects indicated an average of 56 units per staff member and 145 units per each full-time (re)certification staff member. However, there was a wide diversity of responses with respect to the ratio of staff per unit within, as well as between, different types of PHA/projects. PHA Section 8 projects reported the highest number of units per staff (106 units per staff member, on average) and highest number of units per full-time (re)certification staff (234 on average). Owner-administered projects had the lowest number of units per staff (30) and units per full-time (re)certification staff (80). Overall, 87 percent of PHA/projects (re)certification staff had over one year of experience, compared to 72 percent who had over 5 years of experience. 7 For purposes of this study, a PHA-administered Section 8 project is defined as a PHA/county combination representing the Section 8 Voucher households living in a particular county. IV-25

IV. Findings PHA/projects typically required at least a high school diploma/GED for new employees, with only 7 percent stating that no minimum education was required. Owner- administered projects were most likely not to require any education (11%), and PHA Section 8 projects were least likely (3%). ♦ New (Re)Certification Staff. About 38 percent of PHA/projects had new staff assigned to conduct (re)certifications in the past 12 months. These PHA/projects reported 2 new staff members being assigned to conduct (re)certifications in the past 12 months, on average. More PHA Section 8 projects assigned new staff to (re)certifications compared to Public Housing and owner-administered projects (54% versus 34% and 30%, respectively). PHA-administered Section 8 projects also assigned the most new staff to conduct (re)certifications (4 new staff, on average). Fewer owner-administered projects assigned new staff members to (re)certifications, compared to projects in the other programs. They also assigned the fewest new staff to (re)certifications (1 staff member, on average). ♦ New (Re)Certification Staff Training. PHA/projects provided on average 92 training hours to new (re)certification staff in the past 12 months. Three methods of training new staff were most prevalent – working one one-on-one with experienced staff; reading manuals, watching videos, or asking questions; and attending training sessions conducted by the supervisor. PHA-administered Section 8 projects provided the most hours of training (146 hours, on average). Owner-administered projects provided the fewest hours of training (60 hours, on average). ♦ Training of Experienced (Re)Certification Staff. About 77 percent of PHA/projects trained experienced staff in the past 12 months. This year, owner-administered projects provided more training to experienced staff, compared to projects in the other two programs. Among all projects, an average of 5 experienced staff members received an average of 33 training hours. Most PHA/projects usually or always trained using self- training, training sessions conducted by the supervisor, and training conducted by other experienced staff. ♦ Communicating Information about Changes in HUD Policies. PHA/projects used a variety of methods to communicate with staff about changes in HUD PHA/Owner policies affecting eligibility or rent calculations. One-on-one discussions between the managers and the staff was used most frequently, followed by staff meetings, distributing copies of HUD announcements to staff, and distributing a memo that described the changes and provided instructions for implementation. PHA/projects found answers to staff questions by referring to HUD PHA/owner memos or manuals, asking HUD field offices or other HUD staff, and asking questions at a HUD training session. Many PHA/projects reported that they had to figure out the answers themselves by conducting internal meetings, talks, or training with supervisors, directors, or other senior staff. An increasing number of PHA/projects used Internet and web-based resources and training to obtain answers to questions. ♦ Quality Control via Work Monitoring. Most PHA/projects conduct quality control monitoring of (re)certification work. PHA/projects typically have the supervisor conduct work monitoring, although an increasing number are turning to outside auditors to monitor their work. PHA/projects most frequently randomly spot checked a percent of IV-26

IV. Findings all cases, but other methods were also used, such as reviewing cases of new staff and checking cases on certain dates or times of the year. During the review process PHA/projects often or occasionally (71%) found mistakes in calculating rent, and missing or incomplete verifications of income (62%). The most commonly stated reason for errors was tenants providing inaccurate or incomplete information (87%). ♦ Issues in Conducting Tenant Interviews. The average duration of the typical initial certification interview was 36 minutes, while the average duration of a typical recertification interview was 30 minutes. PHA-administered Section 8 and owner- administered projects reported longer initial and recertification interviews, while Public Housing projects reported the shortest. PHA/projects overall were more likely to start the annual recertification process 3 to 6 months before the effective date (56%). Owner- administered projects were most likely. Fifty-six percent of PHA/projects overall were likely at 72 percent, compared to Public Housing projects and owner-administered projects. ♦ Using Computers and Software Programs. Almost all PHA/projects are using computers to support processing (re)certifications, as well as a wide variety of purposes. The number of PHA/projects using computers and software has been increasing. The most frequently reported uses for the computers were to calculate rent, print 50058/50059 forms, input verified information, and print letters to the tenants. Interestingly, one of the least frequently reported use of computers was to interview tenants and record answers. ♦ Use of PIC/TRACS. Ninety-seven percent of PHA/projects transmit 50058/50059 data via PIC/TRACS, and about 81 percent of all 50058/50059 data were transmitted to HUD via PIC/TRACS. Owner-administered projects transmitted only about a half of their 50058/50059 data to HUD directly and slightly less than a half through another agency or using other methods. Almost all PHA/projects kept copies of all 50058/50059 data in the household file, not just the most recent ones. ♦ Verification Procedures. More than 88 percent of PHA/projects (compared with 82% in FY 2006 and 85% in FY 2005) verify the components of tenant information at least occasionally, and more than 73 percent (compared with 72% in FY 2006 and 75% in FY 2005) always verify tenant information. (Re)certification staff are usually responsible for keeping track of verification requests and returns. Most PHA/projects keep track of outstanding verification in the tenant file. PHA/projects reported that it was often or sometimes difficult to verify sporadic, infrequent, or seasonal employment; sources of income other than employment; income from employment; and medical expenses. Most PHA/projects use various procedures to get verification information, including calling the third-party, sending letters to the third-party, calling the tenants, and sending letters to the tenants. When none of these procedures produced the verification information, most PHA/projects resorted to accepting other, less preferred verification information. When asked to name the causes of problems that emerged when obtaining complete verifications, the two major causes reported by PHA/projects were tenants providing incomplete or inaccurate information, and employers not responding to requests in a timely manner. TASS and EIV were most frequently used to verify Social Security/SSI benefits, employment income, and disability status and dual entitlement benefits. Most IV-27

IV. Findings PHA/projects also used other methods such as pay stubs, third party verification, and employer information to supplement EIV information. G. The Relationship between Characteristics and Practices, and Error Multivariate analysis was conducted to further examine gross rent error and different types of error that occurred in the certification/recertification process. The goal was to identify and estimate the relationship between tenant characteristics, project characteristics, and project practices and gross rent error. These analyses address study objectives 5, 6, 8, 12, and 13. A brief summary of the findings is provided below. A more detailed description of the analyses is found in Appendix F. To address Objective 5 (differences in error by program type), results from both bivariate tabulations and multiple regression analyses indicated that the Public Housing program had lower average gross rent error than the owner-administered program. In the multiple regression analysis, this difference remained statistically significant even after other project-level variables and tenant-level variables were held constant. The PHA-administered Section 8 program was found to have significantly higher gross error than the owner-administered program in bivariate tabulations, but this difference was not found to be statistically significant in the multiple regression analysis. The implication being that the descriptive differences identified in bivariate tabulations are further accounted for by tenant and project factors. Given similar tenant and project characteristics, in comparison with the owner-administered program, the Public Housing program’s lower gross error persisted; whereas the PHA-administered Section 8 program’s higher gross error disappeared. The latter finding is important as it suggests that tenant characteristics for the Section 8 program may have imposed greater difficulty in quality control processes as they apply to rent calculation. Note that the finding from the FY 2006 multiple regression analysis was somewhat different, where Public Housing program did not show a significant difference from the owner- administered program. The PHA-administered Section 8 program did have significantly higher gross error than the owner-administered program. The shifting pattern of error difference cross administrative type requires continued research to understand the underlying factors that lead to the differences. To address Objective 6 (impact on error by tenants vs. project/staff), this analysis found that, consistent with previous results, the impact of tenant characteristics on rent error was apparently greater than project characteristics and practices. As shown in variance partitioning with HLM unconditional model and the sequential OLS models, tenants’ high income and multiple sources of income and allowance items were related to both higher gross rent error and different types of error. These tenant variables should be seen as important indicators of risk for rent errors and be targeted by program intervention. To address Objective 8 (error concentration in program/project), this analysis found that, to a varying extent, a number of program/project features did relate to rent error and different types of error. Specifically, projects that retained experienced certification staff (over five years) had lower average rent error. This finding is compatible with what was found in the FY 2006 analysis where staff educational requirements were related to lower rent error, namely, staff quality is a critical factor in reducing errors. We suggest focusing more on staff quality data IV-28

IV. Findings collection and analysis in the future. One possibility is to collect additional staff information and job descriptions. The information would allow accurate comparison of staff quality across local providers in connection to the quality control results. To address Objective 12 (rent errors differentiated by using automated systems), the analysis provided evidence that computer application in certification operation helped reduce different types of processing error. Use of TASS or EIV, however, was not yet clear in consistently relating to rent error or different types of error. Again, more systematic data collection and analysis on computer application may help assess the impact of automation. For example, we may consider collecting data items on the respondents’ knowledge and skills specific to computer use in certification/verification. To address Objective 13 (other tenant or project characteristics related to errors), the analysis found that projects whose survey respondents reported perceived error causes tended to have lower chance of certain types of staff error. It is arguable that staff’s ability to report difficulties causing mistakes is equivalent to knowledge of the problems and hence potential ways to deal with them. However, far more information is needed to learn how project staff acquires information on different types of error in daily practice and the ways they identify and address risks of improper certification and payment determination. So far, we have revealed a very limited impact of project factors on rent error. It is vexing that, as organized activities of managing housing assistance, project variation did not seem highly meaningful with the statistical analysis. As suggested in earlier analyses, we will continue to improve our data collection tools to gather more relevant information on quality control. In depth understanding of the actual operation of the local providers is needed to more accurately conceptualize the organization structures differentiated by program types. For instance, we ponder collecting data to cover issues that are decision-maker specific (e.g., the owner, the managing contractor, or the PHA), rather than data in reference to a generic project. Furthermore, we note the limitation of statistical analysis, which typically can only describe broad patterns and identify strong, clear-cut relationships between causal and effect factors. To learn the complex and subtle differences in housing program operation that involve vastly different individuals and groups, qualitative research is needed. Quantitative data collection and analysis, in fact, can be benefited tremendously by rich, deep, personalized qualitative observations of the local housing offices daily activities. H. Rent Reasonableness The Housing Choice Voucher Program (HCVP) assists low-income families in obtaining housing in the private market. Public housing authorities are responsible for administering the program and ensuring that the rents paid for dwellings leased by participants in the HCVP are reasonable in comparison with rental units in the private, unassisted local market. High rents can waste government funds and inadvertently raise private market rents. HUD regulations require PHAs to conduct a rent reasonableness determination before units are leased, before rent increases are granted to owners, and when Fair Market Rents decrease by at least 5 percent. This analysis examines whether PHAs fulfilled the requirement for documenting rent reasonableness determinations, but does not investigate whether rents were in fact reasonable. IV-29

IV. Findings Methodology. Each of the 146 PHAs, administering the Section 8 Voucher program for households participating in the study, were asked to describe their standard rent reasonableness processes and provide copies of the forms used when determining rent reasonableness. This information was used to classify the methods used by PHAs to determine rent reasonableness. In addition, field interviewers were instructed to search the tenant files for each of the 789 voucher households in the tenant sample to locate the documents supporting the rent reasonableness certification. For new certifications (there were 115) 8 field interviewers searched the file for the initial rent reasonableness certification and recorded its date. For annual recertifications (674), field interviewers examined case files for evidence of when the current rent to owner became effective. If the rent became effective within the past two years, the case file was searched for a rent reasonableness certification and the date of certification. The owner’s rent certification on the Request for Tenancy Approval (RTA) form was considered a rent reasonableness certificate. Findings Pertaining To Rent Reasonableness Methods Used By PHAs. The most common method of determining rent reasonableness is the unit-to-unit comparison (see Exhibit IV-25). Fifty three percent of the housing authorities reported using this method as either the only method used or the predominant method. Twenty four percent reported using unit-to-unit methodology as a component in combination with other methods. The unit-to-unit method is similar to the standard real estate appraisal technique of comparing a unit to similar private, unassisted units. Rent amounts are sometimes modified for differences in unit characteristics, such as size, age, amenities, housing services, maintenance, and utilities. Exhibit IV-25a PHAs by Predominant Rent Reasonableness Method (unweighted) 2006 2006 2007 2007 Method Number Percent Number Percent Unit-to-Unit Comparison 74 56% 78 53% Unit-to-Market Comparison 11 8% 12 8% Point System 19 14% 17 12% Other or Rent Control 9 7% 9 6% No Single Predominant Method 17 13% 26 18% No information 2 2% 4 3% Total 132 100% 146 100% The unit-to-market comparison approach estimates the average and/or range of “market” rents for units with similar characteristics in the private, unassisted market. Valuation adjustments are based on typical units in the private market. Eight percent of housing authorities reported using this method solely or primarily. However, 27 percent of the PHAs used the unit-to-market method in combination with other methods. Twelve percent of housing authorities indicated that their primary method of making rent reasonableness determinations was based on a point system; 8 In FY 2007, portability move-ins (19 cases) were classified as annual recertifications. In FY 2006 they were categorized as new admissions. IV-30

IV. Findings Thirteen percent reported using the point system in combination with other methods. Using this system, units are assigned points based on their condition and attributes and comparisons are made to unassisted units. Eighteen percent of the PHAs used a combination of methods equally, meaning no predominant method was identified. Six percent of PHAs used some other method to determine rent reasonableness or the rents for their properties were restricted by rent control. The frequency of various combinations of rent reasonableness methodologies are addressed in Exhibit IV-25b. Exhibit IV-25b PHAs using Rent Reasonableness Method Combinations (unweighted) 100 % Method Predominant Method Equivalent Method Lesser Component Unit-to-Unit Total times cited = 113 49 29 14 21 Unit-to-Unit and Unit-to-Market 12 5 11 Unit-to-Unit and Point Ranking 8 5 6 Unit-to-Unit and Professional Judgment 20 4 15 Unit-to-Market Total times cited = 51 10 2 7 32 Unit –to-Market and Unit-to-Unit 1 6 27 Unit-to-Market and Point Ranking 0 1 0 Unit-to-Market and Professional Judgment 1 0 18 Point Ranking Total times cited = 36 15 2 2 17 Point Ranking and Unit-to-Unit 1 1 17 Point Ranking and Unit-to-Market 0 1 12 Point Ranking and Professional Judgment 1 0 11 Other and Rent Control Total times cited = 12 0 9 0 3 Other and Unit-to-Unit 5 0 3 Other and Unit-to-Market 5 0 3 IV-31

IV. Findings Each methodology is considered based on its proportion in the mix of methods. The total times cited refers to the number of PHA’s which used the rent reasonableness method to any extent. When the mix of methods sum to a number higher than the times cited in the column subheading, it indicates that there were more than two methods involved. For example, unit-to-unit methodology is cited by one hundred thirteen PHA’s and is used as a sole method of rent reasonableness determination in forty nine Section 8 voucher programs. It is the predominant method in twenty nine voucher programs. In twenty of the twenty nine programs, project staff report using professional judgment as a component of their rent reasonableness methodology. Twelve of these twenty nine voucher programs include unit-to-market analysis and four include point ranking. Without identifying specific mixes and proportions, one can see that the twenty nine voucher programs which primarily use the unit-to-unit method often include a mix of two or more other methods for determining rent reasonableness with professional judgment being the most frequent associated method. Findings Pertaining To Rent Reasonableness Documentation Found In Tenant Files for New Admissions. In FY 2007, 71 percent of new admission files contained rent reasonableness documents compared to 88 percent in FY 2006 (see Exhibit IV-26a) 9 . The absence of documentation does not necessarily indicate a determination was not completed, only that it was not properly documented. Of those files that had documentation, 61 percent contained a statement signed by the PHA staff certifying that the rent is reasonable (see Exhibit IV-26b). Exhibit IV-26a Rent Reasonableness Documentation for New Admissions Status 80% 88% 71% 20% 12% 29% i 81% 68% 61% 4% 10% 11% 11% 16% 24% 4% 6% 4% 2005 2006 2007 Determination Documented No Determination Documented Total 100% 100% 100% Exhibit IV-26b Type of Rent Reasonableness Documentation for New Admissions Type A signed statement certify ng that the rent is reasonable Comparable units documented by the property owner in section 12a of HUD 52517 Comparable units documented on other documents Any other reference to rent reasonableness 2005 2006 2007 Total 100% 100% 100% 9 As noted previously, in prior studies portability move-ins were categorized as new admissions rather than annual recertifications. Therefore we calculated the percent of households with rent reasonableness documents when new admissions and portability move-ins households are classed together. The result indicates 76 percent of the files contained rent reasonableness documents, a significant decrease from FY 2006. IV-32

IV. Findings HUD requires that rent reasonableness determinations be conducted before signing the contract and lease. The timeliness of the rent reasonableness determination was evaluated by comparing the lease date with the rent reasonable certification date in the case file. Exhibit IV-27 provides a summary of how the date of the rent reasonableness determination relates to the initial lease date for those households where reference to the rent reasonableness determination was found in the file. If the lease effective date occurred before the determination, the rent reasonableness determination had no impact on the rent charged. The percent of rent reasonable determinations made after the rent had been established as part of the initial lease agreement increased from FY 2006 (11 percent) to FY 2007 (15 percent). Exhibit IV-27 Timing of Most Recent Rent Reasonableness Determination—New Admissions 2% 3% 5% Determination-Certification Chronology 2005 2007 More than 4 months before lease date Up to 4 months before lease date 85% 75% 77% After lease date—up to 2 months 3% 9% 10% After lease date—greater than 2 months 4% 2% 5% Date missing 6% 12% 3% Total 100% 100% 100% Findings Pertaining To Rent Reasonableness Documentation Found In Tenant Files for Annual Recertifications. Annual recertifications require rent reasonableness documents only when owners increase rental rates. We examined case files to determine when the current rent to owner first became effective. The case file was searched for the rent reasonableness determination when rent reasonableness determinations were performed in the previous two years. In FY 2007, about 65 percent of these case files had certified rent reasonableness documents within the past two years compared to 69 percent in FY 2006 (see Exhibit IV-28a). Exhibit IV-28a Rent Reasonableness Documentation for Annual Recertifications Status 2005 2006 2007 Determination Documented 65% 69% 65% No Determination Documented 35% 31% 35% Total 100% 100 100% Type of reference to rent reasonableness documentation was recorded for households where documentation of the rent reasonableness determination was found. Of the files that had documentation within the last two years, 63 percent contained a statement signed by the PHA staff certifying that the rent is reasonable (see Exhibit IV-28b). IV-33

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

IV. Findings Exhibit IV-28b Type of Rent Reasonableness Documentation for Annual Recertifications Where Documentation of the Rent Reasonableness Determination Was Found Type 2006 2007 A signed statement certifying that the rent is reasonable 68% 63% Comparable units documented by the property owner in section 12a of HUD 52517 4% 10% Comparable units documented on other documents 16% 22% 12% 5% Any other reference to rent reasonableness Total 100% The current rents to owner in the lease agreements were compared with the dates of the rent reasonable documents. If the lease effective date occurred before the determination, the rent reasonableness determination had no impact on the rent charged. In FY 2007, about 9 percent of the rent reasonable determinations were made after rents had been established, compared with 22 percent in FY 2006 (see Exhibit IV-29). Exhibit IV-29 11% 8% 15% Timing of Most Recent Rent Reasonableness Determination—Annual Recertifications Determination-Certification Chronology 2005 2006 2007 More than 4 months before lease date Up to 4 months before lease date 42% 47% 42% After lease date—up to 2 months 3% 3% 2% After lease date—greater than 2 months 13% 19% 7% Date missing 31% 23% 34% Total 100% 100% 100% Conclusion. PHAs are not fully documenting rent reasonableness determinations as required by HUD regulations, and a large percentage of existing rent determinations have been made on the basis of less formal means of evaluating rents. These findings may be partially attributable to the PIH notice issued May 16, 2003 (notice PIH 2003-12) that supports a more streamlined rent reasonable process. For example, a PHA need not consider all nine criteria cited in 24 CFR 982.507(b) to fully comply with the regulation. PIH 2003-12 also asserts that “each PHA should use appropriate and practical procedures for determining rental values in the local market.” This statement may also be intended to justify less formal methods of rent determination. I. Utility Allowance Analysis As part of the FY 2007 HUDQC study, two separate analyses were conducted of the utility allowances (UA) provided to households assisted through PHA-administered Section 8 Voucher programs. The first analysis focused on whether there was documentation in the tenant file indicating how the utility allowance amount used in rent determination was calculated, and whether those documents were used correctly in calculating the utility allowance amounts. The IV-34

IV. Findings second analysis focused on identifying discrepancies between the utility allowance on the 50058 form (AC utility allowance), and the utility allowance determined by using the appropriate Utility Allowance Schedule provided by the PHA staff (QC utility allowance) that often varied by unit type, effective date of (re)certification and location within a county. To support these analyses, PHAs were asked to provide utility allowance schedules used for actions effective in FY 2007 as well as details regarding how the utility allowance was calculated. Then the field interviewers were asked to copy documents showing calculation of utility allowances found in tenant files at the PHA office. Documentation of Utilities and Utility Allowance Values. One-hundred and forty-six PHA administered Section 8 Housing Choice Voucher programs, administered by 133 housing authorities (several of which administered multiple Voucher programs in different counties) participated in the 2007 HUDQC study. According to information provided by those PHAs more than half (51%) of the programs used HUD Form 52517 (Request for Tenancy Approval) as the official source for identifying the utilities for which the households were responsible; and majority (73%) of the programs used HUD Form 52667 (Schedule of Allowances for Tenant Furnished Utilities) to calculate the value of the utilities paid by the tenants. Exhibit IV-30a provides the information on the type of documents used as the official source for identifying utilities for which the households were responsible, as well as the type of documents used to calculate the value of the utilities paid by the tenants. Exhibit IV-30a Types of Documents Used by PHA to Identify Utilities and Calculate the Utility Allowance Value Type of Document Used For Identifying Utilities For Calculating the Utility Allowance Value Number of Percent of Section 8 Section 8 Programs Programs Number of Percent of Section 8 Section 8 Programs programs HUD Form 52517 (Tenancy Approval) 75 51% 8 6% HUD Form 52641 (HAP Contract) 13 9% 2 1% HUD Form 52667 (Allowances Schedule) 17 12% 107 73% Other (Lease, Reports, Comparisons etc) 20 14% 25 17% Various combinations of above 21 14% 4 3% Total 146 100% 146 100% Data in this exhibit are not weighted Seven hundred and eighty-nine households, assisted through PHA-administered Section 8 Housing Choice Voucher Programs, participated in this study. Field interviewers were able to locate worksheets or other documents indicating how the utility allowance was calculated for 749 households (95%). Discrepancies between the AC and QC Utility Allowances. The analysis related to discrepancies between the AC and QC utility allowance was conducted in two steps. First the utility allowance value from the 50058 (AC) was compared to the utility allowance amount on the worksheet found in the household files (WS). Second, the utility allowance value from the 50058 (AC) was compared to the QC allowance amount. IV-35

IV. Findings The QC utility allowance was calculated also in two steps. In the first step, the utilities for which the tenants were responsible were identified by using documents – usually PHA utility allowance Worksheets – found in tenant files that indicated those specific utilities. In the second step, the identified household specific utilities were mapped onto the utility allowance schedule, and the total summed up to determine the QC allowance amount. Comparison of AC and WS Utility Allowances. The utility allowance amount for the 749 households from the 50058 (AC) was matched with amount on the utility allowance worksheet obtained from the tenant files (WS). For 90 percent of the households (672 units) the 50058 utility allowance amount (AC) matched with the Worksheet amount (WS). This included 71 households that did not have any utility expenses because either they were included in the rent or the owner paid all utilities. For three percent of the households, the worksheet provided was for the incorrect period of time. Hence, we could not determine whether the utility allowance amount used in the rent calculation was correct. In the remaining 7 percent of the households there were discrepancies between the amount on the worksheet (WS) and the AC amount. Exhibit IV- 30b lists the description of the findings from the comparison between the utility allowance listed on the 50058 (AC) and the amount on the worksheets (WS) found in tenant files. Exhibit IV-30b Utility Allowance Worksheet (WS) vs. 50058 (AC) Utility Allowance Comparison Findings Number Percent Outcome 672 90% 50058 (AC) amount matched with Worksheet (WS) amount 25 3% Worksheet in file for incorrect period of time 25 3% Discrepancy due to math error or other clerical errors 27 4% Discrepancy – Unable to determine reasons 749 100% Total Data in this exhibit are not weighted. Comparison of AC and QC Utility Allowances. The utility allowance amount on the 50058 form (AC) was matched with the QC utility allowance amount. As in the previous comparison, we were unable to calculate the QC utility allowance in 5 percent (41 households) of the cases because their worksheet was not available and consequently the specific utilities the household paid for could not be identified. Furthermore, we were unable to calculate the QC utility allowance in 2 percent of the cases because the worksheets in the files did not include specific utilities or other critical information needed for QC allowance calculation; and in another 2 percent because the appropriate Utility Allowance Schedule was not available. Exhibit IV-30c below differentiates between the cases whose QC allowance amount was able to be calculated and lists the reasons and number of cases whose QC utility allowance amount was not able to be calculated. IV-36

______________________ IV. Findings Number Percent Outcome Exhibit IV-30c Availability of all Information to Enable QC Utility Allowance Calculation QC UA amount calculated 721 91 % Appropriate Worksheet and Schedule available Yes 41 5% UA Worksheet or other comparable document not available No 15 2% Appropriate UA Schedule not available No 12 2% Worksheet was missing critical information No 789 Data in this exhibi 100% Total t are not weighted. For the 721 cases whose QC utility allowance amounts were calculated, the QC utility allowance was compared to the AC utility allowance amounts. In 88 percent of those households, the AC and QC utility allowance values matched. The remaining (discrepant) 12 percent were categorized into 4 broad categories. Three of the 4 categories fall under administrative errors, whereas for the remaining cases, we were unable to determine the reason for the discrepancy. Exhibit IV-30d below presents the findings from this analysis. Exhibit IV-30d QC vs. AC Utility Allowance Comparison Findings Number Percent Outcome 639 88% QC UA matched amount on 50058 14 2% Discrepancy due to difference in number of bedrooms used 20 3% Discrepancy due to math error / transfer error Discrepancy – incorrect schedule used (schedule for improper effective date 13 2% or locality rates used) 35 5% Discrepancy – Unable to determine reasons 721 100% Total Data in this exhibit are not weighted. Note: When calculating the QC rent, the AC utility allowance amount, and not the QC allowance amount was used. J. Payment Standard Analysis As part of the FY 2007 HUDQC study, a special analysis was conducted to determine if PHAs are using the correct payment standards. This special analysis was conducted independently of the rent calculation error findings presented elsewhere in this chapter, and the payment standard Analysis did not affect the rent calculation determinations. This analysis consisted of two parts. First, the payment standard on the 50058 form was compared to the payment standard schedules IV-37

IV. Findings provided by the PHA. Second, the payment standard on the 50058 form was compared to the Fair Market Rent for the appropriate geographical area. The findings from these two comparisons are presented below. Background. Payment Standards are used in the Section 8 Voucher Program when determining the tenant’s portion of the rent-to-owner. They must be kept current and set between 90 and 110 percent of the Fair Market Rent (FMR). If a PHA does not ensure that their payment standards are within this range, or they misunderstand how new FMRs affect their payment standards, errors in tenant rent determinations will result. PHAs may apply payment standards incorrectly resulting in errors in tenant rents. A PHA may have several payment standard areas with complex borders, sometimes making it difficult to select the correct payment standard for any given address within the jurisdiction. PHAs may also err by applying the family-size payment standard (the size authorized for the family as shown on the voucher) in lieu of the lesser of either the family-size payment standard or the payment standard for the unit size (number of bedrooms in the unit). Other potential areas for error include whether a PHA has been authorized to use FMRs based on the 50 th percentile of the rents in the area; whether the PHA has been authorized to use Success Rate payment standards based on the 50 th percentile of rents; and whether the PHA continues to be eligible for these higher subsidy standards. Another complication allows PHAs to change the payment standard only at the time of the annual recertification or before moving to a new address. Thus, even if a change in the family composition requires an interim recertification with several family members moving in or out, the payment standard used in determining the rent should not be changed at the interim recertification. The complexity of the payment standard guidelines increases errors, but most of the errors found were not due to these complex guidelines. Comparison of the Payment Standard on the 50058 Form to the Payment Standard Schedules Provided by the PHA. The first analysis consisted of comparing the payment standard on the 50058 form (AC) to the payment standard schedule (QC) provided by the PHA. For all voucher households in the study, the appropriate QC payment standard was selected and compared to the AC payment standard. The selection of the QC payment standard from the schedules provided by the PHA was based on: ♦ the lower of either the number of bedrooms in the unit, or the number of authorized bedrooms for the household on the voucher, ♦ the effective date of action, and ♦ the determination and application of any exception listed on the information provided by the PHA staff. For every household where the AC and QC payment standard did not match, a call was placed to the PHA staff for clarification and, if appropriate, to gather payment standard schedules for previous years. Through the calls, often other complications were discovered and taken into consideration when selecting the QC payment standard. The types of complications included: ♦ A decrease in the payment standards for units, requiring the PHA and Macro to use the previous (higher) payment standard for the first recertification after the decrease. Many IV-38

IV. Findings PHAs only sent the payment standards for a specific time period. Calls were made to get the historic payment standard Schedules. ♦ Households that were granted exceptions for special circumstances such as living in a house with additional amenities or setting the payment standard to the Gross Rent for Enhanced Vouchers. ♦ Housing Authorities using higher payment standards for Exception Rent Areas. ♦ Housing Authorities using payment standards from a previous Housing Authority for port-in households understanding the rates would be adjusted at the next annual re examination. There were 789 Housing Choice Voucher households in the study. For the majority (92%) of the households, the AC payment standard matched the QC payment standard. However, there were 64 households (8%) with discrepant payment standards. Forty (63%) of these discrepant households were elderly or disabled households. Elderly and disabled households are identified separately because they are often entitled to individual exemptions to the payment standard rules. Fifty-one (80%) of the total discrepancies were attributable to one of six common reasons. The most typical reason for a discrepancy between the AC and QC payment standard was that the Gross Rent was used in lieu of the payment standard. Also, the use of either the incorrect number of bedrooms or the use of the incorrect payment standard schedule (i.e., schedule was for a different geographic area or time period) accounted for a cumulative 32 percent of the discrepancies found. In addition, the payment standard used by the PHA was not available for 8 households (12%). Section 12 of the 50058 was incomplete or missing from the tenant file, and the PHA staff were not able to provide the payment standard for that case. These households were considered as having a discrepant payment standard and categorized as having a payment standard that was below 90 percent of the Fair Market Rent. Exhibit IV-31 below summarizes the number and percent of households where the QC and AC payment standard did not match by reason. IV-39

IV. Findings Exhibit IV-31 Number and Percent of Households with Payment Standard Discrepancies Number of (Elderly /Disabled) Number of Percent of /Disabled) 1 0 1 % 13 6 30 % 7 3 16 % 5 5 16 % 0 3 5 % 7 1 12 % 7 6 20 % 40 24 The FMR. Exhibit IV-32 Fair Market Rent % 3 405 5 1.0 % 16 353 7 2.9 % i 19 758 12 3.9 % Households Reason Households (Non-Elderly Households Fair Market Rent was Used Instead of the Payment Standard Gross Rent was Used Instead of the Payment Standard Incorrect Number of Bedrooms was Used Incorrect Payment Standard Schedule was Used Overdue Recertification; Payment Standard Amount Did Not Match the FY 2007 Payment Standard Section 12 of the 50058 was Incomplete or Missing Other Reasons – Typo, Enhanced Rate Incorrectly Applied Total 100 % Data provided in this exhibit are not weighted. Comparison of the Payment Standard on the 50058 Form to the Fair Market Rent for the Appropriate Geographic Area. The second analysis consisted of comparing the payment standard on the 50058 form (AC) to the Fair Market Rents (FMR) for the appropriate area. payment standard for 758 of the households (96%) fell within the 90 to 110 percent FMR band; 19 households (2.4%) used an amount that was less than 90 percent of the FMR; and 12 households (1.5%) exceeded 110 percent of the FMR. Exhibit IV-32 below summarizes the number and percent of households by the relationship of the payment standard to the acceptable Percentage of Households Meeting Payment Standard Requirements Under 90% 90–110 Over 110% Cases Outside the 90– 110% Band Non-Elderly or Disabled Elderly or Disabled Payment Standard Compared w th Fair Market Rent The analysis of the households that fell outside the 90 to 110 percent FMR band indicated that 27 households (87%) fell outside of the 90 to 110 percent band of the FMR for six general reasons; the household’s voucher was administered by a PHA that was granted an exemption by HUD; the household received an exemption because of elderly or disability status; Gross Rent was used instead of the established payment standard; the incorrect number of bedrooms was used when determining the appropriate payment standard, the Recertification was overdue and the AC payment standard fell outside of the updated FMR 90 to 110 percent band; or Section 12 of the IV-40

IV. Findings 50058 was incomplete or missing. Exhibit IV-33 summarizes the number and percent of households that fall outside the 90 to 110 percent FMR band by categories. With 26 households not ratified by any exemptions, two percent of all Housing Choice Voucher households had a payment standard less than 90 percent of the FMR, and one percent had a payment standard exceeding 110 percent of the FMR. Therefore, a total of three percent of households did not meet HUD’s payment standard requirements. Exhibit IV-33 Details of Cases Falling outside the 90-110% of FMR Band Fair Market Rent Percent of Cases Outside the 90–110% Under 90% Over 110% Band Households Assisted by a PHA Granted an Exemption by

HUD 0 1 3 %

Households with Elderly or Disabled Members Receiving

an Exemption 2 2 13 %

Discrepancy Due to Incorrect Number of Bedrooms 4 4 26 % Discrepancy Due to Use of Gross Rent Instead of

Payment Standard 2 3 16 %

Discrepancy Due to Overdue Recertification 1 0 3 % Section 12 of the 50058 form was Incomplete or Missing 8 0 26 % Other Reason – No Explanation Provided by PHA Staff

during Follow-up or Typo 2 2 13 %

Total Number of Cases outside 90-110% FMR Band 19 12 100 % Data provided in this exhibit are not weighted. Comparison of the FY 2006 to the FY 2007 Payment Standard Analysis Results. The same payment standard Analysis was conducted for the FY 2006 study. Of the 795 Housing Choice Voucher households in the FY 2006 study, the AC and the QC payment standard matched for 609 (77%) households. Additionally, 121 (15.2%) households had payment standards that did not fall within the 90 to 110 percent Fair Market Rent band. Of those 121 households, 86 cases were not granted any exemptions. Therefore, a total of 11 percent of the Housing Choice Voucher households included in the FY 2006 did not meet HUD’s payment standard requirements. For cases in the FY 2006 payment standard Analysis where the AC and the QC payment standard did not match, 42 cases (23%) incorrectly applied a new, lower payment standard at the first recertification after a decrease as opposed to using the lower payment standard at the second recertification of the decrease, as outlined in the HUD Policy Form 24 CFR 982.505. For the FY IV-41

IV. Findings 2007 analysis, no cases were discovered where a new, lower payment standard was incorrectly applied in the first recertification after a decrease. Exhibit IV-34 below summarizes the results from the FY 2006 and FY 2007 Payment Standard Analysis. Exhibit IV-34 Comparison of the FY 2006 to FY 2007 Payment Standard Analysis FY 2006 FY 2007 Number Percent Number Percent Housing Choice Voucher Sample 795 789 Households where the AC and QC Payment Standard Did

Not Match 186 23 % 64 8 %

Households where the AC Payment Standard Did Not Meet

the 90 to 110 Percent of FMR Threshold 121 15 % 31 4 %

Households that Were Not Exempt from the 90 to 110

Percent of FMR Threshold – Did Not Meet HUD’s Payment

Standard Requirements 86 11 % 26 3 %

Data provided in this exhibit are not weighted. K. PIC/TRACS Analysis The households included in this study were matched against the PIC/TRACS data files using identifying information (a combination of the Social Security Number, name, and date of birth) for the head of each household. Because this study covers FY 2007, an attempt was made to use historical PIC/TRACS files to identify the 50058/50059 data for the specific effective date and type of action for which study data were collected. PIC/TRACS data were received for any household (in the study sample) that were in the historical databases used by HUD analysts even if the specific study effective date and type of action did not match. When matching on the specific study effective date and type of action, only 1613 of the 2404 households in the study were represented. Therefore, most of the PIC/TRACS analysis for this report was based on the broader match (PIC/TRACS data received for any household in the study sample). Using these criteria, PIC records were found for 94 percent of the households in PHA-administered projects; TRACS records were found for 97 percent of the households in owner-administered projects. Of the 2404 households sampled, 2289 households (or 95%) were matched against PIC/TRACS. Analysis was conducted to compare the average dollars in gross rent error for households that matched PIC/TRACS with those that did not. Exhibit IV-34a provides the percentage of households in each of the three program types by presence or absence in PIC/TRACS, and the average dollars in error based on all households in the study. Exhibit IV-34b provides the same information, but uses only households with rent error as its base. These exhibits demonstrate that proportionally an equal number of households in error matched against PIC/TRACS data. IV-42

IV. Findings Exhibit IV-34a PIC/TRACS Data by Program Type and Average Gross Dollars in Error for all Households PIC/TRACS PRESENT PIC/TRACS ABSENT Percent of Average Dollars Percent of Average Dollars Administration Type Households in Error Households in Error Public Housing 94% $12 6% $32 $19 6% $23 PHA-Administered Section 8 94%

$17 6% $26

Total PHA-Administered 94% $13 3% $12 Total Owner-Administered 97% 95% $15 5% $23 Percent of in Error Percent of in Error 93% $39 7% $87 94% $50 6% $59 Total PHA-Administered 94% $47 6% $68 98% $35 3% $37 95% $43 5% $63 Total As presented in Exhibit IV-34b the average dollars in error for households in error is higher for households when PIC/TRACS data is absent in PHA-administered programs, but about the same in owner-administered programs. Exhibit IV-34b PIC/TRACS Data by Program Type and Average Gross Dollars in Error for Households in Error PIC/TRACS PRESENT PIC/TRACS ABSENT Administration Type Households Average Dollars Households Average Dollars Public Housing PHA-Administered Section 8 Total Owner-Administered Total IV-43

IV. Findings Exhibit IV-35 presents the percentage of households and average dollars in error for households matched/not-matched with PIC/TRACS by payment type. Essentially the same proportion of households with and without matched PIC/TRACS data had proper payments. Exhibit IV-35 Average Gross Dollars in Error by Payment Type and PIC/TRACS Data PIC/TRACS PRESENT PIC/TRACS ABSENT Percent of Average Dollars Percent of Average Dollars Payment Type Households in Error 1 Households in Error 1 18% $56 21% $80 18% $29 16% $39 64% n/a 63% n/a $15 $23 1 l Exhibit IV-36 -$5 -$9 $12 $32 PHA-Administered Section 8 -$5 -$15 $19 $23 Total PHA-Administered -$5 -$13 $17 $26 -$4 -$3 $13 $12 -$5 -$11 $15 $23 Underpayment Overpayment Proper Payment Total 100% 100% Average dol ar error per under- and overpayment subgroups. Exhibit IV-36 examines net and gross errors by program type and matched PIC/TRACS data. This table provides no new insights about the impact of matching PIC/TRACS data but highlights the importance of reviewing both gross and net rent errors. Average Net and Gross Dollars in Error by Administration Type and PIC/TRACS Data for all Households Average Net Rent Error Average Gross Rent Error PIC/TRACS PIC/TRACS PIC/TRACS PIC/TRACS Administration Type Present Absent Present Absent Public Housing Total Owner-Administered Total For households where PIC/TRACS data matched on specific study effective date and type of action, further analysis was conducted to determine if certain key variables matched. The key variables included gross income, net income, total tenant payment, and tenant rent . Exhibit IV-37 provides the percentage of households where the data gathered through the QC process matched that in PIC/TRACS. IV-44

IV. Findings Exhibit IV-37 PIC PIC PIC PIC No Match 1.2% 2.2% 2.0% 2.6% 8.4% 12.8% 27.1% Match 98.8% 97.8% 98.0% 97.4% 97.1% 91.6% 87.2% 72.9% Percentage of Matched and Non-Matched Dollar Amounts for Key Variables Matching Variables from the 50058/50059 Form and PIC/TRACS Total Tenant Gross Income Net Income Tenant Rent Payment Match Status TRACS TRACS TRACS TRACS 2.9% Total 100% 100% 100% 100% 100% 100% 100% 100% IV-45

V. Recommendations This section discusses recommended changes to the study that will improve the data collection process or the quality of the data used in the analysis, as well as policy actions that could be taken to reduce error. Section A discusses changes to the quality control process itself. Section B addresses policy recommendations. Note that these recommendations have not changed significantly from recommendations made in previous final reports. However, if further reduction in error is desired, it continues to be important to learn more about local policies and procedures that impact error, and methods of changing those processes to reduce error. A. Modifying the Quality Control Process The current methodology used by Macro to conduct its quality control study is based on the successes and failures of previous studies, and meets the established objectives. However, there are some recommendations that would be helpful for expanding the utility of data products as well as improving the overall efficiency of ongoing quality control studies. These include the following: 1) Continue the HUD quality control studies as a regular, ongoing effort to monitor and manage HUD rent determination processes. Ongoing evaluation of the subsidy programs administered by HUD is essential to the management of those programs. The primary goal of the quality control studies is to measure rent errors. However, these studies also give HUD the opportunity to learn more about alternatives to reducing rent errors, and better management of current and changing conditions at PHAs/projects. Annual evaluations facilitate more accurate cross-year comparisons of rent errors. They also allow for data collection and analysis staff to develop specific expertise with HUD policy areas, and develop tailored solutions for improving data quality. Further, other HUD-related topics could be investigated (e.g., the changing demographics of HUD tenants) and piggybacked on to the rent error data collection processes. Data collected through the quality control studies provides detail not available through other HUD sources (e.g., PIC/TRACS) that could be used to track such trends as the extent to which income and expense items are verified, or the number of sources of employment income received by a particular household or household member. 2) Gather information to document the outcome of the HUD quality control studies. Overall, the HUDQC studies indicate that both the percent of errors and dollars associated with those errors have decreased in the last six years. However, there is no information on changes in tenant behavior related to the identification and reduction of error. One might want to assume that reducing error should save HUD money. However, because the housing programs managed by HUD are not entitlement programs (meaning not everyone who is eligible for the program is entitled to benefits), as soon as an ineligible household is removed from the roles, another household takes that household’s place. The subsidy for the replacement household could be even higher than the subsidy for the previously subsidized household. The existing quality control studies identify the dollars associated with error, but do not identify an overall reduction in subsidy dollars. To really understand the overall impact of the quality control studies on subsidy funding, additional V-1

V. Recommendations information is needed regarding both the tenants receiving the subsidies and the PHA/projects administering the housing benefits. 3) Expand contractor access to verification obtained through inter-agency agreements. Despite increasing rates of third-party verification, a large proportion of tenant income and expenses are not being verified. This is especially important given the study results indicate a significant relationship between third-party verification of certain types of income and rent errors. During the current study, household-level information was used to match sample household members with Social Security data files through the Enterprise Income Verification (EIV) system. Through this electronic match, verification was obtained for most sample household members’ Social Security and Supplemental Security Income (SSA/SSI) benefits. However, there were many household members where a match between the study electronic files and the SSA/SSI electronic files was not found when expected and other situations where irresolvable discrepancies were identified. If Macro as the contractor for the HUDQC study could have access to the SSA/SSI database, these mismatches and discrepancies could be investigated further. 4) Collect more information regarding PHA/project policies and practices. Each PHA establishes its own policies, procedures, and forms for collecting the information that is ultimately used to calculate tenant rent. The differentiation in these practices should have some (possibly major) impact on the rent error, yet the analysis of the project practices and characteristics collected in the Project Staff Questionnaire designed for this study does not demonstrate the expected impact. Therefore, we recommend that focus groups and cognitive interviewing be used to identify additional PHA/project level factors that may impact error. This additional information could be used to revise the Project Staff Questionnaire to include questions focused on the specific practices expected to influence errors. As the data are already starting to reflect, as rent error decreases it will become increasingly difficult for HUD and PHA/project staff to continue to make changes that will reduce the error. Analysis of more detailed project-level data will assist in this process. 5) Continue to investigate PIC/TRACS data for sampling and other purposes. Ideally PIC/TRACS data would be used to select the quality control sample, and provide the actual data used by the PHA/project staff when calculating rent (in place of abstracting 50058/50059 Form data from the tenant file). However, the most recent match of the study sample households with PIC/TRACS data indicated that only 95 percent of the sample households are included in the PIC/TRACS databases. This is an improvement over the findings from the match using the FY 2006 study sample households which indicated that 83 percent of the sample households were included in the PIC/TRACS databases. However, this information continues to confirm that consideration should not be given to using these data for selecting the household sample until there is some assurance that the databases are all inclusive. Even if it is determined that PIC/TRACS data include all households receiving assistance, using the PIC/TRACS data for selecting the household sample may not be appropriate unless the data are available for the specific period of time covered by the study. V-2

V. Recommendations 6) Continue to expand existing computer systems and processes that further automate data collection, processing, and reporting functions. Most of the data for the current study were collected using an automated data collection system. This system continues to be enhanced for each study so it now, not only simplifies the data collection process and reduces the number of data collection errors, but also allows for review of the data at Macro headquarters as the data are being collected. While the existing systems work well, there are additional improvements that can be made to the data collection software, the field monitoring software, and the processing and tracking of third-party verifications. The next series of improvements should be aimed at increasing the amount of third-party verification obtained by the contractor. Expanding and investing in better automated systems will yield large dividends in terms of costs, time required to collect and process data, as well as the breadth, depth, and quality of data. B. Policy Actions This study was not designed to provide recommendations regarding basic program objectives and policies. However, the findings from this study suggest that some major procedural changes should be considered when establishing and revising policy. Again, the recommendations in this section remain essentially the same. While HUD has initiated several initiatives in the last few years, the errors associated with the programs included in this study are no longer decreasing. Additional action is needed. The suggestions below are examples of the type of actions that need to be taken. Overall PHA/projects must be held accountable for their work, but HUD must provide the tools needed to accomplish the work accurately. 1) HUD should continue to require both PHAs and owners to use the information available through the Department of Health and Human Services’ “New Hires” income matching database. The majority of subsidy overpayment errors are associated with earned income, and a large majority of tenant income underreporting also relates to earned income. The “New Hires” income matching database provides the opportunity to correct errors associated with reported and unreported income. However, our experience working with the “New Hires” data indicates that caution needs to be taken when using the information provided by the database. The data are extremely helpful in identifying unreported sources of income. However, the data are not current and often contain errors. Great care needs to be taken when using these data to insure that income is only counted when it is clear that it is received by the tenant and not simply because it is identified through the New Hires database. 2) HUD should continue expanding support of the occupancy function and conducting outreach campaigns to PHAs and owners informing them of the Department's occupancy- related resources. Provision of detailed, current occupancy handbooks is essential in addition to providing a mechanism for answering questions as they surface. Specifically, HUD should develop a nationwide, consistent, reliable approach for providing guidance and support to both PHAs and owners. It is also critical that there be a close link between the team that responds to field concerns and the staff responsible for writing HUD notices and guidance documents. The team responding to field questions and concerns knows what the problems are that face the field. These problems should be the subject of the guidance that comes from HUD. V-3

V. Recommendations 3) HUD should provide the PHA/owners with the forms, training, and other tools needed to determine rent correctly. Rent calculation error could be reduced if HUD would provide structured forms for interviewing tenants, obtaining verifications, and calculating rent. Ideally, these tools would be provided in the form of computer-assisted interview software that minimizes the number of questions that need to be asked. Such systems would ensure that tenants are asked about all income sources and expenses that affect their rent. Manuals and training materials explaining how to implement requirements correctly and calculate rent accurately should be provided. To the extent that HUD program rules can be simplified, provision of automated and manual tools would be easier. HUD experts and local housing staff should be given an opportunity to work together to develop these tools and systems needed to reduce rent error. Many local PHA/owners have already developed forms, training materials, manuals, automated systems, and monitoring processes that have enabled them to provide accurate, efficient service to the tenants they serve. HUD should learn from these PHA/owners and develop materials that will help those PHA/owners who for one reason or another have not been as successful. 4) HUD should continue to implement its on-site monitoring program, and PHA/owners should be held accountable for implementing HUD regulations and calculating rent accurately. An on-site monitoring system that includes reviews at both the local and Federal level is essential to improving accountability. PHA/owners with excessive errors should be required to develop corrective action plans and show improvement within specified time periods. HUD has initiated extensive on-site monitoring efforts since the 2000 QC study, in contrast with its policies of most of the previous two decades. The most obvious explanation for the magnitude of error reductions in subsidy determinations between 2000 and 2007 is improved HUD monitoring and the expectation of such monitoring. However, as the dollars associated with rent error stop declining further action will be needed to help the PHAs and owners focus on policies and procedures that lead to error. Monitoring can be conducted at a variety of different levels. We recommend that HUD require PHA/owners to perform their own quality control reviews on a percentage of income determinations and rent calculations. Agencies that have aggressively sought to improve performance of their programs have had some significant successes, and one of the most frequently used error reduction strategies includes the establishment of internal quality control review procedures. In addition to agency monitoring, HUD Field Offices and/or other national-level well- trained staff should conduct a re-review of a percentage of the cases reviewed at the local level to ensure that the quality control reviews are being conducted correctly, or select their own random sample of files for review. This type of oversight not only identifies errors, but also prevents them. In addition, it demonstrates HUD’s concern and focuses PHA/owner attention on tenant income and rent. 5) Federal laws, regulations, and HUD requirements should be simplified to the extent possible. The current statutory environment poses substantial obstacles to efficient, accurate income and rent calculations. It contains dozens of requirements that may all be well-intentioned and have potentially desirable impacts but which, taken as a whole, make V-4

V. Recommendations the income and rent determination process extremely complex. HUD has sought to issue guidance on virtually all aspects of current income and rent determination requirements, but some of the legislative provisions were written without any thought as to implications for their administrative complexity. While determining which income to count, which expenses to allow, and annualizing that information in a program with multiple objectives may always be complicated, the various specialized provisions that relate to small subparts of the population could be eliminated or simplified. The policy related to students is the most recent example of such complex policies. PHA and project staff are required to gather a series of information to determine whether students continue to be eligible to receive assisted housing. For students who do not meet certain criteria, PHA/project staff are required to determine the eligibility of the student’s parents. This new policy, while well intentioned, just adds to the complex rules PHA/project staff are required to implement when determining eligibility and calculating rent for assisted households. 6) HUD should consider requiring some reexaminations to be completed less often than annually. Many years ago, the reexaminations for elderly and disabled families were conducted biannually rather than annually. HUD should consider implementing this policy again or possibly conducting reexaminations for selected populations every three years. To remove the issues related to incorrect subsidies because of the annual increase in Social Security benefits, the policy could require adding the annual SSA cost of living adjustment (COLA) to the total annual income for the households included in this group. With the time-savings made available by this change in policy, PHA/project staff could spend more time conducting required reexaminations, following up on suspected cases of fraud, and conducting more internal monitoring of tenant files. V-5

Appendix A—Rent Calculations

Appendix A—Rent Calculations 1. Public Housing a. Obtain the Total Tenant Payment (TTP). b. Determine if the family includes any ineligible noncitizens. IF YES, continue.

If NO, go to d.

c. Determine if the family includes any citizens or eligible noncitizens. IF YES, go to #3 (continuation). IF NO, go to #4 (temporary deferral). MARKER (marks the return point after determining continuation of assistance status) d. Obtain the Utility Allowance. e. Determine if the tenant selected the Flat Rent. IF NO, go to f. IF YES, the QC RENT equals the Flat Rent. Go to g. f. The amount of the tenant’s rent (QC RENT) is the lower of: a. (TTP), minus d. (Utility Allowance), or the Flat Rent*. g. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error. *Note: If there is no Flat Rent, the QC rent will be capped with the Ceiling Rent to determine the dollar amount of error. 2. Section 8 Vouchers a. Obtain TTP. b. Obtain the Gross Rent. c. Obtain Utility Allowance. d. If TTP is greater than Gross Rent, then set TTP to Gross Rent. e. Obtain Payment Standard 1 (the Payment Standard is based on the lower of the Unit (actual) Bedroom Size, and Family (eligible) Bedroom Size).

f. Obtain the household’s Adjusted Monthly Income.

1 For Project Based Vouchers, the Payment Standard equals the Gross Rent. A-1

Appendix A—Rent Calculations g. Subtract e. (Payment Standard) from b. (Gross Rent). If the Payment Standard is higher than the Gross Rent, use 0. h. Add a. (TTP) to g. (Gross Rent minus Payment Standard). i. Determine if this is the initial occupancy for this dwelling unit. (Item 12b on the 50058 is yes). IF YES, continue. IF NO, the Family Share = h. Go to l. j. Calculate 40 percent of the household’s Adjusted Monthly Income (f.). k. Determine if j. (40 percent of Adjusted Monthly Income) is equal to or greater than h. (TTP plus Gross Rent minus Payment Standard). IF YES, the Family Share = h. Go to l. IF NO, procedural error. Family Share = h. Go to l. l. Determine if the family includes any ineligible noncitizens. IF YES, continue. If NO, go to n. m. Determine if the family includes any citizens or eligible noncitizens. IF YES, go to #3 (continuation). IF NO, go to #4 (temporary deferral). MARKER (marks the return point after determining continuation of assistance status) n. Subtract c. (Utility Allowance) from the Family Share (h.). This is the QC RENT. o. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error. 3. Section 8 Enhanced Voucher a. Determine if household is receiving an Enhanced Voucher. If YES, continue. If NO, use regular Voucher formula. b. Obtain the Total Tenant Payment.

c. Obtain the Gross Rent.

d. Determine the lesser of b. (TTP) or c. (Gross Rent). e. Determine if the family includes any ineligible noncitizens. IF YES, continue. If NO, go to g. f. Determine if the family includes any citizens or eligible noncitizens. IF YES, go to #3 (continuation). IF NO, go to #4 (temporary deferral). MARKER (marks the return point after determining continuation of assistance status) g. Obtain the Utility Allowance. h. Subtract g. (Utility Allowance) from d. (the lesser of TTP or Gross Rent). This is the Family Rent to Owner (QC RENT). A-2

Appendix A—Rent Calculations i. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error. 4. Section 8 Project-Based, Section 202, Section 811, Section 8 Moderate Rehabilitation a. Obtain the Gross Rent (Gross Rent equals the Contract Rent plus the Utility Allowance). b. Obtain the TTP. c. Determine if the family includes any ineligible noncitizens. IF YES, continue. If NO, go to e. d. Determine if the family includes any citizens or eligible noncitizens. IF YES, go to #3 (continuation). IF NO, go to #4 (temporary deferral). MARKER (marks the return point after determining continuation of assistance status) e. Obtain the Utility Allowance. f. If Subsidy Type on 50059 = 7 or 8 (PRAC), go to h. g. Subtract e. (Utility Allowance) from b. (TTP) or a. (Gross Rent) whichever is lower. This is the QC RENT. Go to i. h. Subtract e. (Utility Allowance) from b. (TTP). This is the QC RENT. i. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error. 5. Manufactured Home Space Rental for Section 8 Vouchers a. Obtain the Rent to Owner. b. Obtain the owner maintenance and management charges for the space. c. Obtain the Utility Allowance d. Add together a. (Rent to Owner), b. (owner maintenance and management charges), and c. (utility allowance). This is the Space Rent. e. Obtain the TTP. f. Obtain the Payment Standard. g. Subtract f. (Payment Standard) from d. (Space Rent). h. Add e. (TTP) to g. (the amount by which the Space Rent exceeds the Payment Standard). This is the Family Share. A-3

Appendix A—Rent Calculations i. Determine if this is the initial occupancy for this dwelling unit. (Item 12b on the 50058 is yes). IF YES, continue. IF NO, the Family Share = h. Go to m. j. Obtain the household’s Adjusted Monthly Income. k. Calculate 40 percent of the household’s Adjusted Monthly Income. l. Determine if k. (40 percent of Adjusted Monthly Income) is equal to or greater than h. (TTP plus Space Rent minus Payment Standard). If YES, the Family Share = h.; go to m. If NO, Procedural Error. The family is not entitled to assistance in this unit. m. Determine if the family includes any ineligible noncitizens. IF YES, continue. If NO, go to o. n. Determine if the family includes any citizens or eligible noncitizens. IF YES, go to #3 (continuation). IF NO, go to #4 (temporary deferral). MARKER o. Subtract c. (Utility Allowance) from h. (Family Share) to determine QC Rent (Family Rent to Owner). p. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error. Special Calculations for Household with Ineligible Noncitizens 1. Continuation of Assistance a. Determine if the family was receiving assistance on June 19, 1995. IF YES, continue. IF NO, the FAMILY is eligible for prorated assistance; go to #5 (proration formula). b. Determine if the FAMILY head or spouse is a citizen or eligible noncitizen. IF YES, continue. IF NO, the FAMILY is eligible for prorated assistance; go to #5 (proration formula). c. Determine if the FAMILY includes any ineligible members other than the head, spouse, and child or parent of the head or spouse. IF NO, continue. IF YES, the FAMILY is eligible for prorated assistance; go to #5 (proration formula). d. Determine if the FAMILY was granted continuation of assistance before November 29, 1996. IF YES, the FAMILY is eligible for full continuation of assistance. Return to MARKER. IF NO, the FAMILY is eligible for prorated assistance; go to #5 (proration formula) 2. Temporary Deferral of Termination of Assistance a. Determine if Temporary Deferral of Termination of Assistance has been granted. If YES, continue. If NO, go to d. A-4

Appendix A—Rent Calculations b. Determine the date Temporary Deferral of Assistance was granted. c. Determine if more than 18 months have passed since Temporary Deferral of Termination of Assistance was granted. IF YES, go to d. IF No, the FAMILY is entitled to Temporary Deferral of Termination of Assistance; go to MARKER. d. Determine if the FAMILY includes a refugee under Section 207 of the Immigration and Naturalization Act or an individual seeking asylum under Section 208 of that Act. IF YES, the Family is entitled to ongoing Deferral of Termination of Assistance; go to MARKER. IF NO, continue. e. Determine if the FAMILY was receiving assistance on June 19, 1995. If YES, the Family is eligible for Temporary Deferral of Termination of Assistance; go to MARKER. f. Determine if the FAMILY is exercising its hearing rights (waiting for a decision from an INS or PHA/owner appeal). If YES, go to MARKER. IF NO, continue. g. Determine if the PHA is making reasonable efforts to evict. IF YES, go to MARKER. IF NO, Procedural Error, HOUSEHOLD IS INELIGIBLE. 3. Proration Formula for Public Housing a. Determine if this is a Public Housing case? IF YES, continue. IF NO, go to #4.

b. Determine the number of FAMILY members.

c. Determine the number of eligible FAMILY members.

d. Obtain the TTP.

e. Obtain the 95 th percentile of Gross Rents for similarly sized public housing units in order

to determine the public housing maximum rent. f. Determine if the Family pays a Flat Rent. IF NO, go to i. IF YES, continue. g. Obtain the Flat Rent. h. If g. (Flat Rent) is greater than or equal to e. (Maximum Rent), there is no prorated rent. Use the Flat Rent; go to n. If g. (Flat Rent) is less than the e. (Maximum Rent), subtract the Flat Rent from the Maximum Rent. This is the Family’s Maximum Subsidy. Go to j. i. Subtract d. (TTP) from e. (Maximum Rent) to determine Maximum Subsidy. j. Divide h. or i. (Maximum Subsidy) by b. (number of FAMILY members) and multiply by c. (number of eligible members) to determine the Eligible Subsidy for the FAMILY. k. Subtract j. (Eligible Subsidy) from e. (Maximum Rent) to obtain the prorated TTP. l. Obtain the Utility Allowance. A-5

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Appendix A—Rent Calculations m. The amount of the tenant’s rent (QC RENT) is k. (prorated TTP) minus l. (Utility

Allowance). Did the Family accept the prorated rent? Y/N. IF NO, go to #4.

n. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error 4. Proration Formula for All Section 8 Programs a. Obtain the Rent to Owner (voucher). b. Obtain the Utility Allowance c. Obtain the Gross Rent.

Voucher Gross Rent = Rent to Owner plus the Utility Allowance.

d. Obtain the TTP. e. Obtain the Payment Standard (Voucher). f. Obtain the HAP. Owner Administered: HAP = Gross Rent minus TTP. Voucher: HAP = Gross Rent or Payment Standard (whichever is less) minus the TTP. Enhanced Voucher: HAP = Gross Rent minus the Payment Standard. g. Record the number of FAMILY members.

h. Record the number of eligible FAMILY members.

i. Divide f. (HAP) by g. (total number of FAMILY members), and then multiply the result

by h. (number of eligible FAMILY members) to obtain the prorated HAP. j. If Manufactured Home Space Rental, return to MARKER. k. Subtract i. (prorated HAP) from c. (Gross Rent) to obtain the prorated Family Share. l. Subtract b. (Utility Allowance) from k. (Prorated Family Share) to determine the prorated QC RENT. m. Determine if the QC RENT equals the ACTUAL RENT. IF YES, no error. IF NO, dollar error. A-6

Appendix B—Weighting Procedure

Appendix B—Weighting Procedure This appendix describes the procedures followed in weighting the sample data. Study Population. The universe under study includes all projects and tenants located in the continental United States, Alaska, Hawaii, and Puerto Rico. The following programs are included in the sample: ♦ PIH-administered Public Housing (Public Housing) ♦ PIH-administered Section 8 (PHA-administered Section 8)

ƒ Moderate Rehabilitation

ƒ Housing Choice Voucher Program

♦ Office of Housing-administered projects (owner-administered)

ƒ Section 8 New Construction/Substantial Rehabilitation

ƒ Section 8 Loan Management

ƒ Section 8 Property Disposition

ƒ Section 202 Project Rental Assistance Contracts (PRAC)

ƒ Section 202/162 Project Assistance Contracts (PAC)

ƒ Section 811 PRAC

The frames used to draw the sample include many out-of-scope projects such as projects in the Move-to-Work program and projects that have been demolished or that are no longer assisted housing. Many of these projects were identified before the sample was drawn, but others were not and had to be replaced. In addition, at times projects resulting from a merger of two or more projects or that were split into two or more were identified, resulting in difficult sampling decisions. Weighting Strategy. The weighting procedure usually begins with the determination of the probability of selection of every unit in the sample. The use of purposive replacement for out-of- scope projects for any of several reasons makes the sample weight calculations complicated. The determination of an actual probability of selection for a replacement is impossible to make. A sampling weight proportional to what the probability would have been if the project had been selected originally is a reasonable estimate. The probability of selection of a tenant was thus the product of the following combinations: 1) The probability of selection of the Primary Sampling Unit (PSU) 2) The probability of selection of a sub-PSU if the PSU was split 3) The probability of selection of the project from the PSU 4) The probability of selection of the tenant from the project. The four probabilities were multiplied together and formed the preliminary weights. The weights were then adjusted to be added to estimates of the national total of tenants in each program. The weights summed to 1,320,000 for the owner-administered programs, 955,000 for Public Housing, and 1,858,000 for the PHA-administered Section 8 Program. B-2

Appendix B—Weighting Procedure Primary Sampling Unit Probabilities. Each PSU was sampled with probabilities proportional to size. The size measure used was the number of tenants adjusted to obtain equal expectation for the three major types of programs in the study. The number of tenants of each kind in a PSU was multiplied by an inflation factor to make all three numbers equal. The size measures were then added; the PSU probability of selection was its size measure divided by the sum of the size measures nationwide, multiplied by the number of PSUs to be selected (60). PSUs with probabilities greater than one could be selected more than once (Sampling with Minimal Replacement). For weighting purposes, probabilities greater than one were set to 1.0. Some PSUs were divided into multiple geographic areas and one of these smaller geographic areas was selected with probabilities proportional to size. This resulted in the same probability that would have ensued had the division taken place before the sample was drawn. Project Probabilities. This was defined as the minimum of kt/T and one, where k is the number of projects in the program selected from the PSU, t is the number of tenants in the project and T is the number of tenants in the program that are in the PSU. The PHA-administered Section 8 projects could have a probability greater than one for sampling purposes (meaning they could be sampled more than once) but for the other two major program types, if the calculated probability exceeded one, it was set to one and all the other probabilities were readjusted so they added to the allocation for the program in the PSU. For weighting purposes probabilities greater than one among PHA-administered Section 8 projects were set to one. Tenant Probabilities. This is the total number of tenants sampled from the project divided by the estimated number of tenants whose annual recertifications were conducted during the study period. The estimate was obtained by multiplying the total number of tenants by the proportion of tenants selected who were in scope for the study (i.e., who were subsidized by one of the programs). For example, if six tenants were reviewed to find four tenants who were both in scope and available for interviewing, one who was out of town, and one who was not subsidized, from a list of 120 tenants, then the estimate would be 120 x (5/6) = 100 tenants. Post-Stratification. The sample was designed to obtain similar numbers of tenants in each of the following three categories of projects: ♦ Public Housing projects ♦ PHA-administered Section 8 projects ♦ Owner-administered projects HUD provided approximate totals for each of the three categories. The sampling frame totals did not correspond exactly to these numbers and required extensive adjustments. This was in part because the numbers were approximations; but also in part because the geographic areas affected by the 2005 hurricanes were excluded from the frame, but included during the weighting process. To recapitulate, the weights were adjusted so that they add up to the totals provided by the external source, so the sum of the weights would have been the same had a different sample been selected. Trimming the Weights. The final step was the trimming of the weights. Weights more than three times the median weight were set to three times the median weight and all the weights were readjusted. Large weights usually resulted from incorrect frame information. B-3

Appendix B—Weighting Procedure Effective Sample Size. The weights led to an effective sample size (because of the weighting) of 759 (down from an actual size of 804) for the Office of Housing-administered projects, 726 for the Public Housing projects (down from 800), and 738 for the PHA-administered Section 8 projects (down from 800). Variance Estimation. Standard errors were obtained for a number of estimates using a delete-a- group Jackknife procedure (JK1 in WESVAR). This was implemented using twenty replicate groups and creating twenty sets of replicate weights. This procedure is considered more robust with respect to design characteristics than the Taylor Series method used in the previous cycle (Kott, 1998 1 ). 1 Kott, P. S. (1998), “Using the Delete-a-Group Jacknife Variance Estimator in Practice,” Proceedings of the Annual

Meeting of the American Statistical Association, Section on Survey Research Methods, pp. 763-768.

Alexandria, VA: American Statistical Association.

B-4

Appendix C—Source Tables Based on Quality Control Data

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 1a. Verification of QC Rent Components Third Party Verbal or In Writing, or Documentation, or EIV FULLY VERIFIED # of Cases # of Cases # of Cases Cases Cases Cases Earned Income 74 (5.7%) 56 (4.2%) 1,181 (90.1%) 3 (.1%) 36 (1.4%) 2,436 (98.4%) Public Assistance 33 (6.5%) 478 (93.5%) Other Income 122 (16.1%) 22 (2.9%) 615 (81.0%) Asset Income 13 (1.6%) 101 (12.5%) 697 (85.9%) 46 (25.9%) 8 (4.5%) 123 (69.6%) 2 ) 118 (8.9%) 216 (16.3%) 988 (74.7%) 2008.10.05 HUD QC FY 2007 FULLY VERIFIED # of Cases # of Cases # of Cases Cases Cases Cases Earned Income 235 (17.5%) 90 (6.7%) 1,020 (75.8%) 92 (3.7%) 163 (6.6%) 2,224 (89.7%) Public Assistance 142 (26.1%) 4 (.8%) 398 (73.1%) Other Income 330 (38.8%) 25 (2.9%) 496 (58.3%) Asset Income 92 (11.3%) 190 (23.4%) 530 (65.3%) 66 (37.3%) 6 (3.4%) 104 (59.3%) 2 ) 240 (18.1%) 446 (33.7%) 636 (48.1%) NOT VERIFIED PARTIALLY VERIFIED Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) Pension, Etc. Child Care Expense Disability Expense (100.0% Medical Expense Table 1b. Verification of QC Rent Components Third Party In Writing NOT VERIFIED PARTIALLY VERIFIED Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) Pension, Etc. Child Care Expense Disability Expense (100.0% Medical Expense 2008.10.05 C-1

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 1c. Verification of QC Rent Components Third Party In Writing, or EIV FULLY VERIFIED # of Cases # of Cases # of Cases Cases Cases Cases Earned Income 203 (15.1%) 86 (6.4%) 1,056 (78.5%) 63 (2.6%) 141 (5.7%) 2,274 (91.7%) Public Assistance 134 (24.6%) 4 (.8%) 406 (74.6%) Other Income 330 (38.8%) 22 (2.6%) 499 (58.6%) Asset Income 92 (11.3%) 190 (23.4%) 530 (65.3%) 66 (37.3%) 6 (3.4%) 104 (59.3%) 2 ) 224 (17.0%) 426 (32.2%) 672 (50.8%) 2008.10.05 HUD QC FY 2007 FULLY VERIFIED # of Cases # of Cases # of Cases Cases Cases Cases Earned Income 1,329 (98.8%) 8 (.6%) 8 (.6%) 2,471 (99.7%) 7 (.3%) Public Assistance 538 (98.8%) 6 (1.2%) Other Income 842 (99.0%) 3 (.3%) 6 (.6%) Asset Income 811 (99.8%) 1 (.2%) 175 (99.3%) 1 (.7%) 2 ) 1,305 (98.7%) 12 (.9%) 4 (.3%) NOT VERIFIED PARTIALLY VERIFIED Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) Pension, Etc. Child Care Expense Disability Expense (100.0% Medical Expense Table 1d. Verification of QC Rent Components Third Party - Verbal NOT VERIFIED PARTIALLY VERIFIED Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) Pension, Etc. Child Care Expense Disability Expense (100.0% Medical Expense 2008.10.05 C-2

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 1e. Verification of QC Rent Components Documentation FULLY VERIFIED # of Cases # of Cases # of Cases Cases Cases Cases Earned Income 1,231 (91.5%) 31 (2.3%) 83 (6.2%) 2,314 (93.4%) 113 (4.5%) 52 (2.1%) Public Assistance 478 (87.9%) 4 (.8%) 61 (11.3%) Other Income 732 (86.1%) 17 (2.1%) 101 (11.8%) Asset Income 622 (76.5%) 117 (14.4%) 74 (9.1%) 157 (89.3%) 2 (1.1%) 17 (9.6%) 2 ) 936 (70.8%) 300 (22.7%) 86 (6.5%) 2008.10.05 HUD QC FY 2007 ) FULLY VERIFIED # of Cases # of Cases # of Cases Cases Cases Cases Earned Income 1,302 (96.8%) 14 (1.0%) 29 (2.2%) 2,420 (97.6%) 32 (1.3%) 27 (1.1%) Public Assistance 536 (98.6%) 8 (1.4%) Other Income 848 (99.7%) 3 (.3%) Asset Income 812 ) 176 ) 2 ) 1,279 (96.8%) 31 (2.3%) 11 (.9%) NOT VERIFIED PARTIALLY VERIFIED Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) Pension, Etc. Child Care Expense Disability Expense (100.0% Medical Expense Table 1f. Verification of QC Rent Components EIV (Enterprise Income Verification NOT VERIFIED PARTIALLY VERIFIED Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) Pension, Etc. (100.0% Child Care Expense (100.0% Disability Expense (100.0% Medical Expense 2008.10.05 C-3

2008.10.05 Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 2. Percent of Households by Payment Type and Program Type UNDERPAYMENT PROPER PAYMENT OVERPAYMENT Total # of # of Row % Row % Row % Cases Cases of Col % of # of Cases of Col % of # of Cases of Col % of (in Row % of Col % of (in 1,000) Cases Cases (in 1,000) Cases Cases (in 1,000) Cases Cases 1,000) Cases Cases Public Housing 156 (16.4%) (20.5%) 663 (69.4%) (25.0%) 136 (14.2%) (18.8%) 955 (100.0%) (23.1%) Section 8 PHA l OWNER 350 506 257 (18.8%) (18.0%) (19.5%) (45.8%) (66.3%) (33.7%) 1,142 1,805 841 (61.5%) (64.2%) (63.7%) (43.2%) (68.2%) (31.8%) 366 502 222 (19.7%) (17.8%) (16.8%) (50.6%) (69.3%) (30.7%) 1,858 2,813 1,320 ) ) ) (45.0%) (68.1%) (31.9%) l l 257 763 (19.5%) (18.5%) (33.7%) ) 841 2,646 (63.7%) (64.0%) (31.8%) ) 222 724 (16.8%) (17.5%) (30.7%) ) 1,320 4,133 ) ) (31.9%) ) ADMINISTERED Tota Owner- Administered ADMINISTERED (100.0% (100.0% (100.0% Tota Tota (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% HUD QC FY 2007 Table 2(S). Percent of Households by Payment Type and Program Type (Proper Payment based on exact match of Actual and QC Rent) UNDERPAYMENT PROPER PAYMENT OVERPAYMENT Total # of Cases Row % Col % of # of Cases Row % Col % of # of Cases Row % Col % of # of Cases Row % of Col % of (in 1,000) of Cases Cases (in 1,000) of Cases Cases (in 1,000) of Cases Cases (in 1,000) Cases Cases Section 8 PHA l OWNER (45.0%) (68.1%) (31.9%) l l 356 968 (27.0%) (23.4%) (36.8%) ) 636 2,113 (48.2%) (51.1%) (30.1%) ) 327 1,052 (24.8%) (25.5%) (31.1%) ) 1,320 4,133 ) ) (31.9%) ) ADMINISTERED Tota Owner- Administered ADMINISTERED Tota Tota (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% Public Housing 191 (20.0%) (19.7%) 544 (57.0%) (25.8%) 220 (23.1%) (20.9%) 955 (100.0%) (23.1%) 421 (22.6%) (43.5%) 933 (50.2%) (44.1%) 505 (27.2%) (48.0%) 1,858 (100.0%) 611 (21.7%) (63.2%) 1,477 (52.5%) (69.9%) 725 (25.8%) (68.9%) 2,813 (100.0%) 356 (27.0%) (36.8%) 636 (48.2%) (30.1%) 327 (24.8%) (31.1%) 1,320 (100.0%) C-4 2008.10.05

2008.10.05 Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 3. Dollar Rent Error by Program Type ACTUAL RENT (MONTHLY) QC RENT (MONTHLY) GROSS RENT ERROR (MONTHLY) Sum # of Dollar Ave. Cases Col % of Amount Dollar (in 1,000) Cases (in 1,000) Amount Public Housing 955 (23.1%) 218,505 228.80 Sum Sum Dollar # of Dollar Ave. # of Amount Ave. Cases Col % of Amount Dollar Cases Col % of (in Dollar (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases 1,000) Amount 955 (23.1%) 224,044 234.60 955 (23.1%) 12,447 13.03 PHA OWNER Section 8 l 1,858 2,813 1,320 (45.0%) (68.1%) (31.9%) 1,858 2,813 1,320 (45.0%) (68.1%) (31.9%) 1,858 2,813 1,320 (45.0%) (68.1%) (31.9%) 19.51 17.31 12.57 l l 1,320 4,133 (31.9%) ) 1,320 4,133 (31.9%) ) 1,320 4,133 (31.9%) ) 12.57 15.80 ADMINISTERED ADMINISTERED Tota Owner- Administered 392,125 610,631 276,013 211.05 217.07 209.10 402,903 626,947 281,784 216.85 222.87 213.47 36,251 48,698 16,593 Tota Tota (100.0% 276,013 886,643 209.10 214.53 (100.0% 281,784 908,731 213.47 219.87 (100.0% 16,593 65,291 C-5

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007

Table 4. Dollar Error Amount by Payment Type and Program Type

Sum Sum Sum # of Amount Ave. # of Ave. # of Ave. Cases Col % of (in Cases Col % of Amount Cases Col % of Amount Cases 1,000) Amount Cases Amount (in 1,000) Cases Amount PHA OWNER Section 8 l 156 350 506 257 (20.5%) (45.8%) (66.3%) (33.7%) 8,866 56.72 67.41 64.11 43.64 136 366 502 222 (18.8%) (50.6%) (69.3%) (30.7%) 3,581 5,387 26.33 34.64 32.39 24.28 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 257 763 (33.7%) ) 43.64 57.22 222 724 (30.7%) ) 5,387 24.28 29.90 1,320 4,133 (31.9%) ) HUD QC FY 2007 ( Sum Sum Sum # of Ave. # of Ave. # of Ave. Cases Col % of Amount Cases Col % of Amount Cases Col % of Amount (in 1,000) Cases Amount (in 1,000) Cases Amount Cases Amount PHA OWNER Section 8 l 191 421 611 356 (19.7%) (43.5%) (63.2%) (36.8%) 8,943 46.93 56.49 53.51 32.16 220 505 725 327 (20.9%) (48.0%) (68.9%) (31.1%) 3,739 5,690 16.98 25.72 23.07 17.38 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 356 968 (36.8%) ) 32.16 45.64 327 1,052 (31.1%) ) 5,690 17.38 21.30 1,320 4,133 (31.9%) ) UNDERPAYMENT (MONTHLY) OVERPAYMENT (MONTHLY) QC RENT (MONTHLY) Dollar Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) (in 1,000) (in 1,000) ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered 23,570 32,436 11,205 12,681 16,262 224,044 402,903 626,947 281,784 234.60 216.85 222.87 213.47 Tota Tota (100.0% 11,205 43,641 (100.0% 21,649 (100.0% 281,784 908,731 213.47 219.87 2008.10.05 Table 4 S). Dollar Error Amount by Payment Type and Program Type (Proper Payment based on exact match of Actual and QC Rent) UNDERPAYMENT (MONTHLY) OVERPAYMENT (MONTHLY) QC RENT (MONTHLY) Dollar Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) (in 1,000) (in 1,000) ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered 23,757 32,700 11,461 12,979 16,718 224,044 402,903 626,947 281,784 234.60 216.85 222.87 213.47 Tota Tota (100.0% 11,461 44,161 (100.0% 22,408 (100.0% 281,784 908,731 213.47 219.87 C-6

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 5. Gross and Net Rent Error by Program Type GROSS RENT ERROR (MONTHLY) NET RENT ERROR (MONTHLY) QC RENT (MONTHLY) Sum Dollar Sum Sum # of Amount Ave. # of Dollar Ave. # of Dollar Ave. Cases Col % of (in Dollar Cases Col % of Amount Dollar Cases Col % of Amount Dollar (in 1,000) Cases 1,000) Amount (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases (in 1,000) Amount PHA OWNER Section 8 l 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) 13.03 19.51 17.31 12.57 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) -5,284 -5,818 -5.53 -5.86 -5.75 -4.41 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 1,320 4,133 (31.9%) ) 12.57 15.80 1,320 4,133 (31.9%) ) -5,818 -4.41 -5.32 1,320 4,133 (31.9%) ) HUD QC FY 2007 ( Sum Sum Sum # of Ave. # of Ave. # of Ave. Cases Col % of Amount Cases Col % of Amount Cases Col % of Amount Cases Amount (in 1,000) Cases Amount (in 1,000) Cases Amount PHA OWNER Section 8 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) 13.28 19.77 17.57 12.99 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) -5,204 -5,771 -5.45 -5.80 -5.68 -4.37 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l 1,320 4,133 (31.9%) ) 12.99 16.11 1,320 4,133 (31.9%) ) -5,771 -4.37 -5.26 1,320 4,133 (31.9%) ) ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered 12,447 36,251 48,698 16,593 -10,890 -16,174 224,044 402,903 626,947 281,784 234.60 216.85 222.87 213.47 Tota Tota (100.0% 16,593 65,291 (100.0% -21,992 (100.0% 281,784 908,731 213.47 219.87 2008.10.05 Table 5 S). Gross and Net Rent Error by Program Type (Proper Payment based on exact match of Actual and QC Rent) GROSS RENT ERROR (MONTHLY) NET RENT ERROR (MONTHLY) QC RENT (MONTHLY) Dollar Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) (in 1,000) (in 1,000) ADMINISTERED ADMINISTERED Public Housing Group Total Owner- Administered 12,681 36,737 49,418 17,151 -10,778 -15,982 224,044 402,903 626,947 281,784 234.60 216.85 222.87 213.47 Tota Group Total (100.0% 17,151 66,569 (100.0% -21,753 (100.0% 281,784 908,731 213.47 219.87 C-7

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 6. Case Type by Program Type OVERDUE l # of # of # of # of Cases Col % of Cases Col % of Cases Col % of Cases (in Col % of of Cases Cases of Cases Cases of Cases Cases 1,000) Cases Cases OWNER Section 8 l 148 280 428 212 (15.5%) (15.1%) (15.2%) (16.1%) (23.1%) (43.8%) (66.9%) (33.1%) 783 1,538 2,321 1,108 (82.0%) (82.8%) (82.5%) (83.9%) (22.8%) (44.9%) (67.7%) (32.3%) 24 39 64 (2.5%) (2.1%) (2.3%) (38.2%) (61.8%) ) 955 1,858 2,813 1,320 ) ) ) ) (23.1%) (45.0%) (68.1%) (31.9%) l l 212 641 (16.1%) (15.5%) (33.1%) ) 1,108 3,429 (83.9%) (83.0%) (32.3%) ) 64 (1.5%) ) 1,320 4,133 ) ) (31.9%) ) CERTIFICATIONS RECERTIFICATIONS/NON- RECERTIFICATIONS/OVERDUE Tota Row % Row % Row % Row % of (in 1,000) (in 1,000) (in 1,000) PHA ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered (100.0% (100.0% (100.0% (100.0% (100.0% Tota Tota (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% 2008.10.05 C-8

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 7. Percent of Newly Certified Households Meeting Certification Criteria # of Cases # of Cases % of Cases % of Cases 624 (97.5%) 16 (2.5%) 625 (97.6%) 15 (2.4%) 612 (95.6%) 28 (4.4%) Income 639 (99.7%) 2 (.3%) i Criteria 581 (90.6%) 60 (9.4%) Met Criterion Did Not Meet Criterion (in 1,000) (in 1,000) Citizenship Social Security Number Consent Form Low and Very Low Meets All Elig bility 2008.10.05 C-9

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 7b. Percent of Newly Certified Households Meeting Certification Criteria by Program Type # of Cases # of Cases % of Cases % of Cases PUBLIC HOUSING Income 146 140 144 148 (99.0%) (94.6%) (97.5%) ) 2 8 4 (1.0%) (5.4%) (2.5%) i Criteria 136 (92.0%) 12 (8.0%) Income 278 278 263 280 (99.1%) (99.1%) (93.7%) ) 2 2 18 (.9%) (.9%) (6.3%) i Criteria 258 (92.0%) 22 (8.0%) Income 200 208 205 210 (94.3%) (97.8%) (96.7%) (99.1%) 12 5 7 2 (5.7%) (2.2%) (3.3%) (.9%) i Criteria 187 (87.9%) 26 (12.1%) Met Criterion Did Not Meet Criterion (in 1,000) (in 1,000) Citizenship Social Security Number Consent Form Low and Very Low (100.0% Meets All Elig bility PHA-ADMINISTERED SECTION 8 Citizenship Social Security Number Consent Form Low and Very Low (100.0% Meets All Elig bility OWNER-ADMINISTERED Citizenship Social Security Number Consent Form Low and Very Low Meets All Elig bility 2008.10.05 C-10

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 8. Dollar Error Amount by Payment Type and Case Type UNDERPAYMENT (MONTHLY) OVERPAYMENT (MONTHLY) QC RENT (MONTHLY) Sum Dollar Ave. Sum Dollar Ave. Sum Dollar Ave. # of Cases Col % of Amount (in Dollar # of Cases Col % of Amount (in Dollar # of Cases Col % of Amount (in Dollar (in 1,000) Cases 1,000) Amount (in 1,000) Cases 1,000) Amount (in 1,000) Cases 1,000) Amount ON .00 l Non- Overdue 100 100 627 (13.1%) (13.1%) (82.2%) 5,450 5,450 54.56 54.56 55.55 110 110 602 (15.2%) (15.2%) (83.2%) 4,042 4,042 36.69 36.69 28.64 641 641 3,429 (15.5%) (15.5%) (83.0%) l Overdue l 36 663 763 (4.7%) (86.9%) ) 3,362 93.72 57.62 57.22 12 614 724 (1.6%) (84.8%) ) 357 30.88 28.68 29.90 64 3,492 4,133 (1.5%) (84.5%) ) HUD QC FY 2007 Sum Ave. Ave. Ave. # of Cases Col % of Amount # of Cases Col % of Amount # of Cases Col % of Amount Cases (in 1,000) Amount Cases Amount Cases Amount ON .00 l Non- Overdue 137 137 790 (14.2%) (14.2%) (81.7%) 5,534 5,534 40.30 40.30 44.62 148 148 889 (14.1%) (14.1%) (84.5%) 4,135 4,135 27.92 27.92 20.14 641 641 3,429 (15.5%) (15.5%) (83.0%) l Overdue l 40 830 968 (4.1%) (85.8%) ) 3,375 84.09 46.53 45.64 15 904 1,052 (1.4%) (85.9%) ) 367 24.62 20.21 21.30 64 3,492 4,133 (1.5%) (84.5%) ) CERTIFICATION RECERTIFICATI Tota 34,829 17,250 112,621 112,621 777,696 175.83 175.83 226.80 Tota Tota (100.0% 38,192 43,641 (100.0% 17,608 21,649 (100.0% 18,413 796,109 908,731 289.91 227.95 219.87 2008.10.05 Table 8(S). Dollar Error Amount by Payment Type and Case Type (Proper Payment based on exact match of Actual and QC Rent) UNDERPAYMENT (MONTHLY) OVERPAYMENT (MONTHLY) QC RENT (MONTHLY) Sum Dollar Sum Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) CERTIFICATION RECERTIFICATI Tota 35,252 17,906 112,621 112,621 777,696 175.83 175.83 226.80 Tota Tota (100.0% 38,627 44,161 (100.0% 18,273 22,408 (100.0% 18,413 796,109 908,731 289.91 227.95 219.87 2008.10.05 C-11

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 TABLE 9. Largest Component Error for Households with Rent Error (Annual Dollars) # of Cases Col % of Amount Cases Amount Earned Income 362 (24.3%) 2,887 316 (21.2%) 2,075 Public Assistance 87 (5.8%) 2,492 Other Income 166 (11.2%) 2,437 Asset Income 53 (3.6%) 1,502 51 (3.4%) 622 40 (2.7%) 400 57 (3.8%) 2,259 342 (23.0%) 972 No Error 13 (.9%) 0 0 l 1,487 ) 1,957 HUD QC FY 2007 # of Cases Col % of Amount # of Cases Col % of Amount Cases (in 1,000) Amount Cases (in 1,000) Amount 292 (19.7%) 292 (19.7%) Section 8 716 (48.1%) 716 (48.1%) l 1,008 (67.8%) 1,008 (67.8%) i 479 (32.2%) 479 (32.2%) l 479 (32.2%) 479 (32.2%) l 1,487 ) 1,487 ) Sum Dollar Ave. Dollar (in 1,000) (in 1,000) 1,044,257 Pension, Etc. 654,980 216,730 405,306 79,432 Dependent Allowance 31,917 Elderly HH Allowance 15,927 Child Care Allowance 129,267 Medical Allowance 332,330 Tota (100.0% 2,910,147 2008.10.05 Table 10. Total and Largest Dollar Error by Program Type for Households with Rent Errors TOTAL DOLLAR IN ERROR LARGEST DOLLAR ERROR Sum Dollar Sum Dollar Ave. Dollar Ave. Dollar (in 1,000) (in 1,000) PHA ADMINISTERED Public Housing 621,578 2,126.47 519,627 1,777.69 1,923,679 2,687.57 1,632,911 2,281.34 Tota 2,545,258 2,524.87 2,152,538 2,135.30 OWNER ADMINISTERED Owner-Admin stered 912,898 1,907.03 757,610 1,582.64 Tota 912,898 1,907.03 757,610 1,582.64 Tota (100.0% 3,458,156 2,325.94 (100.0% 2,910,147 1,957.35 2008.10.05 C-12

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 11. QC Rent Components by Payment Type and Administration Type PHA ADMINISTERED OWNER ADMINISTERED # of Cases Col % of (in 1,000) Cases Earned Income 199 Total Row % of # of Cases Col % of Row % of # of Cases Col % of Row % of Cases (in 1,000) Cases Cases (in 1,000) Cases Cases 51 250(7.1%) (79.5%) (3.9%) (20.5%) (6.0%) ) 198 (7.1%) (66.7%) 99 (7.5%) (33.3%) 297 (7.2%) ) Public Assistance 43 (1.5%) (81.2%) 10 (.7%) (18.8%) 52 (1.3%) ) Other Income 113 (4.0%) (74.6%) 39 (2.9%) (25.4%) 152 (3.7%) ) Asset Income 61 (2.2%) (41.0%) 88 (6.6%) (59.0%) 148 (3.6%) ) 23 (.8%) (62.1%) 14 (1.0%) (37.9%) 36 (.9%) ) 21 (.8%) (85.9%) 4 (.3%) (14.1%) 25 (.6%) ) 15 (.5%) (50.7%) 14 (1.1%) (49.3%) 29 (.7%) ) 4 (.1%) ) 4 (.1%) ) 109 (3.9%) (45.9%) 128 (9.7%) (54.1%) 237 (5.7%) ) No Error 2 (.1%) (23.7%) 6 (.4%) (76.3%) 8 (.2%) ) Earned IncomePROPER PAYMENT 164 (5.8%) (82.6%) 34 (2.6%) (17.4%) 198 (4.8%) ) 205 (7.3%) (57.5%) 152 (11.5%) (42.5%) 356 (8.6%) ) Public Assistance 65 (2.3%) (91.5%) 6 (.5%) (8.5%) 71 (1.7%) ) Other Income 103 (3.7%) (82.0%) 23 (1.7%) (18.0%) 126 (3.0%) ) Asset Income 152 (5.4%) (62.7%) 90 (6.8%) (37.3%) 242 (5.9%) ) 34 (1.2%) (85.8%) 6 (.4%) (14.2%) 39 (1.0%) ) 15 (.5%) (92.0%) 1 (.1%) (8.0%) 16 (.4%) ) 10 (.4%) (65.5%) 5 (.4%) (34.5%) 15 (.4%) ) 124 (4.4%) (44.6%) 154 (11.7%) (55.4%) 278 (6.7%) ) No Error 1,216 (43.2%) (70.2%) 515 (39.0%) (29.8%) 1,731 (41.9%) ) Earned IncomeOVERPAYMENT 154 (5.5%) (82.5%) 33 (2.5%) (17.5%) 186 (4.5%) ) 147 (5.2%) (59.0%) 102 (7.7%) (41.0%) 249 (6.0%) ) Public Assistance 37 (1.3%) (74.7%) 13 (1.0%) (25.3%) 50 (1.2%) ) Other Income 76 (2.7%) (81.9%) 17 (1.3%) (18.1%) 93 (2.3%) ) Asset Income 66 (2.3%) (54.1%) 56 (4.2%) (45.9%) 122 (2.9%) ) 68 (2.4%) (89.4%) 8 (.6%) (10.6%) 76 (1.8%) ) 61 (2.2%) (86.4%) 10 (.7%) (13.6%) 70 (1.7%) ) 45 (1.6%) (84.9%) 8 (.6%) (15.1%) 53 (1.3%) ) 1 (.0%) ) 1 (.0%) ) 171 (6.1%) (55.0%) 140 (10.6%) (45.0%) 311 (7.5%) ) No Error 6 (.2%) ) 6 (.1%) ) / 2,813 ) (68.1%) 1,320 ) (31.9%) 4,133 ) ) UNDERPAYMENT (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Dependent Allowance (100.0% Elderly HH Allowance (100.0% Child Care Allowance (100.0% Disability Allowance (100.0% (100.0% Medical Allowance (100.0% (100.0% (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Dependent Allowance (100.0% Elderly HH Allowance (100.0% Child Care Allowance (100.0% Disability Allowance Medical Allowance (100.0% (100.0% (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Dependent Allowance (100.0% Elderly HH Allowance (100.0% Child Care Allowance (100.0% Disability Allowance (100.0% (100.0% Medical Allowance (100.0% (100.0% (100.0% TOTAL w Rent Error Calc (100.0% (100.0% (100.0% (100.0% C-13

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 12a. Elderly/Disabled Allowances NON-ELDERLY/DISABLED HH ELDERLY/DISABLED HH l # of Cases Col % of # of Cases Col % of # of Cases Col % of Cases Cases Cases Cases Cases Cases 1,748 (99.8%) ) 1,748 (42.3%) ) 3 (.2%) (2.7%) 108 (4.6%) (97.3%) 111 (2.7%) ) 2,273 (95.4%) ) 2,273 (55.0%) ) l 1,751 ) (42.4%) 2,382 ) (57.6%) 4,133 ) ) HUD QC FY 2007 HH W/DEPENDENT l # of Cases Col % of # of Cases Col % of # of Cases Col % of Cases Cases Cases Cases Cases Cases 2,315 (99.7%) ) 2,315 (56.0%) ) 7 (.3%) (4.5%) 145 (8.0%) (95.5%) 151 (3.7%) ) 1,667 (92.0%) ) 1,667 (40.3%) ) l 2,322 ) (56.2%) 1,811 ) (43.8%) 4,133 ) ) Tota Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) No Allowance (100.0% (100.0% Incorrect Allowance (100.0% Correct Allowance (100.0% (100.0% Tota (100.0% (100.0% (100.0% (100.0% 2008.10.05 Table 12b. Dependent Allowances HH W/OUT DEPENDENT Tota Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) No Allowance (100.0% (100.0% Incorrect Allowance (100.0% Correct Allowance (100.0% (100.0% Tota (100.0% (100.0% (100.0% (100.0% 2008.10.05 C-14

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 13. Calculation Errors on Form 50058/59 58 59 l # of Errors # of Cases # of Errors # of Cases # of Errors # of Cases HOUSEHOLD 223 212 62 44 284 255 AND INCOME 362 220 163 78 525 298 1,537 1,213 144 61 1,681 1,274 FAMILY RENT AND SUBSIDY INFORMATION 491 283 65 38 556 322 Tota (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) COMPOSITION NET FAMILY ASSETS ALLOWANCES AND ADJUSTED INCOME 2008.10.05 C-15

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 14. Consistency Errors on Form 50058/59 58 59 l # of Errors # of Cases # of Errors # of Cases # of Errors # of Cases GENERAL INFORMATION 48 48 89 71 137 119 HOUSEHOLD 630 156 47 39 677 195 AND INCOME 275 272 4 3 279 275 273 272 14 14 287 286 FAMILY RENT AND SUBSIDY INFORMATION 96 96 26 22 122 118 Tota (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) COMPOSITION NET FAMILY ASSETS ALLOWANCES AND ADJUSTED INCOME 2008.10.05 C-16

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15a. Verification of Form 50058/59 Rent Components Third Party Verbal or In Writing, or Documentation, or EIV l Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 128 (9.7%) 295 (22.5%) 890 (67.8%) 1,312 ) 108 (4.4%) 286 (11.7%) 2,062 (84.0%) 2,456 ) Public Assistance 53 (10.7%) 73 (14.7%) 372 (74.6%) 498 ) Other Income 204 (25.3%) 108 (13.3%) 494 (61.3%) 806 ) Asset Income 46 (6.6%) 65 (9.4%) 584 (84.0%) 694 ) 15 (11.7%) 28 (21.6%) 85 (66.6%) 127 ) 68 (7.1%) 171 (17.7%) 727 (75.2%) 966 ) HUD QC FY 2007 / Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 429 (32.7%) 175 (13.3%) 708 (53.9%) 1,312 ) 1,316 (53.6%) 110 (4.5%) 1,031 (42.0%) 2,456 ) Public Assistance 153 (30.8%) 56 (11.3%) 288 (57.9%) 498 ) Other Income 385 (47.8%) 64 (8.0%) 356 (44.2%) 806 ) Asset Income 250 (36.0%) 38 (5.4%) 407 (58.6%) 694 ) 40 (31.3%) 19 (14.6%) 69 (54.1%) 127 ) 581 (60.1%) 64 (6.7%) 321 (33.2%) 966 ) NO VERIFICATION VERIFICATION Tota Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 Table 15b. Verification of Form 50058 59 Rent Components Third Party In Writing NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% C-17

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15c. Verification of Form 50058/59 Rent Components Third Party In Writing or EIV Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 363 (27.7%) 226 (17.2%) 724 (55.1%) 1,312 ) 760 (30.9%) 186 (7.6%) 1,511 (61.5%) 2,456 ) Public Assistance 146 (29.2%) 56 (11.3%) 296 (59.5%) 498 ) Other Income 383 (47.5%) 64 (8.0%) 359 (44.5%) 806 ) Asset Income 250 (36.0%) 38 (5.4%) 407 (58.6%) 694 ) 40 (31.3%) 19 (14.6%) 69 (54.1%) 127 ) 508 (52.6%) 78 (8.1%) 380 (39.3%) 966 ) HUD QC FY 2007 / Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 1,295 (98.7%) 3 (.2%) 15 (1.1%) 1,312 ) 2,452 (99.8%) 1 (.0%) 3 (.1%) 2,456 ) Public Assistance 488 (97.9%) 10 (2.1%) 498 ) Other Income 795 (98.7%) 11 (1.3%) 806 ) Asset Income 693 (99.8%) 1 (.2%) 694 ) 126 (99.0%) 1 (1.0%) 127 ) 961 (99.5%) 5 (.5%) 966 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 Table 15d. Verification of Form 50058 59 Rent Components Third Party - Verbal NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% C-18

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15e. Verification of Form 50058/59 Rent Components Documentation Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 1,123 (85.6%) 61 (4.7%) 128 (9.7%) 1,312 ) 1,941 (79.0%) 73 (3.0%) 442 (18.0%) 2,456 ) Public Assistance 416 (83.5%) 17 (3.4%) 65 (13.1%) 498 ) Other Income 648 (80.4%) 41 (5.1%) 117 (14.5%) 806 ) Asset Income 553 (79.7%) 17 (2.5%) 124 (17.8%) 694 ) 104 (81.4%) 9 (7.0%) 15 (11.5%) 127 ) 792 (82.0%) 33 (3.4%) 141 (14.6%) 966 ) HUD QC FY 2007 / ) Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 1,256 (95.7%) 41 (3.1%) 16 (1.2%) 1,312 ) 1,975 (80.4%) 64 (2.6%) 418 (17.0%) 2,456 ) Public Assistance 490 (98.4%) 8 (1.6%) 498 ) Other Income 803 (99.7%) 3 (.3%) 806 ) Asset Income 694 ) 694 ) 127 ) 127 ) 930 (96.2%) 7 (.7%) 30 (3.1%) 966 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 Table 15f. Verification of Form 50058 59 Rent Components EIV (Enterprise Income Verification NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% (100.0% Medical Expense (100.0% C-19

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15g. Verification of Form 50058/59 Rent Components Third Party Verbal or In Writing, or Documentation, or EIV l Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 31 (9.1%) 98 (29.0%) 209 (61.9%) 338 ) 25 (4.6%) 67 (12.3%) 456 (83.2%) 548 ) Public Assistance 13 (11.0%) 23 (19.6%) 81 (69.4%) 116 ) Other Income 59 (34.7%) 21 (12.1%) 91 (53.2%) 171 ) Asset Income 21 (15.5%) 11 (8.0%) 103 (76.6%) 134 ) 4 (18.7%) 3 (15.6%) 14 (65.7%) 22 ) 15 (7.9%) 34 (18.3%) 136 (73.8%) 185 ) PHA-Administered Section 8 Earned Income 78 56 (11.0%) (5.7%) 135 120 (19.2%) (12.3%) 491 804 (69.8%) (82.0%) 703 980 ) ) Public Assistance 28 (10.6%) 41 (15.5%) 196 (73.8%) 266 ) Other Income 102 (22.3%) 66 (14.3%) 290 (63.3%) 458 ) Asset Income 10 (5.0%) 16 (8.5%) 167 (86.5%) 193 ) 8 (11.5%) 15 (23.0%) 44 (65.5%) 67 ) 9 (4.3%) 37 (16.8%) 174 (78.9%) 220 ) i Earned Income 20 (7.3%) 62 (22.9%) 189 (69.8%) 271 ) 27 (2.9%) 99 (10.6%) 802 (86.4%) 928 ) Public Assistance 12 (10.4%) 9 (8.0%) 94 (81.6%) 116 ) Other Income 43 (24.0%) 21 (12.0%) 113 (64.0%) 177 ) Asset Income 15 (4.1%) 38 (10.4%) 314 (85.5%) 367 ) 3 (8.2%) 9 (22.7%) 27 (69.1%) 39 ) 44 (7.9%) 100 (17.9%) 417 (74.2%) 561 ) NO VERIFICATION VERIFICATION Tota Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Owner-Admin stered (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 C-20

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15h. Verification of Form 50058/59 Rent Components Third Party In Writing Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 112 (33.1%) 58 (17.0%) 168 (49.9%) 338 ) 322 (58.7%) 25 (4.6%) 202 (36.8%) 548 ) Public Assistance 37 (31.9%) 18 (15.5%) 61 (52.6%) 116 ) Other Income 96 (56.0%) 12 (7.2%) 63 (36.8%) 171 ) Asset Income 49 (36.6%) 7 (5.2%) 78 (58.2%) 134 ) 7 (31.9%) 2 (10.1%) 13 (58.0%) 22 ) 110 (59.5%) 15 (7.9%) 60 (32.6%) 185 ) PHA-Administered Section 8 Earned Income 247 550 (35.1%) (56.1%) 75 36 (10.6%) (3.7%) 382 394 (54.3%) (40.2%) 703 980 ) ) Public Assistance 73 (27.6%) 31 (11.5%) 162 (60.8%) 266 ) Other Income 192 (41.9%) 39 (8.6%) 227 (49.6%) 458 ) Asset Income 80 (41.3%) 6 (3.2%) 107 (55.5%) 193 ) 17 (25.8%) 11 (15.8%) 39 (58.4%) 67 ) 120 (54.4%) 11 (4.9%) 90 (40.7%) 220 ) i Earned Income 71 (26.1%) 43 (15.8%) 157 (58.1%) 271 ) 445 (47.9%) 48 (5.2%) 435 (46.9%) 928 ) Public Assistance 43 (37.0%) 8 (6.5%) 65 (56.4%) 116 ) Other Income 98 (55.4%) 13 (7.2%) 66 (37.4%) 177 ) Asset Income 121 (32.9%) 24 (6.6%) 222 (60.5%) 367 ) 16 (40.5%) 6 (15.0%) 17 (44.5%) 39 ) 351 (62.6%) 39 (7.0%) 171 (30.5%) 561 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Owner-Admin stered (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% C-21

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15i. Verification of Form 50058/59 Rent Components Third Party In Writing or EIV Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 89 (26.4%) 75 (22.3%) 173 (51.3%) 338 ) 140 (25.6%) 51 (9.3%) 357 (65.1%) 548 ) Public Assistance 35 (30.4%) 18 (15.5%) 63 (54.2%) 116 ) Other Income 96 (56.0%) 12 (7.2%) 63 (36.8%) 171 ) Asset Income 49 (36.6%) 7 (5.2%) 78 (58.2%) 134 ) 7 (31.9%) 2 (10.1%) 13 (58.0%) 22 ) 84 (45.2%) 20 (11.0%) 81 (43.7%) 185 ) PHA-Administered Section 8 Earned Income 204 277 (29.1%) (28.2%) 107 79 (15.3%) (8.0%) 392 625 (55.7%) (63.8%) 703 980 ) ) Public Assistance 73 (27.6%) 31 (11.5%) 162 (60.8%) 266 ) Other Income 189 (41.3%) 39 (8.6%) 230 (50.1%) 458 ) Asset Income 80 (41.3%) 6 (3.2%) 107 (55.5%) 193 ) 17 (25.8%) 11 (15.8%) 39 (58.4%) 67 ) 93 (42.5%) 18 (8.4%) 108 (49.1%) 220 ) i Earned Income 69 (25.6%) 43 (15.8%) 159 (58.6%) 271 ) 343 (37.0%) 56 (6.1%) 529 (57.0%) 928 ) Public Assistance 37 (31.8%) 8 (6.5%) 71 (61.7%) 116 ) Other Income 98 (55.4%) 13 (7.2%) 66 (37.4%) 177 ) Asset Income 121 (32.9%) 24 (6.6%) 222 (60.5%) 367 ) 16 (40.5%) 6 (15.0%) 17 (44.5%) 39 ) 331 (59.0%) 39 (7.0%) 191 (34.1%) 561 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Owner-Admin stered (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 C-22

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15j. Verification of Form 50058/59 Rent Components Third Party - Verbal Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 335 (99.1%) 3 (.9%) 338 ) 547 (99.8%) 1 (.2%) 548 ) Public Assistance 116 ) 116 ) Other Income 167 (97.7%) 4 (2.3%) 171 ) Asset Income 134 ) 134 ) 22 ) 22 ) 184 (99.5%) 1 (.5%) 185 ) PHA-Administered Section 8 Earned Income 692 978 (98.4%) (99.8%) 3 (.4%) 8 2 (1.2%) (.2%) 703 980 ) ) Public Assistance 257 (96.7%) 9 (3.3%) 266 ) Other Income 455 (99.3%) 3 (.7%) 458 ) Asset Income 191 (99.2%) 1 (.8%) 193 ) 67 ) 67 ) 218 (99.0%) 2 (1.0%) 220 ) i Earned Income 268 (98.9%) 3 (1.1%) 271 ) 927 (99.9%) 1 (.1%) 928 ) Public Assistance 114 (98.6%) 2 (1.4%) 116 ) Other Income 173 (97.9%) 4 (2.1%) 177 ) Asset Income 367 ) 367 ) 37 (96.8%) 1 (3.2%) 39 ) 559 (99.7%) 2 (.3%) 561 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% (100.0% Medical Expense (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% (100.0% Medical Expense (100.0% Owner-Admin stered (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 C-23

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15k. Verification of Form 50058/59 Rent Components Documentation Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 290 (85.8%) 20 (5.8%) 28 (8.4%) 338 ) 473 (86.2%) 4 (.7%) 72 (13.1%) 548 ) Public Assistance 94 (80.6%) 5 (4.1%) 18 (15.2%) 116 ) Other Income 140 (82.0%) 8 (4.9%) 22 (13.1%) 171 ) Asset Income 111 (82.3%) 4 (2.7%) 20 (14.9%) 134 ) 19 (86.8%) 1 (5.5%) 2 (7.7%) 22 ) 165 (89.5%) 3 (1.8%) 16 (8.7%) 185 ) PHA-Administered Section 8 Earned Income 609 788 (86.6%) (80.4%) 22 37 (3.2%) (3.8%) 72 155 (10.2%) (15.8%) 703 980 ) ) Public Assistance 229 (86.3%) 11 (4.0%) 26 (9.7%) 266 ) Other Income 380 (82.9%) 24 (5.2%) 54 (11.8%) 458 ) Asset Income 141 (73.1%) 7 (3.7%) 45 (23.2%) 193 ) 57 (85.7%) 5 (7.1%) 5 (7.1%) 67 ) 168 (76.5%) 11 (4.8%) 41 (18.7%) 220 ) i Earned Income 224 (82.8%) 19 (7.1%) 27 (10.1%) 271 ) 680 (73.3%) 32 (3.5%) 216 (23.2%) 928 ) Public Assistance 93 (80.0%) 2 (1.5%) 21 (18.5%) 116 ) Other Income 128 (72.5%) 8 (4.8%) 40 (22.8%) 177 ) Asset Income 301 (82.1%) 6 (1.8%) 59 (16.1%) 367 ) 27 (71.0%) 3 (7.7%) 8 (21.4%) 39 ) 459 (81.7%) 19 (3.4%) 84 (14.9%) 561 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% Owner-Admin stered (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% Medical Expense (100.0% 2008.10.05 C-24

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 15l. Verification of Form 50058/59 Rent Components EIV (Enterprise Income Verification) Matched # of Cases # of Cases # of Cases # of Cases Cases Cases Cases Cases Earned Income 320 (94.6%) 13 (4.0%) 5 (1.4%) 338 ) 401 (73.2%) 22 (3.9%) 126 (22.9%) 548 ) Public Assistance 114 (98.5%) 2 (1.5%) 116 ) Other Income 171 ) 171 ) Asset Income 134 ) 134 ) 22 ) 22 ) 166 (89.8%) 3 (1.8%) 16 (8.4%) 185 ) PHA-Administered Section 8 Earned Income 667 735 (94.8%) (75.0%) 27 37 (3.9%) (3.8%) 10 208 (1.4%) (21.2%) 703 980 ) ) Public Assistance 266 ) 266 ) Other Income 456 (99.4%) 3 (.6%) 458 ) Asset Income 193 ) 193 ) 67 ) 67 ) 206 (93.8%) 3 (1.5%) 10 (4.6%) 220 ) i Earned Income 269 (99.4%) 2 (.6%) 271 ) 838 (90.3%) 5 (.5%) 85 (9.1%) 928 ) Public Assistance 110 (94.8%) 6 (5.2%) 116 ) Other Income 177 ) 177 ) Asset Income 367 ) 367 ) 39 ) 39 ) 557 (99.3%) 4 (.7%) 561 ) NO VERIFICATION VERIFICATION TOTAL Dollar Amount Not Dollar Amount Matched Row % of Row % of Row % of Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% (100.0% Medical Expense (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% (100.0% Medical Expense (100.0% Owner-Admin stered (100.0% Pension, Etc. (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% Child Care Expense (100.0% (100.0% Medical Expense (100.0% 2008.10.05 C-25

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 16a. QC Rent Component for Household with QC Rent Error (>$5) l # of Cases # of Cases # of Cases % of Cases % of Cases % of Cases Earned Income No Error 2,460 (87.5%) 1,236 (93.7%) 3,697 (89.4%) w/Error 353 (12.5%) 84 (6.3%) 436 (10.6%) No Error 2,468 (87.7%) 1,119 (84.8%) 3,587 (86.8%) w/Error 345 (12.3%) 201 (15.2%) 546 (13.2%) Public Assistance No Error 2,733 (97.2%) 1,298 (98.3%) 4,031 (97.5%) w/Error 80 (2.8%) 22 (1.7%) 102 (2.5%) Other Income No Error 2,623 (93.3%) 1,264 (95.8%) 3,888 (94.1%) w/Error 190 (6.7%) 56 (4.2%) 245 (5.9%) Asset Income No Error 2,686 (95.5%) 1,177 (89.1%) 3,863 (93.5%) w/Error 127 (4.5%) 143 (10.9%) 270 (6.5%) No Error w/Error 2,751 62 (97.8%) (2.2%) 1,298 22 (98.3%) (1.7%) 4,049 84 (98.0%) (2.0%) No Error 2,811 (99.9%) 1,320 ) 4,131 (99.9%) w/Error 2 (.1%) 2 (.1%) No Error 2,517 (89.5%) 1,061 (80.4%) 3,579 (86.6%) w/Error 296 (10.5%) 259 (19.6%) 554 (13.4%) No Error 1,856 (66.0%) 864 (65.4%) 2,720 (65.8%) w/Error 957 (34.0%) 456 (34.6%) 1,413 (34.2%) l 2,813 ) 1,320 ) 4,133 ) 50058 50059 Tota (in 1,000) (in 1,000) (in 1,000) Pensions, Etc. Child Care Expense Disability Expense (100.0% Medical Expense All Components Tota (100.0% (100.0% (100.0% 2008.10.05 C-26

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007

Table 16b. QC Error Cases with Missing Verification in Tenant File

l # of Cases # of Cases # of Cases % of Cases % of Cases % of Cases Earned Income Verified 90 (25.6%) 24 (28.8%) 114 (26.2%) Not Verified 262 (74.4%) 60 (71.2%) 322 (73.8%) Verified 51 (14.7%) 38 (19.1%) 89 (16.3%) Not Verified 294 (85.3%) 163 (80.9%) 457 (83.7%) Public Assistance Verified 33 (41.5%) 3 (15.3%) 37 (35.7%) Not Verified 47 (58.5%) 19 (84.7%) 66 (64.3%) Other Income Verified 47 (25.0%) 10 (18.6%) 58 (23.5%) Not Verified 142 (75.0%) 45 (81.4%) 187 (76.5%) Asset Income Verified 24 (19.3%) 33 (23.3%) 58 (21.4%) Not Verified 102 (80.7%) 110 (76.7%) 212 (78.6%) Verified Not Verified 11 51 (17.8%) (82.2%) 6 16 (29.0%) (71.0%) 17 67 (20.8%) (79.2%) Not Verified 2 ) 2 ) Verified 28 (9.6%) 29 (11.1%) 57 (10.3%) Not Verified 267 (90.4%) 230 (88.9%) 497 (89.7%) 50058 50059 Tota (in 1,000) (in 1,000) (in 1,000) Pension, Etc. Child Care Expense Disability Expense (100.0% (100.0% Medical Expense C-27

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 17. Administrative Error: Number & Percent of Households, Average Dollars in Error For Households with Recalculated 50058/59 Rent Error and Households with QC Rent Error by Administrative Error Type Households wi Households wi % 110 (39.6%) 23.61 927 (62.3%) 47.58 i 167 (60.4%) 16.39 560 (37.7%) 37.84 88 (31.7%) 32.90 338 (22.7%) 42.57 189 (68.3%) 12.91 1,149 (77.3%) 44.31 13 (4.7%) 88.50 77 (5.1%) 43.43 Error 264 (95.3%) 15.81 1,410 (94.9%) 43.94 11 (3.9%) 93.69 53 (3.6%) 33.83 No Income Calculation Error 266 (96.1%) 16.21 1,434 (96.4%) 44.29 32 (11.7%) 44.07 124 (8.4%) 68.12 244 (88.3%) 15.95 1,362 (91.6%) 41.70 7 (2.5%) 78.39 47 (3.2%) 78.40 226 (81.6%) 18.27 1,229 (82.7%) 42.37 44 (16.0%) 15.15 210 (14.1%) 45.19 162 (58.5%) 23.62 1,065 (71.7%) 46.32 115 (41.5%) 13.08 421 (28.3%) 37.83 277 ) 19.25 1,487 ) 43.91 th Recalculated 50058/59 Rent Error th QC Rent Error # of Households in Error # of Households in Error (in 1,000) of Households in Error Average Gross Dollar Error (in 1,000) % of Households in Error Average Gross Dollar Error Transcription Error No Transcr ption Error Consistency Error No Consistency Error Allowances Calculation Error No Allowances Calculation Income Calculation Error Other Calculation Error No Othere Calculation Error Overdue Recertification On-time Recertification Certification Any Admin/proc Error No Admin/proc Error Total Households (100.0% (100.0% 2008.10.05 C-28

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 18. Administrative Error: Number & Percent of Households, Average Dollars in Error For All Households by Administrative Error Type 1,579 (38.2%) 28.35 1,579 (38.2%) -9.95 i 2,554 (61.8%) 8.54 2,554 (61.8%) -2.37 838 (20.3%) 17.42 838 (20.3%) -2.64 3,295 (79.7%) 15.77 3,295 (79.7%) -5.93 127 (3.1%) 26.53 127 (3.1%) -.23 Error 4,006 (96.9%) 15.78 4,006 (96.9%) -5.42 113 (2.7%) 16.41 113 (2.7%) -3.80 No Income Calculation Error 4,020 (97.3%) 16.10 4,020 (97.3%) -5.30 252 (6.1%) 33.90 252 (6.1%) -14.60 3,881 (93.9%) 14.95 3,881 (93.9%) -4.66 64 (1.5%) 58.93 64 (1.5%) -47.36 3,429 (83.0%) 15.50 3,429 (83.0%) -5.06 641 (15.5%) 15.10 641 (15.5%) -2.18 2,092 (50.6%) 23.97 2,092 (50.6%) -8.31 2,041 (49.4%) 8.04 2,041 (49.4%) -2.14 Total 4,133 ) 16.11 4,133 ) -5.26 Gross QC Rent Error Net QC Rent Error # of Households (in 1,000) % of Households Average Dollar Error # of Households (in 1,000) % of Households Average Dollar Error Transcription Error No Transcr ption Error Consistency Error No Consistency Error Allowances Calculation Error No Allowances Calculation Income Calculation Error Other Calculation Error No Othere Calculation Error Overdue Recertification On-time Recertification Certification Any Admin/proc Error No Admin/proc Error (100.0% (100.0% 2008.10.05 C-29

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 19. Occupancy Standards on Form 50058/59 PUBLIC HOUSING l # of Cases # of Cases # of Cases # of Cases % of Cases % of Cases % of Cases % of Cases 0 2 (3.2%) 4 (10.1%) 7 (3.7%) 1 2 (.7%) 1 (.3%) 2 (.2%) 5 (.3%) 2 4 (1.4%) 8 (1.1%) 3 (1.0%) 14 (1.1%) 3 2 (.7%) 18 (3.3%) 20 (2.2%) 4 2 (4.5%) 4 (3.9%) 7 (3.8%) 5+ 1 (9.4%) 1 (2.6%) All Units 13 (1.4%) 35 (1.9%) 4 (.3%) 53 (1.3%) CORRECT 0 65 (96.8%) 40 (89.9%) 66 ) 171 (96.3%) 1 327 (99.3%) 442 (99.7%) 817 (99.8%) 1,586 (99.7%) 2 200 (71.1%) 509 (74.1%) 211 (75.4%) 920 (73.7%) 3 171 (80.5%) 430 (77.4%) 129 (92.6%) 731 (80.5%) 4 30 (55.2%) 60 (58.1%) 6 (43.8%) 96 (56.1%) 5+ 2 (19.2%) 14 (57.6%) 16 (43.4%) All Units 795 (83.3%) 1,495 (80.5%) 1,229 (93.1%) 3,520 (85.2%) OVER-HOUSED 2 77 (27.5%) 170 (24.8%) 66 (23.6%) 314 (25.1%) 3 40 (18.8%) 107 (19.3%) 10 (7.4%) 158 (17.4%) 4 22 (40.3%) 39 (38.0%) 7 (56.2%) 69 (40.1%) 5+ 7 (71.4%) 10 (42.4%) 2 ) 20 (54.0%) All Units 146 (15.3%) 327 (17.6%) 86 (6.5%) 560 (13.5%) PHA-ADMINISTERED SECTION 8 OWNER-ADMINISTERED Tota (in 1,000) (in 1,000) (in 1,000) (in 1,000) UNDER-HOUSED (100.0% (100.0% 2008.10.05 C-30

Appendix C—Source Tables Based on Quality Control Data HUD QC FY 2007 Table 19a. Frequency & Percent of All Households by Number of Bedrooms and Number of Household Members 1 2 3 4 5 6 7 8 9 10 12 N % N % N % N % N % N % N % N % N % N % N % 0 171 96.3% 7 3.7% 1 90.0% 154 9.7% 4 .2% 1 .1% 2 314 25.1% 566 45.4% 254 20.4% 100 8.0% 14 1.1% 3 55 6.1% 103 11.3% 260 28.6% 276 30.4% 152 16.8% 42 4.7% 14 1.5% 5 .6% 1 .1% 4 1 .6% 3 2.0% 10 6.0% 54 31.5% 46 27.0% 20 11.8% 22 12.7 % 8 4.5% 1 .7% 5 3.1% 5+ 3 8.2% 2 6.0% 4 10.0% 6 15.8% 2 5.3% 3 8.6% 4 11.0 % 12 32.4 % 1 2.6% Number of Household Members (in 1,000) 1432 2008.10.05 C-31

Appendix C—Source Tables Based on Tenant File Data

Appendix C—Source Tables Based on Tenant File Data HUD QC FY 2007 [Tenant File]

Table 2. Percent of Households by Payment Type and Program Type

PROPER PAYMENT OVERPAYMENT l # of # of # of # of Cases of Col % of Cases of Col % of Cases (in of Col % of Cases (in Col % of Cases Cases Cases Cases 1,000) Cases Cases 1,000) Cases Cases OWNER Section 8 l 116 199 315 132 (12.1%) (10.7%) (11.2%) (10.0%) (25.9%) (44.6%) (70.5%) (29.5%) 775 1,387 2,162 1,066 (81.1%) (74.7%) (76.9%) (80.8%) (24.0%) (43.0%) (67.0%) (33.0%) 64 272 336 122 (6.8%) (14.6%) (11.9%) (9.2%) (14.1%) (59.3%) (73.4%) (26.6%) 955 1,858 2,813 1,320 ) ) ) ) (23.1%) (45.0%) (68.1%) (31.9%) l 132 (10.0%) (29.5%) 1,066 (80.8%) (33.0%) 122 (9.2%) (26.6%) 1,320 ) (31.9%) l 446 (10.8%) ) 3,229 (78.1%) ) 458 (11.1%) ) 4,133 ) ) HUD QC FY 2007 [Tenant File] ( PROPER PAYMENT OVERPAYMENT l # of # of # of # of Cases Col % of Cases Col % of Cases Col % of Cases Col % of of Cases Cases of Cases Cases of Cases Cases Cases Cases PHA OWNER Section 8 l 148 250 398 207 (15.5%) (13.5%) (14.2%) (15.7%) (24.4%) (41.4%) (65.8%) (34.2%) 683 1,199 1,883 911 (71.6%) (64.5%) (66.9%) (69.0%) (24.5%) (42.9%) (67.4%) (32.6%) 124 408 532 202 (13.0%) (22.0%) (18.9%) (15.3%) (16.9%) (55.7%) (72.5%) (27.5%) 955 1,858 2,813 1,320 ) ) ) ) (23.1%) (45.0%) (68.1%) (31.9%) l l 207 605 (15.7%) (14.6%) (34.2%) ) 911 2,794 (69.0%) (67.6%) (32.6%) ) 202 734 (15.3%) (17.8%) (27.5%) ) 1,320 4,133 ) ) (31.9%) ) UNDERPAYMENT Tota Row % Row % Row % Row % of (in 1,000) (in 1,000) PHA ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered (100.0% (100.0% (100.0% (100.0% Tota (100.0% Tota (100.0% (100.0% (100.0% (100.0% (100.0% 2008.10.05 Table 2 S). Percent of Households by Payment Type and Program Type (Proper Payment based on exact match of Actual and QC Rent) UNDERPAYMENT Tota Row % Row % Row % Row % of (in 1,000) (in 1,000) (in 1,000) (in 1,000) ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered (100.0% (100.0% (100.0% (100.0% Tota Tota (100.0% (100.0% (100.0% (100.0% (100.0% (100.0% 2008.10.05 C-1Tenant File

Appendix C—Source Tables Based on Tenant File Data HUD QC FY 2007 [Tenant File]

Table 3. Dollar Rent Error by Program Type

ACTUAL RENT (MONTHLY) DC RENT (MONTHLY) GROSS RENT ERROR (MONTHLY) Sum Sum Sum Dollar Ave. # of Dollar Ave. Dollar Ave. # of Cases Col % of Amount Dollar Cases Col % of Amount Dollar # of Cases Col % of Amount Dollar (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases (in 1,000) Amount Public 955 (23.1%) 218,505 228.80 955 (23.1%) 222,809 233.31 955 (23.1%) 13,656 14.30 Section 8 PHA 1,858 (45.0%) 1,858 (45.0%) 1,858 (45.0%) 17.04 l 2,813 (68.1%) 2,813 (68.1%) 2,813 (68.1%) 16.11 OWNER 1,320 (31.9%) 1,320 (31.9%) 1,320 (31.9%) 9.08 l 1,320 (31.9%) 1,320 (31.9%) 1,320 (31.9%) 9.08 l 4,133 ) 4,133 ) 4,133 ) 13.86 Housing ADMINISTERED 392,125 211.05 379,798 204.41 31,657 Tota 610,631 217.07 602,607 214.22 45,313 Owner- Administered ADMINISTERED 276,013 209.10 277,228 210.02 11,990 Tota 276,013 209.10 277,228 210.02 11,990 Tota (100.0% 886,643 214.53 (100.0% 879,835 212.88 (100.0% 57,302 2008.10.05 C-2Tenant File

Appendix C—Source Tables Based on Tenant File Data HUD QC FY 2007 [Tenant File]

Table 4. Dollar Error Amount by Payment Type and Program Type

UNDERPAYMENT (MONTHLY) OVERPAYMENT (MONTHLY) DC RENT (MONTHLY) Sum Sum Sum # of Dollar Ave. Dollar Ave. Dollar Ave. Cases Col % of Amount Dollar # of Cases Col % of Amount Dollar # of Cases Col % of Amount Dollar (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases (in 1,000) Amount (in 1,000) Cases (in 1,000) Amount PHA OWNER Section 8 l 116 199 315 132 (25.9%) (44.6%) (70.5%) (29.5%) 8,824 9,734 6,593 76.36 48.86 58.96 50.08 64 272 336 122 (14.1%) (59.3%) (73.4%) (26.6%) 4,832 5,397 74.94 80.74 79.62 44.26 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 132 446 (29.5%) ) 6,593 50.08 56.34 122 458 (26.6%) ) 5,397 44.26 70.21 1,320 4,133 (31.9%) ) HUD QC FY 2007 [Tenant File] ( Sum Sum Sum # of Amount Ave. # of Ave. # of Ave. Cases Col % of (in Cases Col % of Amount Cases (in Col % of Amount (in 1,000) Cases 1,000) Amount (in 1,000) Cases Amount 1,000) Cases Amount PHA OWNER Section 8 l 148 250 398 207 (24.4%) (41.4%) (65.8%) (34.2%) 8,891 9,838 6,789 60.15 39.30 47.04 32.79 124 408 532 202 (16.9%) (55.7%) (72.5%) (27.5%) 4,923 5,574 39.78 54.28 50.91 27.65 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 207 605 (34.2%) ) 6,789 32.79 42.16 202 734 (27.5%) ) 5,574 27.65 44.52 1,320 4,133 (31.9%) ) ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered 18,558 21,923 26,755 222,809 379,798 602,607 277,228 233.31 204.41 214.22 210.02 Tota Tota (100.0% 25,150 (100.0% 32,152 (100.0% 277,228 879,835 210.02 212.88 2008.10.05 Table 4 S). Dollar Error Amount by Payment Type and Program Type (Proper Payment based on exact match of Actual and QC Rent) UNDERPAYMENT (MONTHLY) OVERPAYMENT (MONTHLY) DC RENT (MONTHLY) Dollar Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) ADMINISTERED ADMINISTERED Public Housing Tota Owner- Administered 18,729 22,165 27,088 222,809 379,798 602,607 277,228 233.31 204.41 214.22 210.02 Tota Tota (100.0% 25,519 (100.0% 32,662 (100.0% 277,228 879,835 210.02 212.88 2008.10.05 C-3Tenant File

Appendix C—Source Tables Based on Tenant File Data HUD QC FY 2007 [Tenant File]

Table 5. Gross and Net Rent Error by Program Type

Sum Sum # of Ave. # of Ave. Ave. Cases Col % of Amount Cases Col % of Amount # of Cases Col % of Amount Cases Amount Cases Amount Cases Amount Section 8 PHA l OWNER 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) 14.30 17.04 16.11 9.08 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) -3,992 8,197 -1,195 -4.18 6.56 2.91 -.91 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 1,320 4,133 (31.9%) ) 9.08 13.86 1,320 4,133 (31.9%) ) -1,195 7,002 -.91 1.69 1,320 4,133 (31.9%) ) HUD QC FY 2007 [Tenant File] ( Sum Sum Sum Ave. Ave. Ave. # of Cases Col % of Amount # of Cases Col % of Amount # of Cases Col % of Amount Cases Amount Cases Amount Cases Amount Section 8 PHA l OWNER 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) 14.46 17.22 16.29 9.37 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) -3,968 8,359 -1,216 -4.16 6.63 2.97 -.92 955 1,858 2,813 1,320 (23.1%) (45.0%) (68.1%) (31.9%) l l 1,320 4,133 (31.9%) ) 9.37 14.08 1,320 4,133 (31.9%) ) -1,216 7,143 -.92 1.73 1,320 4,133 (31.9%) ) GROSS RENT ERROR (MONTHLY) NET RENT ERROR (MONTHLY) DC RENT (MONTHLY) Dollar Sum Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing ADMINISTERED Tota Owner- Administered ADMINISTERED 13,656 31,657 45,313 11,990 12,189 222,809 379,798 602,607 277,228 233.31 204.41 214.22 210.02 Tota Tota (100.0% 11,990 57,302 (100.0% (100.0% 277,228 879,835 210.02 212.88 2008.10.05 Table 5 S). Gross and Net Rent Error by Program Type (Proper Payment based on exact match of Actual and QC Rent) GROSS RENT ERROR (MONTHLY) NET RENT ERROR (MONTHLY) DC RENT (MONTHLY) Dollar Dollar Dollar Dollar Dollar Dollar (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) (in 1,000) Public Housing ADMINISTERED Tota Owner- Administered ADMINISTERED 13,814 32,003 45,817 12,363 12,327 222,809 379,798 602,607 277,228 233.31 204.41 214.22 210.02 Tota Tota (100.0% 12,363 58,180 (100.0% (100.0% 277,228 879,835 210.02 212.88 2008.10.05 C-4Tenant File

Appendix D—Consistency and Calculation Errors

Appendix D––Consistency and Calculation Errors

50058—Consistency Errors 50058 ITEM ERROR General Information: 1c. Program Must equal P, CE, VO, or MR

2a. Type of Action Must equal 1 through 15

2b. Effective Date of Action Cannot be earlier than Date of Admission to the Program

(2h)

Household Composition: 3g. Sex 3h. Relationship 3i. Citizenship 3k. Race 3m. Ethnicity 3u. Family Subsidy Status 3v. Effective Date Must equal M or F

Must equal H, S, K, F, Y, E, L, or A

Must equal EC, EN, IN, PV, or XX

Must equal 1 through 4

Must equal 1 or 2

Must equal C, E, F, P, or blank

Should not be blank if 3u equals C

Net Family Assets and Income 6a. Family Member No. 7a. Family Member No. 7b. Income Code 8a. Total Annual Income 8i. Earnings Made Possible by Disability Assistance Expense Must equal a number used in Section 3. Household.

Must equal a number used in Section 3. Household.

Must equal B, F, HA, M, W, G, IW, T, P, S, SS, C, E, I, N,

or U

Must equal Total Annual Income recorded in 7i

Must be <= the sum of Dollars per Year (7d) for Income

Codes (7b) HA, F, W, B, or M

Allowances and Adjusted Income 8h. Maximum Disability Allowance 8j. Allowable Disability Assistance Expense Should only be completed if any member is disabled

• Should be <= Maximum Disability Allowance (8h) • Should be 0 if Medical/Disability Threshold (8f) is > Maximum Disability Allowance (8h) • Should be 0 or blank if Maximum Disability Allowance (8h) is 0 or blank 8k. Total Medical Expenses Should only be completed if the head, spouse, or co-head is 62 or over, or disabled; otherwise it should be blank D-1

Appendix D––Consistency and Calculation Errors 50058 ITEM ERROR 8n. Medical/Disability Assistance • Should equal Total Annual Disability Assistance and Allowance Medical Expense (8m) minus Medical/disability Threshold (8f) if Allowable Disability Expense (8j) is blank or Total Annual Unreimbursed Disability Assistance Expense (8g) is less than Medical /disability Threshold (8f) • Should equal Total Annual Disability Assistance and Medical Expense (8m) if 8 Total Annual Unreimbursed Disability Assistance Expense (8g) and Allowable Disability Expense (8j) is >= Medical/disability Threshold (8f) 8p. Elderly/Disabled Allowance Should be $400 if head, spouse or co-head is 62 or over, or disabled; otherwise it should be 0 or blank 8s. Dependent Allowance Must be completed if the household contains a member

under age 18, disabled, or a full-time student (excluding

the head, spouse, foster child or adult, or live-in

attendant)

8t. Yearly Child Care Cost That Is Not Should only be completed if any member is less than 13

Reimbursed (Child Care Allowance) years old

Family Rent and Subsidy Information 10a. 11q, 12r, 13j, 14s TTP Must equal TTP (9j) or blank 10a. through 14ag. Rent Calculations • If Program (1c) = P: - TTP (10a), must be completed; - Flat Rent (10b), or Tenant Rent (10f), or Mixed Family Tenant Rent (10s) must be completed; - Section 11 through 14 must be blank. • If Program (1c) = VO or C: - Section 11, or 12 must be completed - Tenant Rent (11s or 12k), or Mixed Family Tenant Rent (11ak, or 12 ai) must be completed; - Section 10, 13, and 14 must be blank • If Program (1c) = MR: - Contract Rent to Owner must be completed - Tenant Rent (13k), or Mixed Family Tenant Rent (13x) must be completed; - Sections 10, 11, 12, and 14 must be blank. D-2

Appendix D––Consistency/Calculations

50059 - Consistency Errors 50059 ITEM ERROR General Information: 2. Subsidy Type Must equal 1 through 9 13. Effective Date Cannot be earlier than Date Tenant Moved into Project (16.) 18. Certification Type Must equal 1 through 5 19. Action Processed Must equal 1 through 4, or blank 44. Race of Head of Household Must equal 1 through 4 45. Ethnicity of Head of Household Must equal 1 or 2 Household Composition 43. Sex Must equal M or F 47. Special Status Code Must equal E, S, H, F, I, J, or blank; should be E if Age > 61 49. Eligibility Code (Citizenship) Must equal EC, EN, IC, IN, IP, PV, or XX Net Family Assets and Income 69. Member No. – Income Info Should not be greater than the total number of members 78. Member No. – Asset Info listed in item 38 (Family Member Number) Allowances and Adjusted Income 100. Dependent Allowance 101. Child Care Expense (work) 102 Child Care Expense (school) 105. Disability Allowance Must be completed if Number of Dependents (58) is greater than 0 Should only be completed if any member is less than 13 years old • Should be <= Disability Expenses (104) • Should be 0 if 3% of Annual Income (103) is > Total Disability Assistance Expenses (104) • Should be 0 or blank if Total Disability Assistance Expenses (104) is 0 or blank 106. Total Medical Expenses Should only be completed if the Special Status Code (47) for the head or spouse or co-head = H or E, or if the head, spouse, or co-head is age 62 years old or older 108. Elderly Household Allowance Should be $400 if the Special Status Code (47) for the head or spouse or co-head = H or E; otherwise it should be 0 or blank Family Rent and Subsidy Information: 112. Tenant Rent Should equal the maximum of TTP (111) minus Utility Allowance (33) or 0; or be blank if Utility Reimbursement (113) is greater than 0 113. Utility Reimbursement Should be blank if Item 33 < Item 111 D-3

Appendix D––Consistency/Calculations 50058 - Calculation Errors 50058 ITEM ERROR CALCULATION Household Composition: 3f. Age 8q. Number of Dependents Net Family Assets and Income 6f. Total Asset Value 6i. Imputed Asset Income 6j. Income from Asset 7g. Total Non Asset Income 7i. Total Annual Income Allowances and Adjusted Income 8e. Total Permissible Deductions 8f. 3% of Annual Income 8h. Disability Allowance 8n. Medical Allowance 8p. Elderly/Disabled 8s. Dependent Allowance 8t. Child Care Costs Must equal the age calculated based on Date of Birth (3e) and Effective Date of Action (2b) Must equal the number of household members under 18, with a disability, or a full-time student (other than head, spouse co-head, foster child/adult, or live-in aide) Must equal the sum of all values in Cash Value of Asset (6d) Must equal Total Cash Value of Asset (6f) * Passbook Rate (6h) if Total Value of Assets (6f) is > $5,000. If Total Value of Assets (6f) is <= $5,000 Imputed Asset Income (6i) = 0 Must equal the larger of Total Anticipated Income (6g) or Imputed Asset Income (6i) Must equal the sum of all values in Income After Exclusions (7f) Must equal (Final Asset Income (6j) + Total Income Other Than Assets (7g) Must equal the sum of all values in Amount of Permissible deduction (8d) Must equal 3% * Total Annual Income (8a) Must equal Total Annual Unreimbursed Disability Assistance Expense (8g) minus Medical/Disability Threshold (8f) if there is a disabled household member, and if there is earned income greater than or equal to the disability expense Must equal: Total Annual Disability Assistance and Medical Expense (8m) minus Medical/disability Threshold (8f) if Allowable Disability Assistance Expense (8j) is blank or Total Annual Unreimbursed Disability Assistance Expense (8g) is less than Medical/disability Threshold (8f); or equal Total Annual Disability Assistance and Medical Expense (8m) if Total Annual Unreimbursed Disability Assistance Expense (8g) and Allowable Disability Assistance Expense (8j) is >= Medical/Disability Threshold (8f); if the head, spouse, or co-head is elderly or disabled Must equal $400 if head, spouse, or co-head is elderly or disabled Must equal Number of Dependents (8q) * $480 Must be 0 or blank, if no household member under age 13 D-4

Appendix D––Consistency/Calculations 50058 ITEM ERROR CALCULATION

8x. Total Allowance Must equal Total Permissible Deductions (8e) + Medical /Disability Assistance Allowance (8n) + Elderly/Disability Allowance (8p) + Dependent Allowance (8s) + Total Annual Unreimbursed Childcare Costs (8t) + Total Annual Travel Cost to Work/School (8u) 8y. Adjusted Annual Income Must equal Total Annual Income (8a) minus Total

Allowances (8x)

Family Rent and Subsidy Information 9j. Total Tenant Payment Must equal the highest of TTP if Based on Annual Income (9c), TTP if Based on Adjusted Annual Income (9f), Welfare Rent (9g), Minimum Rent (9h), or Enhanced Voucher Minimum Rent (9i). 12p. Gross Rent Must equal Rent to Owner (12k) + Utility Allowance (12m) Tenant Rent (item number varies by Tenant Rent must equal the recalculated tenant rent based program) on the Rent Calculation rules provided in Appendix A Note: With the exception of tenant rent, negative numbers are always converted to 0 D-5

Appendix D––Consistency/Calculations 50059 - Calculation Errors 50059 ITEM ERROR CALCULATION Household Composition: 51. Age 56. Number of Family Members 57. Number of Non-family Members 58. Number of Dependents Net Family Assets and Income 84. Total Asset Value 85. Actual Income From Asset 87. Imputed Asset Income 73. Earned Income Sum 74. Pension Income Sum 75. Public Assistance Income Sum 76. Other Income Sum 77. Total Non Asset Income 88. Asset Income 89. Total Annual Income Allowances and Adjusted Income 100. Dependent Allowance 101. Child Care Expense (work) 102. Child Care Expense (school) 103. 3% of Annual Income 105. Disability Allowance Must equal age calculated based on Date of Birth (46) and Effective Date of Action (13) Must equal the number of family members listed Must equal the number of family members listed with a relationship code of “L” or “F” Must equal the number of household members under 18, with a disability, or a full-time student (other than head, spouse co-head, foster child/adult, or live-in aide) Must equal the sum of the asset values in Cash Value of Assets (81) Must equal the sum of the income values in Actual Yearly Income From Assets (82) Must equal Total Asset Value (84) * 2%, if Total Value of Assets is > $5,000 Must equal the sum of income values (in item 71) for items with codes B, F, M, or W in Income Type Code (70) Must equal the sum of the income values (in item 71) for items with codes PE, SI, or SS in Income Type Code (70) Must equal the sum of the income values (in item 71) for items with codes TA or G in Income Type Code (70) Must equal the sum of the income values (in item 71) for items with codes CS, I, N, or U in Income Type Code (70) Must equal Earned Income Sum (73) + Pension Income Sum (74) + Public Assistance Income Sum (75) + Other Income Sum (76) Must equal the greater of Imputed Asset Income (87) or Actual Income from Asset (85) Must equal Total Non Asset Income (77) + Income from Asset (88) Must equal Number of Dependents (58) * $480 Must be 0 or blank, if no household member under age 13 Must equal Total Annual Income (89) * .03 Must equal Total Disability Expenses (104) minus 3% of Annual Income (103) if there is a disabled household member, and if there is earned income greater than or equal to the disability expense D-6

Appendix D––Consistency/Calculations 50059 ITEM ERROR CALCULATION

107. Medical Allowance Must equal Total Medical Expenses (106) minus 3% of Annual Income (103) if Total Handicapped Assistance Expense (107a) = 0; or if (Disability Allowance (105) = 0, then Medical Allowance (106) = Total Medical Expenses (106) + Total Handicapped Assistance Expenses (104) –3% of Annual Income (89), if the head, spouse, or co-head is elderly or disabled 108 Elderly Household Allowance Must equal $400 if head, spouse, or co-head is elderly or

disabled

109.Total Allowance Must equal Allowance for Dependents (100) + Child Care

Allowance (101+102) + Allowance for Disability Expenses

(105) + Allowance for Medical Expenses (107) + Elderly

Household Allowance (108)

110. Adjusted Annual Income Must equal Total Annual Income (89) minus Total

Allowances (109)

Family Rent and Subsidy Information 34. Gross Rent Must equal Contract Rent (32) + Utility Allowance (33) 111. Total Tenant Payment Must equal the higher of 30% of Adjusted Income (110),

10% of Total Annual Income (89), Welfare Rent (115), or

$50 (Minimum Rent).

112. Tenant Rent Tenant Rent must equal the recalculated tenant rent based on the Rent Calculation rules provided in Appendix A Note: With the exception of tenant rent, negative numbers are always converted to 0 D-7

Appendix E—Project Staff Questionnaire Descriptive Analysis

Appendix E – Project Staff Questionnaire Descriptive Analysis The Project Staff Questionnaire (PSQ) is a self-administered survey given to project managers and executive directors of PHA/projects 1 included in the FY 2007 study. The objective of the questionnaire was to obtain information regarding 1) project characteristics and practices that promote accurate (re)certifications, and 2) identify difficulties experienced by PHAs/projects in order to identify areas of potential improvement. Topics covered the number and types of PHA/project staff, training received for new (re)certification staff and those with experience, procedures for communicating information from HUD, quality control measures for verifying work done by (re)certification staff, methods and difficulties in obtaining household information, utilization of computer software to automate the (re)certification process, and procedures and difficulties in verifying tenant information. A. Methodology The PSQ, a self-administered survey, was mailed in February 2008 to the executive director or manager of each PHA/Project, and respondents mailed their completed questionnaires back to Macro headquarters. Data were entered into an electronic data base via an automated tool that programmed in skip patterns, missing items, and range of valid responses. PSQs with questionable responses or skip patterns were individually investigated and all of the data issues were resolved. Of the 555 PSQs that were originally mailed, 546 were completed and returned, for a response rate of 98.5 percent. This year’s questionnaire expanded and modified several sections. New open-ended questions relating to changes in policy and procedures were added, as well as questions relating to procedures used to complete the annual recertification process. A new optional section was included asking about PHAs’ policy regarding interim recertifications. B. Results Number and Type of Staff. (Re) certification staffs are those who interview the tenants, gather information from them, calculate rents, track verifications, and supervise other staff in performing move-in certifications and annual recertification’s. In FY 2007, PHA/projects had on average 145 units per staff member responsible for certifying and recertifying households. Owner-administered projects had the lowest unit to staff ratio (79 units per staff member), while PHA-administered Section 8 projects had the highest unit to staff ratio (234 units per staff member). Public Housing projects were in the middle with a ratio of 146 units per staff member. Exhibit E-1a shows the average and median number of units per type of staff member, by program type. Exhibit E-1a also shows the ratio of households to all staff members at the PHA/Project (e.g., administrative staff, maintenance staff). 1 j For purposes of this study, a PHA-administered Section 8 pro ect is defined as a PHA/county combination representing the Section 8 Voucher households living in a particular county. E-1

Appendix E – Project Staff Questionnaire Descriptive Analysis Exhibit E-1a. Number of Units per Staff Member, by Program Type Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total Units per (re)certification staff 146.1 234.3 79.2 145.2 Average ratio Median ratio 112.8 209.9 65.0 103.5 Units per staff member 44.2 105.7 30.2 55.5 Average ratio Median ratio 27.9 96.7 26.0 32.9 200 146 199 545 on (re)certifications. j have ( ) Exhibit E-1b breaks Exhibit E-1b.

(Re)

i ience 88.3% 82.4% 89.4% 87.1% i 75.4% 56.5% 80.2% 71.7% Total Number of PHA/projects Of those staff members, a majority of them worked on a variety of tasks in addition to working PHA-administered Section 8 pro ects were most likely to have staff work primarily on recertifications, with 25 percent of staff working mostly on (re)certifications, and owner-administered projects were most likely to their staff multi-task on other responsibilities, with only 7 percent working mostly on recertifications. Most PHA/projects had staff with more than one year of experience working with (re)certifications. All three program types had similar percentages of re certification staff with over one year of experience. However, when looking at the percentage of staff with over 5 years of experience, PHA-administered Section 8 projects drop down to 56.5 percent, while both Public Housing and owner-Administered projects stay above 75 percent. down the percentages by program type. Percentage of Staff Who Have Worked with certifications for Over 1 Year and 5 Years, by Program Type Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total (Re) certification staff w th over 1 year exper (Re) certification staff w th over 5 years experience When hiring new staff who will be working with (re)certifications, most PHAs/projects had some minimum education requirements. Overall, only about 7 percent of PHAs/projects did not require some education. Owner-administered projects were most likely not to require any education at about 11 percent, while PHA-administered Section 8 projects were least likely at 3 percent. E-2

Appendix E – Project Staff Questionnaire Descriptive Analysis In addition to minimum education requirements, PHAs/projects also in general had other minimum requirements when hiring new staff to work with (re)certifications. Seventy-nine percent of PHAs/projects overall required some computer skills, 71 percent required background checks, 68 percent required administrative or clerical experience, and 62 percent required math or logic skills. However, only 32 percent required special housing related training or certification, and 51 percent required other housing related experience. This shows an emphasis on general office skills and experience over more specific housing related experience. Despite being less likely to have minimum education requirements, owner-administered projects were more likely to have other minimum requirements in most categories than their Public Housing and PHA-administered Section 8 counterparts. Exhibit E-1c and E-1d describe the minimum requirements reported by the PHA/projects for education and other requirements, by program type. Exhibit E-1c.

Minimum Education Requirements for New Employees

5.5% 2.8% 10.6% 6.6% 60.5% 66.0% 63.3% 63.0% l 8.5% 9.0% 7.5% 8.3% Bachel 14.5% 13.2% 7.5% 11.6% Exhibit E-1d. 75.0% 77.4% 83.0% 78.6% Background Checks 66.5% 61.0% 83.0% 71.1% 65.0% 63.0% 73.5% 67.6% 65.5% 58.2% 61.5% 62.1% 48.5% 45.2% 56.5% 50.5% 23.5% 24.0% 45.0% 31.5% Working with (Re)certifications, by Program Type Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total No Minimum Requirements High School/GED Associates/2 years col ege/some college or's Other Minimum Requirements for New Employees Working with (Re)certifications, by Program Type Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total Computer Skills Administrative or Clerical Experience Math or Logic Sills Other Housing Related Experience Special Housing Related Training or Certification E-3

Appendix E – Project Staff Questionnaire Descriptive Analysis Training of New (Re)Certification Staff. PHA/projects train both new and experienced (re)certification staff. The Project Staff Questionnaire collected information about the number of new staff assigned to conduct (re)certifications, as well as the number of hours of training received and the types of training activities used. New staff was defined as staff who were newly assigned to conduct (re)certifications in the past 12 months. In this year’s study, 38 percent of PHA/projects assigned new staff to conduct (re)certifications in the past 12 months. Among the three program types, the PHA-administered Section 8 programs reported the largest proportion of new (re)certification staff members (54.1%), the highest number of new staff assigned to conduct (re)certifications (4 staff, on average), and the highest training hours, on average, for new (re)certification staff (146 hours). By comparison, owner-administered projects had only 30 percent new (re)certification staff, assigned the fewest number of new staff (1), and had 60 hours of training, on average. Public Housing projects had only a slightly higher percentage of new staff and more new staff assigned, and had slightly fewer hours of training than owner-administered projects as shown in Exhibit E-2a. Exhibit 2a. New Recertification Staff Training, by Program Type Program Type PHA- Administered Owner- Section 8 Administered Total Public Housing Average number of new staff assigned to conduct 1.4 (re)certifications 3.7 1.2 2.2 Average number of training hours received by each 56.7 145.9 60.1 92.4 new (re)certification staff Percent of PHA/projects with new(Re)Certification 34.0% 54.1% 29.5% 37.7% staff Note 1: Averages were calculated for PHA/projects that assigned new staff to conduct (re)certifications in the past 12 months. Of the various types of training used for new (re)certification staff, the three most frequently used were: working with experienced staff one-on-one while conducting (re)certifications (90% of PHA/projects), training sessions with the supervisor (85% of PHA/projects), and self training through manuals, videos, or informal questions (62% of PHA/projects). Interestingly in the FY 2006 study, 88 percent of PHAs/projects reported that they used self-training tools such as manuals, videos or informal questioning compared with 62 percent in FY 2007. This represents a substantial drop in the use of these methods. Exhibit E-2b.

Three Most Frequently Used Trainings Types

For New (Re)Certification Staff, by Program Type

Training Methods Usually or Always Used by PHA/projects: PHA- Administered Owner- Section 8 Administered Total New staff worked one-on-one with experienced 88.3% 93.7% 88.1% 90.3%

staff during the conduct of (re)certifications

Supervisor/senior staff held training sessions with 82.4%

new staff explaining procedures 88.6% 84.7% 85.4%

Read HUD/PHA/owner manual, watched videos, or 63.2%

asked informal questions 64.5% 57.6% 62.1%

Note 1: Percentages were calculated for PHA/projects that assigned new staff to conduct (re)certifications in the past 12 months.

Program Type Public Housing E-4

Appendix E – Project Staff Questionnaire Descriptive Analysis Training for Experienced (Re)Certification Staff. For experienced (re)certification staff, about 77 percent of PHA/projects provided training of some form in the past 12 months. On average, PHA/projects trained 5 experienced staff members for an average of 33 hours during the year. Owner-administered projects and PHA-administered Section 8 projects trained the largest percentage of experienced (re)certification staff (81% and 80%, respectively). However, PHA- administered Section 8 projects trained the most number of experienced staff (13 on average), and provided the most hours of training (36 hours, on average). Figures for average number of staff, average number of hours, and percentage of PHA/projects that trained (re)certification staff, by program type are shown in Exhibit E-3a. Exhibit E-3a. PHA- Administered Owner- Section 8 Administered Total Experienced Staff Training, by Program Type Program Type Public Housing Average number of experienced staff receiving training 3.3 12.6 1.5 5.2 Average number of training hours received by each 32.0 36.0 31.7 33.0 experienced (re)certification staff 70.0% 80.1% 81.0% 76.7%Percent of PHA/projects that trained experienced (Re)Certification staff Note 1: Averages were calculated for PHA/projects that provided training to experienced staff. The same three methods that were most commonly used to train new (re)certification staff were also used most commonly to train experienced (re)certification staff, but to less of a degree than with new (re)certification staff. On average, 70 percent of PHA/projects used self-training through manuals, videos, and informal questions to train (re)certification staff, 62 percent used training sessions conducted by a supervisor/senior staff, and 55 percent used experienced staff to work one-on-one with other experienced staff. The use of self-training was more prevalent for training experienced staff than for new staff, and PHA/projects also were less likely to work one- on-one with experienced staff as well. For more detailed figures by individual program type, please refer to Exhibit E-3b. Exhibit E-3b.

Methods for Training Experienced

PHA/projects Usually or Always: PHA- Administered Owner- Section 8 Administered Total Read HUD/PHA/owner manual, watched videos, or asked 71.5%

informal questions 75.2% 66.0% 70.4%

Had supervisor/senior staff hold training sessions with 56.5% 69.3% 62.4% 62.3%

new staff explaining procedures

Had experienced staff work one-on-one with other 45.7% 53.0% 64.2% 54.9%

experienced staff to conduct (re)certifications

Note 1: Percentages were calculated for PHA/projects that provided training to experienced staff.

(Re)Certification Staff, by Program Type Program Type Public Housing E-5

Appendix E – Project Staff Questionnaire Descriptive Analysis The top three topics most frequently covered in training for experienced staff were covered in training over 90 percent of the time. Training related to general HUD policies and rules for conducting (re)certifications was covered about 94 percent of the time, and training related to HUD or PHA/project changes in polices or procedures relating to (re)certifications were covered about 92 percent of the time. Training topics covering tools available in the PHA/project were covered about 90 percent of the time. Training topics did not differ consistently across PHA/projects in different programs, as shown in Exhibit E-3c. Exhibit E-3c.

Experienced Staff Training Topics

in the Past 12 Months, by Program Type

Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total HUD policies and rules for conducting (re)certifications 91.4% 94.9% 95.1% 93.8% Changes in HUD or PHA/project policies or procedures 89.3% 90.6% 95.1% 91.9% related to (re)certifications Tools available in the PHA/project (e.g., software, forms) 90.7% 94.0% 87.0% 90.2% to help in conducting (re)certifications Note 1: Percentages were calculated for PHA/projects that provided training to experienced staff. Transfer of Information about Changes in HUD Policies. PHA/projects used a variety of methods to inform staff about changes in HUD eligibility and rent calculation policies. The most commonly used method by PHA/projects was one-on-one discussions between supervisors and staff at 78 percent. The second most used method was through staff meetings at 70 percent, and distributing to the staff copies of HUD announcements was the third most commonly used method at 65 percent. Distributing detailed staff memos describing the changes and providing instructions for implementation was also used about 54 percent of the time. Exhibit E-4a.

Methods to Communicate Changes in HUD/PHA/Owner

Policies to Staff in the Past 12 Months, by Program Type

Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total One-on-one discussions between supervisors/managers and 74.1% 72.8% 86.4% 77.7% staff Staff Meetings 71.8% 75.7% 61.9% 69.8% Copies of HUD announcement distributed to staff 52.4% 65.4% 79.6% 65.1% Detailed staff memo describing the changes and providing 43.5% 60.3% 61.2% 54.3%

instructions for implementation E-6

Appendix E – Project Staff Questionnaire Descriptive Analysis When PHA/projects had questions concerning HUD policies, they used a variety of methods to seek answers. Eighty-three percent of PHA/projects referred to the HUD/PHA/owner- administered manual, 72 percent asked their HUD field office or other HUD staff, and 51 percent asked questions at a HUD training session. More detailed numbers by program type are shown in Exhibit E-4b. Exhibit E-4b.

Methods for Getting Answers to Questions about

HUD Policies in the Past 12 Months, by Program Type

Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total Referred to HUD/PHA/owner memo or manual 80.0% 80.8% 87.5% 83.0% Asked HUD field office or other HUD staff 64.0% 87.7% 69.0% 72.2% Used Internet/web-based information/training 45.0% 66.4% 52.6% 53.5% Asked questions at a HUD training session 44.0% 57.5% 53.0% 50.9% Quality Control via Work Monitoring. Around 72 percent of PHA/projects usually or always have the team leader or supervisor monitor (re)certification work. More PHA/projects are turning to outside auditors to monitor their work, increasing from 29 percent in FY 2006 to about 33 percent in FY 2007. In order to monitor the quality of work performed by (re)certification staff, PHA/projects used various methods. The most used technique to monitor (re)certifications was reviewing files after completion (76%), followed by using a pre-designed form to check key steps at 70 percent. Other commonly used techniques were computer programs (66%), making individualized notes for each case reviewed (52%), and discussing the (re)certification with staff after completion (47%) as shown in Exhibit E-5a. Exhibit E-5a.

Techniques Used to Monitor (Re)Certifications,

/ 73.0% 85.0% 71.5% 75.7% 67.0% 78.1% 67.0% 70.0% 61.6% 72.5% 65.6% 49.0% 56.8% 50.0% 51.5% i 44.0% 49.3% 48.5% 47.0% in the Past 12 Months, by Program Type Program Type PHA- Administered Owner- PHA projects Usually or Always: Public Housing Section 8 Administered Total Review files after completion Use pre-designed form to check key steps Use computer program 61.5% Make individualized notes for each case reviewed Discuss (re)certification w th staff after completion In determining which cases to select for review, PHA/projects most frequently randomly spot checked a percentage of all cases (65%). Overall, 33 percent of PHA/projects reported reviewing all cases. Other methods commonly used to select cases for review were: checking (re)certifications conducted by new staff (41%), checking cases on certain dates or times of the year (34%), and checking certain cases completed within a given period (33%). E-7

Appendix E – Project Staff Questionnaire Descriptive Analysis In the past year, PHA/projects found various errors during the course of monitoring (re)certifications. The error most reported was finding mistakes in calculating rent, with 71 percent of PHA/projects reporting the error as occurring often or occasionally. Sixty-two percent of PHA/projects occasionally found missing or incomplete verifications of income and 56 percent often or occasionally found missing or incomplete verifications of expenses. Owner- administered projects in general were less likely to find errors than Public Housing projects and PHA-administered Section 8 projects. Exhibit E-5b. l 67.5% 83.6% 65.0% 70.9% ificati 64.0% 77.4% 47.5% 61.5% ificati 51.5% 57.5% 50.5% 56.2% PHA- Exhibit E-5c.

lete 89.5% 95.9% 79.0% 87.4% information l 41.5% 61.0% 34.5% 44.2% 36.0% 53.4% 14.5% 34.4% 27.5% 35.7% 40.0% 34.3% Types of Errors Found in the Past 12 Months, by Program Type Program Type PHA- Administered Owner- Types of errors found often or occasionally: Public Housing Section 8 Administered Total Mistakes in Ca culating Rent Missing or incomplete ver ons of income Missing or incomplete ver on of expenses In this year’s questionnaire, when asked about the cause of errors in eligibility determinations and rent calculations, PHA/projects were asked to focus on the frequency of errors instead of the seriousness of the error. The most commonly stated cause that occasionally or often resulted in errors was tenants providing inaccurate or incomplete information (87%). Other frequently cited reasons were: complex HUD regulations for rent calculations (44%), not having enough staff to handle the workload (34%), and frequent changes in HUD regulations (34%). administered Section 8 projects were most likely to cite tenants providing incomplete/inaccurate information as occasionally or often causing errors at 96 percent. Exhibit E-5c details the most frequently reported causes of some errors. Underlying Causes of Errors in Eligibility Determinations and Rent Calculations in the Past 12 Months, by Program Type Program Type PHA- Administered Owner- Issues Often or Occasionally Causing Errors: Public Housing Section 8 Administered Total Tenants providing inaccurate/incomp Complex HUD regulations for rent calcu ations Not having enough staff to handle the workload Frequent changes in HUD regulations A new question was included in this year’s questionnaire, asking PHA/projects whether HUD, field staff, or outside contractors had reviewed their files in the past 2 years. In the last 2 years, 23 percent overall of PHA/projects had their files reviewed by a HUD related group such as headquarters, field offices, and field staff. About just as many PHA/projects had their files reviewed by outside auditors and contractors at 22 percent of PHA/projects. Only a few had their files audited by a state housing authority at about 5 percent. E-8

Appendix E – Project Staff Questionnaire Descriptive Analysis Conducting Tenant Interviews. When conducting both initial certifications and annual recertifications, the most common method of obtaining household information was by conducting an in-person interview (92% and 90%, respectively). The second most common method was having the tenant complete a form and return it via mail or in-person (62% and 61%, respectively). A typical initial certification interview required about 36 minutes to complete, on average, while a typical recertification interview required only 30 minutes. PHA-administered Section 8 projects took the longest to conduct both initial certifications (40 minutes) and annual recertifications (31 minutes), while Public Housing projects took the least amount of time to conduct both initial certifications (32 minutes) and annual recertifications (28 minutes). Exhibit E-6a.

/ i l 31.8 39.4 38.9 36.4 i l 27.8 31.0 31.6 30.1 PHA/projects were asked how Most PHA/projects date (28.8%). year’s questionnaire. Exhibit E-6b.

10.5% 5.5% 0% 5.9% 40.5% 39.0% 9.5% 28.8% 35.5% 43.2% 77.5% 52.9% 3.0% 0% 8.5% 4.1% Minutes Spent on Typical Interviews, by Program Type Program Type PHA- Administered Owner- PHA projects Usually or Always: Public Housing Section 8 Administered Total Average number of minutes spent on a typ ca initial certification interview Average number of minutes spent on a typ ca annual recertification interview In this year’s questionnaire, a new question was added regarding the procedures used by PHA/projects to complete the annual recertification process. many days prior to the effective date they started the recertification process. started the process 120 days prior to the effective date (52.9%) or 90 days prior to the effective Another new question asked PHA/projects if their annual recertification procedures were the same for households with stable income as for households with more volatile income. An overwhelming majority of PHA/projects stated that procedures were the same for both (97%). In addition to adding questions, some questions were excluded from this A question regarding tenant’s difficulty in responding to questions on certain topics was excluded because its results did not vary significantly from year to year. Number of Months Prior to Effective Date PHA/projects Start the Annual Recertification Process, by Program Type Program Type PHA- Administered Owner- Number of Months Public Housing Section 8 Administered Total Less than 90 days 90 days 120 days Greater than 120 days E-9

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Appendix E – Project Staff Questionnaire Descriptive Analysis Over 56 percent of PHA/projects have a proportion of tenants where less than 20 percent speak a language other than English as their primary language. In these cases where a proportion of tenants did speak a language other than English, PHA/projects used a combination of methods to communicate with their tenants. On average, a majority of tenants brought their own translators (76%). Sixty percent of PHA/projects had bilingual staff available, and 38 percent of PHA/projects brought in translators to communicate with tenants. A smaller percentage of PHA/projects used other methods, such as translating documents, and using Internet translating services to communicate with their tenants (14%). Computers and Software Use. Computer software is playing an increasingly integral part in PHA/projects daily tasks. In the past 12 months, almost all PHA/projects utilized computers and computer software when performing various (re)certification and other administrative tasks. Over 90 percent overall of PHA/projects use computer software to calculate rent, income, or allowances, print the 50058/50059 form, input verified information, print letters to tenants, and record tenant demographics In addition, an increasing number of PHA/projects are using computer software to submit tenant information to HUD. As has been the case, using computer software to interview tenants and record answers was one of the least frequently reported uses. For a more detailed look, by program type of computer utilization, refer to Exhibit E-7a. Exhibit E-7a.

Calcul 96.5% 98.6% 96.0% 96.9% 91.5% 99.3% 94.5% 94.7% 91.5% 93.8% 92.0% 92.3% 91.5% 91.8% 91.5% 91.6% 90.5% 95.9% 85.5% 90.1% 81.0% 94.5% 91.0% 88.3% 29.5% 32.9% 25.5% 28.8% Computer Software Uses in the Past 12 Months, by Program Type Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total ate rent Print the 50058/50059 form Input Verified Information Print letters to the tenants Maintain demographic information about the residents Submit tenant information to HUD Interview tenants and record answers In addition to asking about the different tasks performed by PHA/projects using computers and computer software, the Project Staff Questionnaire also asked more specifically what percent of a PHA/project’s 50058/50059 data was transferred to HUD via PIC/TRACS. Almost all PHA/projects (97%, on average) reported doing so, an increase from 95 percent in 2006. The percentage of PHA/projects who reported transferring all their 50058/50059 data through this method increased from 74 percent in 2006 to 81 percent in 2007. This year we saw an increase in Public Housing projects transmitting their information directly to HUD, as opposed to through another agency or other methods. E-10

Appendix E – Project Staff Questionnaire Descriptive Analysis Exhibit E-7b.

Transmission of 50058/50059 Data to HUD via

PIC/TRACS in the Past 12 Months, by Program Type

Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total Transmitted directly Transmitted through another agency Transmitted by other methods Average percentage of 50058/50059 data transmitted via PIC/TRACS per PHA/Project Percent of PHA/projects using PIC/TRACS to submit some or all 50058/50059 data 88.5% 88.4% 6.0% 6.2% 11.0% 7.5% 49.5% 74.2% 42.0% 19.2% 17.5% 12.5% 96.6% 96.0% 97.9% 96.9% 97.5% 96.6% 98.0% 97.4% This year’s questionnaire posed a new question addressing the increasing trend towards a paperless (or less paper) process of conducting annual recertifications. Most PHA/projects when asked, still keep copies of all 50058/50059’s, including less recent ones, in the household file at 91 percent overall. Owner-administered projects were most likely to keep historical copies in the household file at 94 percent, and PHA-administered Section 8 projects were least likely at 84 percent. Public Housing projects were similar to owner-administered projects at 93 percent. The few PHA/projects who stated that they did not keep historical copies in the household file for the most part printed them from the system when they needed historical information, or kept them archived or on site. Verification Procedures. The most frequently reported methods of keeping track of when verification information was received were: keeping files with outstanding verification in a separate location (70%, on average), making a notation in the tenant file (59%, on average), and marking information using a paper list or tickler file (47%, on average). Exhibit E-8a.

Methods for Keeping Track of

Verification Information, by Program Type

Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total Kept files with outstanding verification in 74.5% 70.5% 64.5% 69.8%

separate location or folder

Kept record in tenant file 58.5% 69.9% 51.0% 58.8%

Marked on a paper list/tickler file (tracking 42.5%

sheet, monitoring form, checklist, or log) 52.7% 48.0% 47.3%

With respect to the person responsible for keeping track of verification requests and returns, a

majority of the time the project (re)certification staff were the ones responsible (85%, on

average). PHA-administered Section 8 projects were least likely to have a supervisor keep track

of verification status (22%), whereas owner-administered projects were most likely (47%).

However, this is most likely due to owner-administered projects relatively small number of staff

in comparison to the other program types.

E-11

Appendix E – Project Staff Questionnaire Descriptive Analysis In last year’s questionnaire, PHA/projects were asked when they started using TASS and EIV to verify income and benefits. This year, the question was modified to ask them if they had used either TASS or EIV in the past year. Overall, 51 percent of PHA/projects had used TASS between October 1, 2006 and September 30, 2007. Eighty-two percent of PHA/projects had used EIV in the same period. Of the PHA/projects who have ever used TASS or EIV, the most frequent uses were to verify: Social Security/SSI benefits (89%), employment income (75%), disability status (64%) and dual entitlement benefits (68%). Owner-administered projects were much less likely to frequently use TASS or EIV to verify information. A breakdown of the frequency of use of TASS and EIV is broken down in Exhibit E-8b. Exhibit E-8b. Social 94.5% 95.2% 74.5% 88.8% 88.4% 89.0% 41.8% 75.0% 75.3% 68.3% 43.2% 63.8% 73.2% 71.0% 48.2% 68.2% 71.0% 30.5% 61.2% 51.5% 49.6% 27.0% 43.8% Frequency of Use: TASS, EIV to Verify Certain Factors, by Program Type Program Type PHA-Administered Owner- Use TASS or EIV usually or always to verify: Public Housing Section 8 Administered Total Security Benefits Employment Income Disability Status Dual Entitlement Benefits Unemployment Benefits 75.8% Black Lung Benefits PHA/projects who used EIV sometimes would sometimes supplement EIV’s information due to discrepancies found between EIV and tenant data, because EIV’s information was out of date, or to double check. Those that supplemented EIV data listed pay stubs, third party verification, and employer information as other methods used to supplement. More than 88 percent of PHA/projects verified all components of tenant information at least occasionally, on average. More than 73 percent always verified tenant information. The types of household information most likely to always be verified were: Social Security Benefits (97% of PHA/projects, on average), Income from employment (96%, on average), Other Sources of Income (94%, on average), and TANF/Welfare benefits (90%, on average). Citizenship was the least likely to always be verified at 73 percent of PHA/projects, on average. Other types of household information less likely to always be verified were: Age of Household Members and Social Security Numbers (74%), Disability Status (79%), and full time student status (80%). Verification rates in all four of the top items verified have varied from year to year, as shown in Exhibit E-8c. E-12

Appendix E – Project Staff Questionnaire Descriptive Analysis Exhibit E-8c.

Yearly Comparison of Rates of Verification

PSQ 2005 PSQ 2006 PSQ 2007 Social 93.7% 89.5% 97.3% 91.3% 86.5% 96.2% 87.4% 82.0% 94.0% 85.4% 81.8% 89.9% most difficult to verify. On the other hand, Other were also reported as being difficult. Exhibit E-8d.

65.6% 78.8% 56.5% 65.8% 65.5% 71.9% 47.5% 60.6% 62.0% 65.1% 50.5% 58.6% 48.0% 62.4% 46.0% 51.1% of Various Items, by Program Type Targets of Verification Procedures Always Verified: Security Benefits Income from Employment Other Sources of Income TANF/Welfare Benefits In addition to identifying how often PHA/projects identified household income, the Project Staff Questionnaire also asked PHA/projects to identify which types of household information were In this year’s questionnaire, instead of asking PHA/projects how difficult different types of household information were to verify, PHA/projects were asked how often they had difficulty. Sporadic Income, in addition to being less likely to be always verified, was listed as the most difficult to verify (66% of PHA/projects). Sources of Income, which was listed as more likely to always be verified, was reported as being more difficult to verify (61%). Income from Employment (59%) and Medical Expenses (51%) Difficulties in Verifying Tenant Information in the Past 12 Months, by Program Type Program Type PHA- Tenant Information Often or Sometimes Administered Owner- Difficult to Verify: Public Housing Section 8 Administered Total Sporadic/infrequent/seasonal employment Other sources of income Income from employment Medical expenses When asked to name causes of problems that emerged when obtaining complete verifications, this year, PHA/projects were asked how often issues emerged, instead of how serious the issues were. PHA/projects cited employers not responding to requests in a timely manner as often or sometimes causing problems at 83 percent. Tenants providing incomplete or inaccurate information (80%) and employers providing incomplete information (76%) were the other two reasons most cited as causing problems. More detailed figures broken down by program type are shown in Exhibit E-8e. E-13

Appendix E – Project Staff Questionnaire Descriptive Analysis Exhibit E-8e.

Causes of Problems in Obtaining Complete

Verifications, in the Past 12 Months, by Program Type

Program Type PHA- Administered Owner- Issues Caused Serious or Some Problems: Public Housing Section 8 Administered Total Employers not responding to requests in timely 87.5% 90.4% 73.5% 83.2% manner Tenants providing incomplete/inaccurate 82.0% 89.7% 72.0% 80.4% information Employers not providing all requested 76.0% 87.0% 67.5% 75.8% information With respect to the level of cooperation of various individuals and institutions when verifying tenant information, insurance companies and health care providers were most likely to be usually uncooperative. Twelve percent of PHA/projects reported insurance companies as being usually uncooperative, 12 percent reported financial institutions and 9 percent of PHA/projects reported health care providers as being so. Overall, the percentage of PHA/projects who viewed insurance companies and health care providers as being usually uncooperative decreased from last year. Exhibit E-8f.

Uncooperativeness of People in Obtaining Verification

) 10.0% 17.8% 8.5% 11.5% Fi 11.5% 14.4$ 10.0% 11.7% 6.5% 12.3% 8.5% 8.8% Information, in the Past 12 Months, by Program Type Program Type PHA- Administered Owner- Usually Uncooperative: Public Housing Section 8 Administered Total Insurance companies (e.g., health insurance nancial institutions (e.g., banks, investment firms) Health care providers (e.g., doctors, physicians, pharmacies) When problems and difficulties arose in verifying information, PHA/projects resolved these issues though a variety of methods. Most prevalently, PHA/projects called third-parties to obtain information (92%). PHA/projects also sent follow-up letters to third-parties (87%), called tenants (86%), sent follow-up letters to tenants (80%), and used electronic verification or data matching such as EIV (77%). On average, 69 percent of PHA/projects reported resorting to accepting other/less preferred verification, up slightly from 63 percent in FY 2006. Owner- administered programs were significantly less likely to resort to accepting less preferred verification. The number of PHA/projects who used EIV when problems and difficulties arose increased from 74 percent in FY 2006 to 77 percent. E-14

Appendix E – Project Staff Questionnaire Descriptive Analysis Exhibit E-8g.

Procedures Used When Verification Was Not Provided

As Requested in the Past 12 Months, by Program Type

Program Type PHA- Administered Owner- Public Housing Section 8 Administered Total Called third-party 91.0% 94.5% 90.5% 91.8% Sent follow-up letter to third-party 87.5% 87.0% 87.5% 87.4% Called tenant 89.0% 87.7% 83.0% 86.4% Sent follow-up letter to tenant 89.5% 85.6% 66.0% 79.9% Used electronic verification or data matching 90.5% 92.5% 52.0% 76.9% (e.g., EIV) Accepted other/less preferred verification 66.0% 84.2% 62.0% 69.4% C. Conclusion. Overall the PSQ analyses portrayed a complex and interesting picture of PHA/Project practices and procedures. Most PHA/projects train (re)certification staff, transfer information about changes in HUD policies to their staff, monitor (re)certification work quality, use computer software for various purposes, and verify most (re)certification information. Some findings differed by program type. Owner-administered PHAs were more likely to differ from the other two program types, perhaps due to their size. They had the fewest staff, fewest (re)certification staff, and fewest units supported by the (re)certification staff, on average. Owner-administered projects also trained the fewest staff for the fewest hours, and were the least likely to use TASS or EIV systems to verify information. They are also much more likely to start the annual recertification process 3 to 6 months before the effective date. Lastly, they seem to have fewer difficulties verifying tenant information, which would explain why they are also the least likely to resort to accepting less preferred verification information when difficulties arose in obtaining that information. This year’s study introduced new questions focusing on: the tenure of (re)certification staff, outside reviews of PHA/project files, and the timing of procedures used to complete the annual recertification process. The questionnaire also expanded the examination of PHA/projects use of EIV and TASS. The results of these questions highlighted some new differences in procedures between owner-administered projects and their counterparts, as mentioned above. For the future studies, it would be helpful to develop and validate additional items specifically targeting potential difficulties in conducting training, using computer software, and getting support from various sources in verifying tenants’ information. Focus groups and cognitive interviewing might aid in revision of the PSQ items by focusing attention on the specific circumstances and issues faced by the PHA/projects. Having detailed indicators of the positive, as well as negative aspects of the (re)certification process, defined by the PHA/Project staff, would provide a more complete picture of the issues faced by the PHA/project, as well as may provide a better link between PSQ information and the estimation of payment and income errors. E-15

Appendix F—Multivariate Analysis

Appendix F: Multivariate Analysis The multivariate analyses sought to further examine gross rent error and different types of error in the certification/recertification process made by PHA/project personnel. We linked the error measures to major project and household characteristics using multivariate analytic techniques. The analysis allowed us to identify potential project or tenant characteristics that help account for (or predict) each error measure, net of the effects of other variables. We attempted to meet the specified study objectives by examining two analytic questions: 1. Holding other conditions constant, what project and household variables (e.g., training for certification staff, household financial characteristics) accounted for rent error and errors made by staff? 2. What was the effect size (or relative strength) of a predictor variable in accounting for rent error? 1 To inform HUD’s program improvement initiative, the focus of this analysis was project variables and types of staff error in connection to rent error. It was necessary to take this focus to address study Objectives 5, 6, 8, 12, and 13 regarding the influence on rent error by project factors vis-à-vis tenant characteristics. Project variables and errors made by staff that contributed to rent error are possibly controllable in future work. Knowledge about project features in relation to rent error is thus more useful for program improvement. Prior studies have found that tenant background and financial situations were predictive of rent errors. Specifically, higher incomes from different sources and the complexity of income/expenses and allowance sources were related to larger rent errors. These variables, however, are beyond the control of staff and not subject to program remedy. In this study, tenant variables were primarily considered as covariates for statistical control in the examination of the effects of error and project operation variables. We did, however, examined joint effects or the interaction between tenant variables and project variables. Conceptual Framework This analysis was largely guided by the conceptual framework used in the previous studies (see Figure F-1). Rent error was considered as the outcome influenced by three sets of factors, household characteristics, project characteristics/practices, and staff errors; whereas each type of error made by staff is thought to relate to household and project factors. Project variables are conceptualized in several subcategories: project organization and staffing, certification practice, verification practice, system automation based on computer application, and project survey respondents’ perception of causes to errors (e.g., staff shortage). The conceptual framework recognizes that different process errors made by staff may be inadequate in typology and measurement to fully explain rent error; and project characteristics and quality control processes may have additional impact on rent error. For example, staff qualifications and the use of computer automation may potentially reduce rent errors through improved quality control in broad areas beyond coping with particular types of process errors. Estimation of the effect size for predictor variables requires valid measurement of each variable, sensible model specifications, and good model fit. In survey data analysis, however, it is always challenging to obtain accurate measures of every variable and specify models that generate robust estimates of effect sizes. F-1

1

Appendix F: Multivariate Analysis Figure F-1 Conceptual Framework for Modeling Rent Error Tenant factors Demographics

Income sources

Allowance for expenses

Elderly or disabled tenant

Types of error Overdue recertification error Allowance calculation error Income calculation error Consistency error Procedure error Transcription error Administrative error Other calculation error Project factors Organization (size, staffing,

training)

Certification

Automation/IT use

Verification

Perceived error causes

Rent error Analytic Approach Project-level data were merged with tenant-level data for the analysis. We took tenant gross rent error and types of process error as outcomes in two sets of models. For gross rent error models, tenant variables, types of error, and project variables were used as predictor variables. For process error types, tenant and project variables were used as predictor variables. The analysis included three steps. First we constructed composite variables to measure key concepts that hypothetically influence rent errors. Second we conducted preliminary analysis to identify predictor variables that were important to account for gross error, test alternative modeling approaches, and perform diagnostic analysis. Finally, we estimated models to address objectives specified in the Analysis Plan and additional issues of interest. All statistical analyses were run with SAS. To adjust the design effect resulting from the clustered sampling design, we used the procedures of SURVEYREG for multiple regression modeling of gross rent error, SURVEYLOGISTIC for modeling different types of error, and SURVEYMEANS or SURVEYFREQ to examine weighted descriptive statistics. For initial analysis we used PROC MIXED for estimating two level variance, and SAS conventional procedures to examine raw data and residual scores of the predicted gross error. Construction of Composite Variables The large number of tenant and project data items required editing and rescaling to build composite indicators of the concepts to explain rent errors. The process is summarized below. General Strategy for Selecting Variables to Include in the Analysis. Predictor variables were selected for analyses based on a mixture of hypothesis-driven and data-driven approaches. First, F-2

Appendix F: Multivariate Analysis variables predictive of rent errors in the multivariate analyses conducted in past HUDQC studies (2000-FY 2006) were selected; this was the primary approach used to select tenant variables. Second, variables were selected based on hypotheses that they would be associated with rent errors; this was the primary approach used to select Project Staff Questionnaire (PSQ) variables. Some variables that were initially selected were excluded based on preliminary analyses. For example, variables were excluded if they were redundant with other variables included in the analysis based on examination of variance inflation factors. Details of the process of constructing and selecting variables for the analysis are explained below. Tenant-Level Data. Tenant data are the source for outcome measures of the analysis (dollar amount of rent errors and types of processing error) and household characteristics and financial conditions. Largely following previous studies, we constructed composite variables from the original data items and tested the variables’ bivariate and multivariate relationships with gross rent error. Specifically, to the extent that such variables were available in the FY 2007 study, we started with tenant variables that were predictive of gross rent error in past HUDQC multivariate analyses (2000-FY 2006). In the process we examined descriptive statistics and missing values. The goal was to determine what variables were related to the error and therefore to be included into the multivariate equations. Exhibit F-1 lists all the modeled variables with descriptive statistics. Gross rent error is the sum of the absolute values of over- and underpayments for each tenant. We took the logarithm of gross rent error to compact its skewed distribution (due to few cases with large error and many with zero error). Tenant background is represented by a binary-coded indicator with one for household heads who were elderly (age 62 or older) or disabled. Tenant financial conditions were represented by a set of measures, each rescaled by the grand mean (known as centered scaling) to make interpretation more straightforward 2 . Tenant financial condition variables were: ♦ Households with elderly or disabled members ♦ Total earned income, in dollar value ♦ Total public assistance income, in dollar value ♦ Total pension income, in dollar value ♦ Number of allowances ♦ Total other income, in dollar value ♦ Number of income and expense sources 2 With such centered scaling, the intercept of the regression model is the log gross rent error for tenants who had mean values on all the predictor variables; and each regression coefficient as the change in log gross rent error associated with one unit change around the grand mean of the given predictor variable. F-3

Appendix F: Multivariate Analysis Exhibit F-1 Variable Mean Std Dev Minimum Maximum Unweighted Descriptive Statistics for Tenant-Level Variables Used in the Multiple Regression Analysis Log gross error 1.293 1.603 0.000 6.919

N household members -0.058 1.496 -1.185 9.815

Gross adjusted income -0.196 8.985 -12.506 89.787

N bedrooms -0.042 1.125 -1.873 24.127

Earned income -0.006 0.727 -0.416 6.584

Other income -0.005 0.503 -0.219 3.781

Public assistance income -0.004 0.337 -0.127 1.873

Pension income -0.010 0.950 -0.936 6.064

Age of household head 0.227 19.567 -39.415 47.585

Total number of sources of

income/expenses 0.025 3.068 -3.116 22.884 Number of allowances -0.005 0.632 -1.286 1.714 Elderly or disabled household 0.578 0.494 0.000 1.000 Proportion of items with 0.001 0.263 -0.186 0.814 transcription errors Proportion of item categories 0.002 0.209 -0.059

without written 3rd party ver 0.941

Overdue recertification error 0.015 0.123 0.000 1.000 Other calculation error 0.058 0.234 0.000 1.000 Allowance calculation error 0.032 0.175 0.000 1.000 Income calculation error 0.027 Consistency error 0.195

Procedure error 0.250

Transcription error 0.380 Administrative error 0.417

Public Housing 0.333

Section 8 0.333

0.161 0.000 1.000 0.396 0.000 1.000 0.433 0.000 1.000 0.486 0.000 1.000 0.493 0.000 1.000 0.471 0.000 1.000 0.471 0.000 1.000 Source: HUDQC FY 2007 tenant-level data

Note: Tenant background variables are centered by grand mean, with variable name start with c_.

The typology of error was defined and measured the same way as in prior studies (see Introduction and Methodology for details), including: ♦ Overdue recertification error ♦ Allowance calculation error F-4

Appendix F: Multivariate Analysis ♦ Income calculation error ♦ Consistency error ♦ Procedure error ♦ Transcription error ♦ Administrative error ♦ Other calculation error Each type of error was indicated by dummy variables with one for error and zero for no error. Three interval measures of error were constructed—transcription and verification error rates and the total number of errors. Note that the verification error used in prior studies was removed from the analysis. The measure was the proportion of the number of verified items over the number of items that were available for verification. If a household did not have any item for verification, then it would be coded as no verification error, though in reality this may not be true. Thus the variable might not accurately indicate a mistake; it represents instead the extent of the verification effort to the extent that items were available for verification. Project-Level Data. The FY 2007 Project Staff Questionnaire (PSQ) survey collected new data items and modified some of the questions in the instrument. The new items were, for example, the proportion of certification staff with at least one year and five years experience, whether certification staff worked on ongoing caseloads, and the proportion of tenants speaking languages other than English. The raw data items were examined systematically through data editing and univariate, bivariate, and initial multivariate analysis. Composite variables were built with original items, giving preference to those with conceptual importance in accounting for the outcomes (rent error and process error), reasonable variation (frequency distributions were not concentrated in one category), and minimal missing cases. Exhibit F-2 lists all the modeled variables with descriptive statistics. Data recoding/rescaling entailed two approaches. One was to create accumulated counts of staff activities (e.g., the number of items to review/monitor, the number of personnel and the number of methods used in review and verification) from original data items that were expected to be relevant to rent error. This approach generated composite variables of interval scale. The other approach was recoding original data items to create dichotomous (dummy) variables. It required examination of the raw data frequency distributions and missing values to ensure adequate differentiation and available cases. For example, in creating dummy variables for perceived difficulty in certification verification, we recoded cases to define “always” vs. others (“somewhat” and “not difficult”). If few cases were in a given category, we combined two categories that were meaningful in making a distinction relevant to rent error (e.g., combine “usually verify” with “always verify”). We also examined the bivariate relationship between the constructed variables with rent error and adjusted the coding to make the variable more relevant to the outcome measures. For example, educational requirements for certification staff provided categories ranging from “no requirement”, “high school graduation or GED”, “2 years of F-5

Appendix F: Multivariate Analysis college”, and “4 years of college”. Two dummy variables were created, with one representing “no requirement” versus “high school or more” and the other contrasting “no college required” against “2 years or more of college”. Testing the two variables, we decided to include only the first variable in modeling due to its closer relationship with rent error. Exhibit F-2 (Page 1 of 2)

Unweighted Descriptive Statistics

for Project-Level Variables Used in the Multiple Regression Analysis Variable Mean Std Dev Minimum Maximum N units supported 1143.76 3686.65 1 44083 Units per staff 62.674 63.241 0.0714 574 Units per cert staff 156.727 124.471 1 728.5 Certified staff total staff ratio 0.451 0.282 0.007 1 % cert staff only do certification 0.232 0.364 0 1 Ongoing caseload (yes=1, no=0) 0.587 0.492 0 1 % cert staff 5+ yr experience 0.606 0.356 0 1 Require hs grad+ (yes=1, no=0) 0.938 0.241 0 1 Other hiring requirement (y=1, n=0) 0.978 0.146 0 1 All experienced cert staff trained (y=1, n=0) 0.448 0.497 0 1 Any policy/procedure change (y=1, n=0) 0.388 0.487 0 1 HUD review files (y=1, n=0) 0.743 0.437 0 1 N months begin process before recert 3.285 1.238 0 5 Require interview for init cert (y=1, n=0) 0.921 0.270 0 1 Require interview for recert (y=1, n=0) 0.831 0.375 0 1 All info used PIC/TRACS transfer (y=1, n=0) 0.800 0.400 0 1 Direct transfer to HUD (y=1, n=0) 0.755 0.430 0 1 Used TASS last yr (y=1, n=0) 0.510 0.500 0 1 Use either TASS or EIV (y=1, n=0) 0.857 0.350 0 1 N activities using computer 9.666 2.443 0 14 Frequency of TASS or EIV last yr 3.656 2.254 0 6 N persons monitor cert 2.070 1.430 0 6 N methods used to monitor 4.008 2.113 0 9 F-6

Appendix F: Multivariate Analysis Exhibit F-2 (Page 2 of 2)

Unweighted Descriptive Statistics

for Project-Level Variables Used in the Multiple Regression Analysis

Variable

Monitored by supervisor 0.718 0.450 0 1 1Monitored by auditor 0.336 0.472 0 Monitored by HUD 0.205 0.404 0 1 1 1 Monitor w computer 0.653 0.476 0 Monitor w pre-form 0.711 0.453 0 N methods used to review cases 2.539 1.557 0 7 16 10 N items always verified 12.990 3.096 0 N items never verified 1.592 1.486 0 Mean Std Dev Minimum Maximum N items very difficult to verify 0.705 1.619 0 14

Supervisor keep track of ver requests 0.347 0.476 0 1

N methods used to follow up 4.998 1.303 0 7 1 5 1 1 Other less preferred verification accepted 0.710 0.454 0 N of perceived errors 0.393 0.820 0 Any perceived errors 0.257 0.437 0 Staff shortage 0.022 0.146 0 Source: HUDQC FY 2007 project-level data

Note: Of the 549 projects contacted, 3 did not respond, thus leaving 20 tenant records without project data.

The following composite/rescaled variables by conceptual groups were used in the final modeling: ♦ Staffing and Workload ƒ Organization size: the total number of households served ƒ Workload: unit/all staff ratio, unit/certification staff ratio, certification staff/all staff ratio, and the assignment of ongoing caseload of specified tenants ƒ Certification staffing experience: percent of staff having more than one-year experience and percent of staff having more than five-years experience ƒ Certification staff educational hiring requirements; a dummy variable indicating requirement of high school/GED or more versus no minimal requirement F-7

Appendix F: Multivariate Analysis ƒ Other credential requirements in hiring: a dummy variable indicating any requirement of qualifications (including special housing-related training/certification, administrative and clerk skills, computer skills, and other skills) versus no additional requirements ƒ Experienced certification staff training: a dummy variable indicating provision of training last year versus no training for experienced certification staff 3 ♦ Certification and Recertification Process ƒ Timeline to start recertification: A number of data items measuring the length of time for projects to start recertification activities before the effective date. Closely correlated, these variables were tested in relation with rent error. The number of months the recertification process started before expiration was used in the modeling ƒ Personal interview requirement in certification/recertification: Two dummy variables—respectively for certification and recertification--indicate a personal interview requirement versus no such requirement ƒ Method of reviewing certification: counts of different methods that were usually or always used to review certification materials, ranging from sitting in on the interview, reviewing files while the (re)certification was in process, discussing the (re)certification with staff, to using a computer program to review ƒ Change in policy and procedure: a dummy variable indicates any changes implemented in policy/procedures regarding eligibility or rent calculation versus no such changes in the past two years ƒ HUD review certification: a dummy variable indicates whether HUD reviewed the (re)certification work in the last year ♦ Automation with Computer/IT Application ƒ PIC or TRACS use: a dummy variable indicates using either PIC or TRACS in operation versus using neither ƒ Direct transfer of data to HUD: a dummy variable indicates directly transferring data to HUD versus transferring data via other organizations ƒ TASS use: a dummy variable indicates using the TASS in process ƒ EIV use: a dummy variable indicates using EIV The measures of certification staff training are important but the data were available only for subgroups of the sampled projects. Items on training of new staff were available for less than one-half of the project sample that hired new staff and assigned them to conduct certifications last year. Therefore it was impossible to use the detailed training measures for new staff (participation, length, methods and topics) in the analysis of the data for the total sample. For experienced staff, less than three quarters of the sampled projects reported to have provided training last year. The detailed data on training were again not available for analysis of the entire sample. F-8

3

Appendix F: Multivariate Analysis ƒ Frequency of automated activities: number of certification activities that were performed by computer ƒ Number of certification activities that were frequently processed via TASS or EIV ♦ (Re)certification Verification. Data on monitoring and case reviewing seemed important to account for rent error. The interval scales were summed across the number of items monitored/reviewed, different personnel categories involved, and various methods used. Binary indicators were created for distinct methods and personnel categories that were expected to impact error (monitor with computer, monitor with predetermined form, monitor by supervisor, HUD/HUD contractor, outside auditor, etc.). Likewise, numbers of items, methods, and personnel involved in verification of certification/recertification were, respectively, summed up to gauge the extent of the verification efforts. These were dummy variables that indicate: ƒ Frequently verified items: counts of items that were usually or always verified, out of 16 items ranging from age of household members, social security numbers, income from employment to full time student status. ƒ Never verified items: counts of items that were never verified, out of the same16 items ƒ Difficult verified items: counts of items that were reported often difficult to verify, out of the same 16 items ƒ Supervisor verification: a dummy variable indicates whether the supervisor kept track of verification ƒ Procedures to follow-up verification: counts of procedures used to follow verification that was not provided, e.g., letters to the third party, call the third party, letters to the tenant, and call the tenant, among others. ƒ Acceptance of less preferred verification: a dummy variable indicates whether the project accepted verifications that were less preferred verification. ♦ Staff perceived error causes. Six items were collected about staff perception of error causes, including staff shortage with the workload, complexity of HUD regulations, and frequent changes in HUD regulations, among others. Since these variables were correlated, we used only two measures: One is a dummy variable indicating if any items were reported to often cause errors versus none of the items reported. The other is a dummy variable representing whether staff shortage was a frequent cause of error. Initial Analyses Initial analyses were conducted iteratively with model testing to check missing data, explore alternative analytic approaches, select predictor variables, and examine residual values of the predicted gross error. Based on such diagnostic analyses, we then specified and tested multiple regression and logistic models of gross rent error and types of error. F-9

Appendix F: Multivariate Analysis Missing Data. There were 3 projects that did not respond to the PSQ survey, resulting in 20 tenant records without corresponding PSQ data (equivalent to 0.8% of the 2,404 tenants sampled). Analysis revealed that these tenants had lower gross rent error than the rest of the sample. For this analysis, these 20 cases were excluded. We believe that this small proportion of unit non-response should cause little bias to our complete-case analysis, based on Harrell’s (2001) rule-of-thumb. Item non-response occurred to a small number of variables. The largest number of missing data was 29, occurring for the item of the number of certification staff with five-years experience. We conducted regression analysis to impute the value, using the total units served, total staff, and the staff with one-year experience as predictor variables. For items with extremely low non-response (only one to three cases), we imputed the value with the mode or mean. Project-Level Variance: Hierarchical Linear Modeling. The HUDQC sampling design and the resulting tenant and project data present a structure where tenants are nested within organizations. With such a design, hierarchical linear modeling (HLM) is more effective than ordinary least square (OLS) regression in estimating effects at the project and household levels if a substantial proportion of the total variance exists between projects (Raudenbush & Bryk, 2002). We tested the unconditional HLM model to estimate PHA/project/owner level variance (Exhibit F-3). Similar to prior studies, we found the project level variance was small, only 5.9 percent of the total variance. So we decided to use ordinary least square regression (OLS) to analyze gross rent error and logistic regression to analyze likelihood of different types of error made by staff. Exhibit F-3 Standard Z Value /

Model and effect Estimate Error t Value

Unconditional model Probability (ANOVA) Variance Estimates at Project and Tenant Levels: Unconditional HLM Modeling Random effect a Project level variance, u j 0.254 0.045 5.60 <.0001 Household variance, r ij 4.032 0.128 31.60 <.0001

Total variance 4.286

Project-level proportion of variance 0.059 Fixed effect a Mean log gross rent error (Intercept), γ 00 1.298 0.039 33.140 <.0001 a

Random effects were tested in Z statistics and fixed effects in t statistics. Source: HUD Quality Control Survey, FY 2007

Model Specification and Testing To select from the large number of tenant and project variables for the analysis, separate regression equations were specified. From each set of variables that represent a given construct, we initially selected predictor variables that were relatively substantial and statistically significant in bivariate relation with the gross rent error. Next, we examined multicollinearity among the selected predictor variables. Multicollinearity may generate unstable estimates of the regression coefficients such that trivial changes in F-10

Appendix F: Multivariate Analysis independent variables may substantially alter the regression coefficients. Variance inflation factors (VIF) are measures of multicollinearity among independent variables in a regression model. In a diagnostic analysis, we examined VIF in OLS regression analysis of selected predictor variables to minimize collinearity in the final selection of predictor variables. For the large set of project level variables, extensive examination was performed in an attempt to identify variables that were predictive of rent errors and variables that were closely correlated. We selected for each conceptual group (as listed above) variables that had relatively large coefficients in relation to gross error or were conceptually more relevant to rent error, although few project variables were found to have strong relationship with gross error. Residual Analysis. Examining the distribution of residual scores of the predicted rent error is a way to ensure reasonable model specification and model fit with the data. Residuals refer to the discrepancies between the observed value of rent error and the predicted value of rent error resulting from the multiple regression model. Residual distribution should be roughly normal and without clear trend if the model fits well. We ran OLS regression with log gross rent error as the dependent variable and the final set of independent variables. The results suggested that the standardized residual values were distributed approximately normal. The trend was moderate: cases with higher predicted rent errors tended to have slightly lower residual scores. Effect Size. Estimating effect size is more informative and challenging relative to identifying significant predictors. Hypothesis tests determine whether or not an effect exists; a “statistically significant” predictor is one for which there is strong evidence that it is associated with rent error. In contrast, effect size estimates indicate the size or degree of association between a predictor and rent error, answering the question about the “relative importance” of a given predictor variable or set of predictor variables in accounting for rent errors. To assess relative effect size, we calculated the proportion of the total variance of gross error accounted for by each group of predictor variables. In addition, we provided the effect size measured with Cohen’s f 2 for predictor groups, not individual predictor variables 4 . This approach was conservative since many project variables were newly developed and their psychometric properties untested. Findings We first present the results of gross rent error modeling, where gross rent error was seen as a function of staff errors as well as household and project characteristics. Then we show the estimates of multiple logistic regression analysis of different types of error made by staff, again from household and project variables. Finally, analysis of three interval measures of staff error, i.e., counts of transcription errors, counts of verification errors, and the total counts of errors, were presented. 4 The effect size for multiple regression analysis may be assessed by comparing the change of the R 2 . Given an R 2 A value resulting from an equation with a set of independent variables A, and an R 2 AB value generated from an equation with the A and another set of independent variables B, Cohen's f 2 is commonly used in the context of sequential (or nested) multiple regression analyses (Cohen, 1988). The f 2 effect size measure for multiple regression is defined as: F-11

Appendix F: Multivariate Analysis Gross Rent Error. Seven models were tested to estimate the effects of different sets of predictor variables in relation to gross rent error (see Exhibit F-4). Linear regression adjusted for design effects (PROC SURVEYREG in SAS) was the modeling technique. Predictor variables representing explanatory concepts were added into the equation in a sequence (known as sequential modeling). Each model allowed us to estimate the effect of a particular set of variables that were added into the equation, the changing effects of previously entered variables, and the model fit statistics. The final model (model 7) included the seven sets of variables representing as specified in the conceptual model: ♦ Project characteristics, ♦ Certification practice, ♦ Verification practice, ♦ Automation with computer/IT application, ♦ Major types of error, and ♦ Tenant factor (characteristics and financial conditions). The predictor variables entered into the sequential models incrementally accounted for the variance of the gross rent error, with the largest share by indicators of the eight types of error made by staff (i.e., overdue recertification error, allowance calculation error, income calculation error, consistency error, procedure error, transcription error, administrative error and other calculation error, 16.4%). Tenant characteristics and financial conditions accounted for 10.3 percent of the variance. The proportion of rent error variance explained by project variables totaled only 3.1 percent (Figure F-2). Corresponding to variance partitioning, the effect size estimates with Cohen’s f 2 also showed that staff errors represented the bulk of the effects on rent error (.203); measures of household attributes also had a sizable effect (.147); and project characteristic effects were relatively small to the prediction of gross rent error (.032). The estimated intercept serves a contrast to interpret estimates from each model. For example, in model 1, the intercept estimated in log scale 1.586, equivalent to $4.59. 5 This was the expected average gross error of a “reference” group of households that had a zero values on each predictor variable. Because of the centered rescaling, the “reference” households were characterized by their projects’ average value on each interval predictor variables (e.g., average unit/staff ratio) and zero value on each dummy coded predictor variables (e.g., zero as owner-administered for Public Housing and Section 8 variables). Interpreting the estimates, we focus only on those that were statistically significant (p<.05 or smaller) as these represent effects that are unlikely due to chance. 5 e l e lDollar amount = , where is a constant approximately 2.72, is the estimated regression intercept in log scale. To convert coefficients in log scale to dollar amount, we add the log-scale estimate to the intercept and use the sum to perform the same calculation as above. The difference between the resulting dollar amount and the intercept- equivalent dollar amount is the estimated effect in dollar amount. F-12

Appendix F: Multivariate Analysis Exhibit F-4 (Page 1 of 3)

Log Gross Rent Error Accounted For By Tenant, Process Error, and Project Variables:

Multiple Regression Coefficients, Design Effect Adjusted

Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7

Variable Project background Certification Verification Automation Error types Perceived problem Tenant characteristics

Intercept 1.586

Project: Total number of units 0.000

Project: Unit per cert staff 0.000

Project: Unit per cert staff 0.000

Project: Cert staff as a % of total staff 0.126

Project:% cert staff only do cert 0.011

Project: Ongoing caseload 0.008

Project:% cert staff 5+ yr experience -0.300

Project: Require hs grad+ -0.067

Project: Other requirement -0.003

Project: All exp cert staff trained -0.091

Project: Public Housing -0.264

Project: Section 8 -0.051

Project: Months process before recert

Project: Require interview for init cert

Project: Require interview for recert

Project: N methods used to review cases

Project: Any policy/procedure change

Project: HUD review files

Project: N items always verified

Project: N items never verified

Project: N items never verified

Project: Supervisor keep track ver requests

Project: N methods used to follow up

1.390 1.780 1.995 1.410 1.351 1.779 0.000 0.000 0.000 0.000 0.000 0.000 0.000 -0.001 0.000 0.001 0.001 0.001 0.000 0.001 0.000 0.000 0.000 0.000 0.147 0.213 0.176 0.056 0.054 -0.006 0.004 -0.032 -0.025 0.099 0.104 0.104 0.008 0.002 -0.011 -0.039 -0.037 -0.042 ** -0.269 ** -0.250 * -0.232 * -0.235 * -0.216 * -0.201 -0.054 -0.047 -0.040 -0.156 -0.167 -0.203 0.055 0.255 0.164 0.172 0.159 0.060 -0.097 -0.094 -0.083 -0.019 -0.022 -0.034 ** -0.215 * -0.178 -0.269 * -0.271 * -0.261 * -0.285 -0.032 -0.010 -0.112 -0.048 -0.047 -0.124 0.017 0.019 0.028 0.026 0.026 0.018 -0.175 -0.158 -0.146 -0.122 -0.131 -0.060 0.147 0.130 0.130 0.143 0.156 0.131 -0.027 -0.010 -0.002 -0.006 -0.008 -0.008 0.047 0.041 0.001 -0.032 -0.035 -0.083 0.122 0.126 0.120 0.051 0.054 0.043 -0.023 -0.017 -0.011 -0.008 -0.006 -0.033 -0.031 -0.043 -0.037 -0.039 -0.009 -0.007 0.000 -0.007 0.003 -0.079 -0.103 -0.132 -0.119 -0.111 -0.074 * -0.057 -0.043 -0.044 -0.025 F-13

Quality Control for Rental Assistance Subsidy Determinations - Final Report for FY 2007

Appendix F: Multivariate Analysis Exhibit F-4 (Page 2 of 3)

Log Gross Rent Error Accounted For By Tenant, Process Error, and Project Variables:

Multiple Regression Coefficients, Design Effect Adjusted

Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Project Variable background Certification Verification Automation Error types Perceived problem Tenant characteristics Project: Other Less preferred verification accepted 0.097

Project: 100% used PIC/TRACS transfer

Project: Direct transfer to HUD

Project: Enterprise Income Verification (EIV)

Project: Used TASS last yr

Project: N activities using computer

Project: Frequency of TASS or EIV use last yr

Tenant: Overdue recertification error

Tenant: Other calculation error

Tenant: Allowance calculation error

Tenant: Income calculation error

Tenant: Consistency error

Tenant: Procedure error

Tenant: Transcription error

Tenant: Administrative error

Project: Any perceived error cause

Project: Staff shortage

Tenant: Elderly or disabled household

Tenant: Earned income

Tenant: Other income

Tenant: Public assistance income

Tenant: Pension income

Tenant: Total number of sources of income/expenses

Tenant: Number of allowances

R 2 0.122 0.047 0.048 -0.082 -0.208 * -0.157 -0.162 -0.108 -0.018 0.012 0.023 0.070 -0.029 -0.125 -0.125 -0.196 -0.283 *** -0.175 * -0.180 * -0.190 -0.004 0.005 0.003 -0.005 0.030 0.024 0.022 0.034 1.003 ** 0.975 ** 1.015 -0.069 -0.057 -0.159 0.234 0.239 0.125 -0.292 -0.288 -0.383 0.023 0.026 -0.114 -0.041 -0.052 0.001 0.931 *** 0.932 *** 0.679 0.412 0.412 0.333 0.144 0.192 -0.135 -0.138 -0.225 0.438 0.323 0.106 0.093 0.054 0.604 0.013 ** 0.017 ** 0.021 ** 0.030 ** 0.194 *** 0.195 *** 0.298 F-14

Appendix F: Multivariate Analysis Exhibit F-4 (Page 3 of 3)

Log Gross Rent Error Accounted For By Tenant, Process Error, and Project Variables:

Multiple Regression Coefficients, Design Effect Adjusted

Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Project background Certification Verification Automation Error types Perceived problem Tenant characteristics Cohen's f 2 0.014 0.003 0.004 0.009 0.203 0.001 0.147 % variance accounted for 0.013 0.003 0.004 0.009 0.164 0.001 0.103 Denominator DF 2324 2324 2324 2324 2324 2324 2324

Source: HUDQC FY 2007 PSQ data merged with the Tenant Survey

Note: Project represents Project-Level variables; Tenant represents Tenant-Level variables

p< 0.05 ** p< 0.01 (test with the null hypothesis that a coefficient [or R 2 ]=0; a significant result indicates that the corresponding variable(s) is associated with gross error).

F-15

Appendix F: Multivariate Analysis Figure F-2 Proportions of Variance of Gross Rent Error Accounted for by Tenant, Process Error, and Project Variables: Multiple Regression Analysis with Design Effect Adjusted Source: HUD Quality Control Survey, FY 2007 Public Housing, PHA-administered Section 8, and owner-administered projects differed in rent error as found in the bivariate analysis. Regression modeling confirmed that Public Housing had a lower average gross error than owner-administered households (-.264 in log scale, equivalent to a reduction $.84), if other conditions equal. Note that this estimate changed little in the seven models where the other predictor variables were held constant. The difference between households under PHA-administered Section 8 projects versus owner-administered projects, however, was not significant net of other effects. The finding may imply that the higher gross error for Section 8 projects identified in bivariate tabulations was attributable largely to the programs’ differences in tenant characteristics. In other words, the Section 8 programs’ higher gross error may be caused by more complex tenant situations. Other variables equal, households served by projects with a higher proportion of experienced certification staff tended to have lower gross errors (-.300 in log scale under Model 1 and -.201 under Model 7); and the effect was significant statistically albeit slightly reduced in magnitude after controlling for increasingly more variables. The implication is that, other conditions on project and tenant levels being equal, more experienced (and stable) staff helped reduce the gross error. Another remarkable finding was with TASS use. Households managed by projects that used TASS tended to have lower gross error, other things being equal. Again, the effect remained significant in Models 4 through 7, after strong effects of staff error and household variables entered. Two types of error made by staff appeared to predict higher gross error. One was overdue recertification, with estimates around 1.000 in log scale, suggesting that overdue recertifications could increase the average gross error substantially, around $7.00, holding other variables constant. Another type of staff error, transcription error, also related to higher gross error, with F-16

Appendix F: Multivariate Analysis estimated .931 under Model 5 and .679 under Model 7, equivalent approximately to an average dollar amount of $6, other things being equal. Model 7 generated estimates of household effects that were similar to those found in prior studies. Other things being equal, household with elderly or disabled household heads had lower gross rent error; higher earned and other incomes were also related to larger gross rent error; and complex financial conditions were related to higher gross error as shown in the significant estimates for the number of income and expense items and the number of allowances. The results from modeling gross rent error in FY 2007 were largely consistent with the findings from studies in prior years. Key points are: ♦ At the project-level, experienced staff and use of TASS in the certification operation were associated with lower gross rent error, given similar tenant characteristics and project conditions. ♦ Overdue certifications and transcription errors contributed substantially to gross error, given other project and household conditions being similar. ♦ Tenants with complex financial conditions (more sources of income, expense and allowance) and with relatively high incomes were likely to have larger gross rent error, other project and household conditions being similar. ♦ Compared with owner-administered programs, the Public Housing program had lower gross error, net of other factors on project and household levels. Other Types of Error. Binary and interval measures of types of staff error were analyzed using, respectively, multiple logistic regression and OLS regression. The logistic models specified the same set of tenant and project characteristics as predictor variables. Exhibit F-5 presents the logit estimates (log odds) and Max-rescaled R 2 from the logistic models of the eight types of error in dummy coding. 6 Max-rescaled R 2 (roughly interpreted as the proportion of variance accounted for by the model) fluctuated from 7 to 30 percent across models. Since effect size was not estimated for predictor variables and no attempt was made to run sequential modeling, we interpret the estimates by only stating whether or not a relationship between the predictor variable and the error existed based on the significant test (with p < 0.01). We emphasize the tentativeness of the results and hope to conduct further analysis to clarify the patterns. All else equal, the following relationships were found in the analyses: 6 We choose to present logit estimates rather than odds ratio because logits can be understood in a similar way as linear regression coefficients. The logistic regression models the relationship between the outcome Y=1 (a given error in our analysis) and the predictor variables through the logit function, the natural logarithm of odds of Y=1. The model assumes a linear relation between the log of odds and predictor variables, X 1 , X 2 , ... , X k , and can be written as: Let p=P(Y=1), then log (p/(1-p)) = intercept + b 1 X 1 + b 2 X 2 + ... + b k X k . Max-rescaled R 2 allows the maximal value of 1 and is recommended as a better approximate of the variance explained by the logistic model, comparable with generalized R 2 (Hosmer & Lemeshow, 2001). F-17

Appendix F: Multivariate Analysis ♦ Overdue Recertification Error. Households served by projects that reported perceived error causes were less likely to have overdue recertification error than those served by projects that reported no perceived error causes. ♦ Calculation Error (other than income and allowance). Households with higher pension income and more allowances were more likely to have error. ♦ Allowance Calculation Error. ƒ Households under projects of Public Housing program were more likely than those under owner-administered program to have allowance calculation error. ƒ Households with a higher other income (other than earned, public assistance, pension income) and greater number of allowance were more likely to have allowance calculation error. ♦ Income Calculation Error. ƒ Households served by project of large size were more likely to have income calculation error. ƒ Households served by project of Public Housing program were more likely than those served by owner-administered program to have income calculation error. ƒ Households served by projects where supervisor kept track of verification versus other staff kept track of verification were more likely to have income calculation error. ♦ Procedure Error ƒ Households served by projects that provided training to all experienced certification staff were more likely to have procedure error than those served by projects that did not provide training to all experienced certification staff. ƒ Households served by projects that used more methods to review certification were more likely to have procedure error. ƒ Households served by Section 8 projects were more likely to have procedure error than those served by owner-administered projects. ƒ Households served by projects that reported perceived error causes were more likely to have procedure error than those served by project that did not reported such perceived causes. ƒ The larger the number of households’ allowances, the more likely the procedure error to occur. F-18

Appendix F: Multivariate Analysis Exhibit F-5 (Page 1 of 2)

Different Types of Process Error Accounted For by Tenant and

Variable n Error i Error Error Error Error Error Procedure Error Error Project-Level Variables: Multiple Logistic Coefficients, Design Effect Adjusted Allowance Income Other Overdue Calculatio Admin strative Consistency Calculation Calculation Recertification Transcription Intercept -6.666 0.285 -1.852 -13.384 -8.227 -42.391 -1.212 -0.181 Project: Total 0.000 0.000 0.000 0.000 ** 0.000 0.000 0.000 0.000 number of units Project: Units per 0.005 -0.002 -0.002 0.001 0.003 -0.002 -0.001 -0.003 staff Project: Units per cert staff -0.001 0.001 0.002 0.001 -0.001 0.001 0.001 0.001 Project: Cert staff as -0.276 0.238 0.615 0.579 -1.304 * -1.551 0.315 0.372 a % of total staff Project:% cert staff -0.033 -0.408 ** 0.136 0.049 -0.466 -0.185 0.011 -0.361 * only do cert Project: Ongoing caseload -0.054 -0.033 0.067 -0.012 -0.270 -0.130 -0.067 0.052 Project:% cert staff 0.490 0.059 0.200 0.186 0.223 -1.569 * 0.224 0.069 5+ yr experience Project: Require hs 0.026 -0.155 -0.224 0.138 -0.408 -0.634 -0.192 -0.145 grad+ Project: Other requirement Project: All exp cert 0.738 0.127 0.000 0.094 -0.027 0.250 *** 0.601 0.090 -6.634 0.096 -7.718 0.063 0.146 0.170 ** 0.052 0.125 * staff trained Project: Public Housing Project: Section 8 1.504 0.396 * * 0.123 -0.003 1.036 0.635 * 1.644 0.680 ** 0.321 0.507 17.016 16.324 0.953 0.755 ** 0.078 -0.039 Project: Months process before -0.097 -0.007 -0.261 -0.072 -0.029 0.397 -0.204 -0.008 recert Project: Require interview for init cert 0.352 0.027 0.543 * 0.036 0.228 1.596 0.423 * -0.003 Project: Require -0.110 -0.006 -0.117 0.019 0.350 -0.153 -0.093 -0.056 interview for recert Project: N methods 0.234 * 0.024 0.162 *** -0.093 -0.075 -0.375 * 0.112 ** 0.024 used to review cases Project: Any policy/procedure 0.040 -0.023 0.012 -0.293 -0.117 -0.030 0.027 -0.012 change Project: HUD review files -0.345 * -0.068 0.149 * -0.375 * 0.177 -0.221 0.065 -0.125 * Project: N items 0.133 -0.023 0.021 0.045 0.028 0.086 0.018 -0.028 always verified Project: N items 0.193 -0.010 -0.072 0.113 -0.069 -0.616 -0.028 0.001 never verified F-19

Appendix F: Multivariate Analysis Exhibit F-5 (Page 2 of 2)

Different Types of Process Error Accounted For by Tenant and

Project-Level Variables: Multiple Logistic Coefficients, Design Effect Adjusted

Allowance Income Other Overdue

Calculation Administrative Consistency Calculation Calculation Recertification Procedure Transcription

Variable Error Error Error Error Error Error Error Error

Project: N items never verified 0.009 -0.010 0.036 -0.059 -0.055 0.106 0.034 -0.014 Project: N items never verified 0.170 -0.039 -0.065 0.505 ** -0.110 0.055 -0.045 -0.059 Project: Supervisor keep track ver requests 0.030 -0.028 -0.035 0.367 0.010 0.206 -0.018 -0.007 Project: N methods used to follow up 0.038 0.004 -0.076 0.235 0.124 0.019 -0.022 -0.005 Project: 100% used PIC/TRACS transfer -0.510 -0.078 -0.195 0.731 -0.297 -0.851 -0.170 -0.087 Project: Direct transfer to HUD 0.402 * 0.029 -0.056 0.183 0.214 -0.492 0.000 0.033 Project: Enterprise Income Verification (EIV) 0.510 0.044 -0.469 -0.876 -0.936 13.337 -0.357 0.070 Project: Used TASS last * yr 0.274 0.144 ** 0.050 -0.095 0.006 0.180 0.049 0.144 * Project: N activities using computer -0.116 -0.039 -0.085 * -0.108 0.001 -0.007 -0.062 * -0.045 Project: Frequency of TASS or EIV use last yr 0.045 0.048 0.009 0.114 0.085 -0.158 0.005 0.055 Project: Any perceived * errors -0.107 -0.042 -0.171 * -0.071 -0.148 -0.642 * -0.170 ** -0.024 Project: Staff shortage 0.747 -0.197 0.142 6.978 -0.459 * -0.484 0.098 0.036

Tenant: Elderly or

disabled household -0.449 -0.102 0.134 -0.118 -1.270 -0.825 -0.153 -0.028

Tenant: Earned income -0.201 0.479 -0.132 -0.406 0.310 * 0.153 -0.052 0.505 *

Tenant: Other income -0.935 * 0.082 -0.026 -0.516 0.194 0.361 -0.130 0.071

Tenant: Public

assistance income 0.467 0.357 * 0.229 -0.600 1.065 0.317 0.328 * 0.171

* Tenant: Pension income -0.280 0.109 -0.053 -0.016 0.448 * 0.257 -0.040 0.102 Tenant: Total number of sources of income/expenses 0.211 0.209 0.095 0.272 -0.080 0.172 * 0.108 0.200 Tenant: Total number of sources of * ** * ** income/expenses 0.728 * 0.373 * 0.420 *** -0.202 0.493 * -0.276 0.398 * 0.387 *** * * * * ** * * ** R 2 0.203 * 0.253 * 0.259 *** 0.204 ** 0.246 * 0.229 * 0.232 * 0.278 *** Source: HUDQC 2007 PSQ Survey merged with the Tenant Survey *p< 0.05 ** p<0.01 F-20

Appendix F: Multivariate Analysis ♦ Transcription Error ƒ Households served by projects that used TASS were more likely to have transcription error than those served by projects that did not use TASS. ƒ The larger the number of households’ allowances, the more likely the procedure error to occur. ♦ Administrative Error ƒ Households served by projects where certification staff did not work on tasks other than certification were less likely to have administrative error compared with those served by projects where certification staff also worked on other tasks. ƒ Households served by projects that used TASS were more likely to have administrative error than those served by projects that did not use TASS. ƒ The larger the number of households’ allowances, the more likely the administrative error to occur. Internal Measures of Staff Error. Three interval measures of errors—the numbers of transcription errors, of verification errors, and of all errors, were modeled in multiple linear regression analysis (Exhibit F-6). The patterns revealed in the models of the transcription errors and verification errors were largely consistent with the findings from logistic regression analyses of the binary coded transcription error and verification error as described above. In short, for most types of error, household higher incomes and complex financial conditions contributed to greater chance of occurrence. The patterns were comparable with those found in the gross rent error models. The estimates for using the TASS systems were confusing in relation to different measures of error. It was positively associated with the likelihood of administrative error and transcription error in binary code; but negatively associated with counts of transcription error, verification error, and the total errors. Also, the significant positive estimates for training for certification staff on consistence error and procedure error is perplexing. There may be unknown confounding factors that require further analysis. Summary Multivariate analysis was conducted to further examine gross rent error and different types of error that occurred in the certification/recertification process. The goal was to identify and estimate the relationship between tenant characteristics, project characteristics, and project practices and gross rent error. These analyses address study objectives 5, 6, 8, 12, and 13. A brief summary of the findings is provided below. A more detailed description of the analyses is found in Appendix F. To address Objective 5 (differences in error by program type), results from both bivariate tabulations and multiple regression analyses indicated that the Public Housing program had lower average gross rent error than the owner-administered program. In the multiple regression analysis, this difference remained statistically significant even after other project-level variables and tenant- level variables were held constant. The PHA-administered Section 8 program was found to have significantly higher gross error than the owner-administered program in bivariate tabulations, but F-21

Appendix F: Multivariate Analysis this difference was not found to be statistically significant in the multiple regression analysis. The implication being that the descriptive differences identified in bivariate tabulations are further accounted for by tenant and project factors. Given similar tenant and project characteristics, in comparison with the owner-administered program, the Public Housing program’s lower gross error persisted; whereas the PHA-administered Section 8 program’s higher gross error disappeared. The latter finding is important as it suggests that tenant characteristics for the Section 8 program may have imposed greater difficulty in quality control processes as they apply to rent calculation. Note that the finding from the FY 2006 multiple regression analysis was somewhat different, where Public Housing program did not show a significant difference from the owner-administered program. The PHA-administered Section 8 program did have significantly higher gross error than the owner-administered program. The shifting pattern of error difference cross administrative type requires continued research to understand the underlying factors that lead to the differences. To address Objective 6 (impact on error by tenants vs. project/staff), this analysis found that, consistent with previous results, the impact of tenant characteristics on rent error was apparently greater than project characteristics and practices. As shown in variance partitioning with HLM unconditional model and the sequential OLS models, tenants’ high income and multiple sources of income and allowance items were related to both higher gross rent error and different types of error. These tenant variables should be seen as important indicators of risk for rent errors and be targeted by program intervention. To address Objective 8 (error concentration in program/project), this analysis found that, to a varying extent, a number of program/project features did relate to rent error and different types of error. Specifically, projects that retained experienced certification staff (over five years) had lower average rent error. This finding is compatible with what was found in the FY 2006 analysis where staff educational requirements were related to lower rent error, namely, staff quality is a critical factor in reducing errors. We suggest focusing more on staff quality data collection and analysis in the future. One possibility is to collect additional staff information and job descriptions. The information would allow accurate comparison of staff quality across local providers in connection to the quality control results. To address Objective 12 (rent errors differentiated by using automated systems), the analysis provided evidence that computer application in certification operation helped reduce different types of processing error. Use of TASS or EIV, however, was not yet clear in consistently relating to rent error or different types of error. Again, more systematic data collection and analysis on computer application may help assess the impact of automation. For example, we may consider collecting data items on the respondents’ knowledge and skills specific to computer use in certification/verification, perhaps in a form of mini quiz. To address Objective 13 (other tenant or project characteristics related to errors), the analysis found that projects whose survey respondents reported perceived error causes tended to have lower chance of certain types of staff error. It is arguable that staff’s ability to report difficulties causing mistakes is equivalent to knowledge of the problems and hence potential ways to deal with them. However, far more information is needed to learn how project staff acquires information on different types of error in daily practice and the ways they identify and address risks of improper certification and payment determination. F-22

Appendix F: Multivariate Analysis Exhibit F-6 (Page 1 of 2)

Numbers of Transcription Errors, of Verification Errors, and of All Errors Accounted for by Tenant and Project-Level Variables:

Multiple Regression Coefficients, Design Effect Adjusted

N of transcription N of verification Intercept Project: Total number of units Project: Units per staff Project: Units per cert staff Project: Cert staff as a % of total staff Project:% cert staff only do cert Project: Ongoing caseload Project:% cert staff 5+ yr experience Project: Require hs grad+ Project: Other requirement Project: All exp cert staff trained Project: Public Housing project Project: Section 8 project Project: Months process before recert Project: Require interview for init cert Project: Require interview for recert Project: N methods used to review cases Project: Any policy/procedure change Project: HUD review files Project: N items always verified Project: N items never verified Project: N items never verified Project: N items never verified Project: Supervisor keep track ver requests Project: N methods used to follow up Project: 100% used PIC/TRACS transfer errors errors N of all errors 0.263 0.122 1.952 0.000 0.000 0.000 ** 0.000 0.000 -0.001 0.000 0.000 0.001 -0.006 0.013 0.144 -0.041 ** -0.014 -0.180 0.005 -0.006 -0.014 0.018 0.006 0.122 0.022 0.022 0.257 * -0.027 -0.047 -0.244 -0.021 -0.011 -0.233 *** 0.022 0.020 0.385 ** -0.004 0.010 0.162 -0.009 0.006 -0.115 ** 0.012 0.017 0.217 -0.008 -0.015 -0.043 0.004 0.001 0.048 0.002 0.007 0.049 0.023 -0.016 0.034 -0.001 -0.001 -0.001 0.001 -0.003 -0.017 -0.001 0.003 0.013 0.033 * -0.004 0.097 0.002 -0.007 -0.010 0.001 0.012 0.011 -0.013 0.001 -0.156 F-23

Appendix F: Multivariate Analysis Exhibit F-6 (Page 2 of 2)

Numbers of Transcription Errors, of Verification Errors, and of All Errors Accounted for by Tenant and Project-Level Variables:

Multiple Regression Coefficients, Design Effect Adjusted

N of transcription N of verification errors errors N of all errors Project: Direct transfer to HUD 0.000

Project: Enterprise Income Verification (EIV) 0.003

Project: Used TASS last yr -0.040

Project: N activities using computer -0.007

Project: Frequency of TASS or EIV use last yr 0.004

Project: Any perceived error 0.020

Project: Staff shortage -0.022

Tenant: Elderly or disabled household 0.002

Tenant: Earned income 0.063

Tenant: Other income 0.016

Tenant: Public assistance income 0.019

Tenant: Pension income 0.012

Tenant: Total number of sources of income/exp 0.017

Tenant: Total number of sources of income/exp -0.003

R 2 0.013 -0.041 -0.008 -0.116 ** -0.026 ** -0.219 ** ** 0.000 -0.046 ** 0.004 0.036 -0.003 0.165 0.055 0.164 -0.026 -0.142 *** -0.002 0.197 ** 0.059 *** 0.045 -0.013 0.299 ** 0.001 0.039 *** 0.002 0.140 *** 0.008 0.263 *** 0.105 *** 0.051 *** 0.166 Denominator DF 2324 2324 2324 Source: HUD Quality Control Survey, FY 2007 *p< 0.05 ** p<0.01 F-24

Appendix F: Multivariate Analysis So far, we have revealed a very limited impact of project factors on rent error. It is vexing that, as organized activities of managing housing assistance, project variation did not seem highly meaningful with the statistical analysis. As suggested in earlier analyses, we will continue to improve our data collection tools to gather more relevant information on quality control. In depth understanding of the actual operation of the local providers is needed to more accurately conceptualize the organization structures differentiated by program types. For instance, we ponder collecting data to cover issues that are decision-maker specific (e.g., the owner, the managing contractor, or the PHA), rather than data in reference to a generic project. Furthermore, we note the limitation of statistical analysis, which typically can only describe broad patterns and identify strong, clear-cut relationships between causal and effect factors. To learn the complex and subtle differences in housing program operation that involve vastly different individuals and groups, qualitative research is needed. Quantitative data collection and analysis, in fact, can be benefited tremendously by rich, deep, personalized qualitative observations of the local housing offices daily activities. References Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum Harrell, F.E. (2001). Regression modeling strategies. New York: Springer. Hosmer, D.W. & Lemeshow, S. (2001). Applied Logistic Regression, Textbook and Solutions Manual. New York: Wiley & Sons. Raudenbush, S. W. & Bryk, A. S. (2002). Hierarchical linear models: Applications and data

analysis methods (2 nd ed.). Thousand Oak, CA: Sage Publishing.

F-25