Deriving Local Trend Factors

for Fair Market Rent Estimation

Multidisciplinary Research Team

March 19, 2019

Prepared by: 2M

2M Research

1521 N. Cooper St., Suite 600

Arlington, TX 76011

Table of Contents

List of Exhibits ........................................................................................................................ iii

Tables

........................................................................................................................ iii

Figures

......................................................................................................................... v

Acknowledgments .................................................................................................................. vii

Executive Summary ............................................................................................................... viii

1 Introduction ..................................................................................................................... 1

1.1 Understanding of the Problem .............................................................................................................. 1

1.2 HUD’s Recent Efforts to Improve Trend Factor Calculations ................................................................... 1

1.3 Study Objectives ................................................................................................................................... 3

1.4 Contents of This Report ........................................................................................................................ 4

2 Data Sources and Issues ................................................................................................... 5

2.1 Dependent Variables ............................................................................................................................ 5

2.2 Exogenous Variables ............................................................................................................................. 5

2.3 Alternate Rent Series Using Commercial Data ....................................................................................... 7

3 Methodology .................................................................................................................... 9

3.1 The HUD Model .................................................................................................................................... 9

3.2 Model Building Strategy ...................................................................................................................... 10

3.3 Testing and Measuring Model Performance ......................................................................................... 13

4 Phase I Results ................................................................................................................ 15

4.1 Selecting Competing Rent and Utility Models ....................................................................................... 15

4.2 Forecast Error Comparison ................................................................................................................... 18

4.3 Coefficient Significance of Selected Model ........................................................................................... 21

4.4 Forecasting Rent and Utilities .............................................................................................................. 22

4.5 Estimating a Local Trend Factor ............................................................................................................ 24

5 Phase II Results ............................................................................................................... 27

5.1 Axiometrics Data ................................................................................................................................. 28

5.2 Methods to Assess the Data ................................................................................................................. 29

5.3 Zillow Data .......................................................................................................................................... 33

5.4 Axiometrics Forecasts .......................................................................................................................... 38

6 Accuracy of Alternative Models ...................................................................................... 40

7 Conclusion ...................................................................................................................... 44

i

Appendix A. Lag Selection Results for Exogenous Variables ................................................ A1

Appendix B. Residual Correlation Diagnostic Results .......................................................... B1

Appendix C. Parameter Estimate Results for the National Input Rent Model ...................... C1

Appendix D. Forecast Results .............................................................................................. D1

References ....................................................................................................................... 81

ii

LIST OF EXHIBITS

Tables

Exhibit 1-1 | Documents Received from HUD ...................................................................................................... 2

Exhibit 2-1 | Exogenous Variables Considered in the Development of Models .................................................... 7

Exhibit 3-1 | Model Selection for the United States Using Minimum Schwarz Bayesian Information Criterion .... 11

Exhibit 4-1 | Test for Unit Root and Stationarity for Rent Series ......................................................................... 15

Exhibit 4-2 | Test for Unit Root and Stationarity for Utility Series ...................................................................... 16

Exhibit 4-3 | ARIMA Model for Rent by Geographic Area ................................................................................... 17

Exhibit 4-4 | ARIMA Model Utility by Geographic Area ...................................................................................... 18

Exhibit 4-5 | Average Root Mean Square Errors Across Validation Period by ARIMA Specification and Geographic

Area, Rent Model .............................................................................................................................................. 19

Exhibit 4-6 | Average Root Mean Square Errors Across Validation Period by ARIMA Specification and Geographic

Area, Utility Model............................................................................................................................................ 20

Exhibit 4-7 | Parameter Estimates for the United States .................................................................................... 21

Exhibit 4-8 | Percentage of Significant Coefficients ............................................................................................ 22

Exhibit 5-1 | Cases and Geographic Coverage of Axiometrics Data ..................................................................... 27

Exhibit 5-2 | Analysis Variables Created from the Axiometrics Data ................................................................... 28

Exhibit 5-3 | Number (%) of Quarters in Each Primary Sampling Unit for the Metrics Comparing the Axiometrics

Rent Measure to the Bureau of Labor Statistics Measure ................................................................................... 32

Exhibit 5-4 | Summary of Metrics Comparing the Axiometrics Rent Measure to the American Community Survey

Measure for Core-Based Statistical Areas .......................................................................................................... 33

Exhibit 5-5 | Number (%) of Quarters in Each Primary Sampling Unit for the Metrics Comparing the Zil ow Rent

Measure to the Bureau of Labor Statistics Measure ........................................................................................... 37

Exhibit 5-6 | Summary of Metrics Comparing the Zil ow Rent Measure to the American Community Survey

Measure for Core-Based Statistical Areas .......................................................................................................... 38

Exhibit 5-7 | Comparisons of Axiometrics Data Forecasts to American Community Survey Values ...................... 39

Exhibit 6-1 | Comparison of Estimated Fair Market Rents for Select Primary Sampling Units, FY19 ..................... 40

Exhibit 6-2 | Comparison of Change in Rent from 2016 to 2017 ......................................................................... 41

Exhibit 6-3 | FY19 Core-Based Statistical Area Estimated Trend Factors by Region ............................................. 42

Appendix Tables

Exhibit A-1 | Lag Selection for Civilian Employment Using Minimum Schwarz Bayesian Information Criterion

(Panel A) .......................................................................................................................................................... A1

iii

Exhibit A-2 | Lag Selection for Civilian Employment Using Minimum Schwarz Bayesian Information Criterion

(Panel B) .......................................................................................................................................................... A1

Exhibit A-3 | Lag Selection for Residential Fixed Investment Using Minimum Schwarz Bayesian Information

Criterion (Panel A) ............................................................................................................................................ A2

Exhibit A-4 | Lag Selection for Residential Fixed Investment Using Minimum Schwarz Bayesian Information

Criterion (Panel B) ............................................................................................................................................ A3

Exhibit C-1 | Parameter Estimates for Northeast—Al Classes ........................................................................... C1

Exhibit C-2 | Parameter Estimates for Northeast—Midwest—Al Classes .......................................................... C1

Exhibit C-3 | Parameter Estimates for South—Al Classes .................................................................................. C2

Exhibit C-4 | Parameter Estimates for West—Al Classes ................................................................................... C2

Exhibit C-5 | Parameter Estimates for Northeast—Class A ................................................................................ C3

Exhibit C-6 | Parameter Estimates for Midwest—Class A .................................................................................. C3

Exhibit C-7 | Parameter Estimates for South—Class A ....................................................................................... C4

Exhibit C-8 | Parameter Estimates for West—Class A ........................................................................................ C4

Exhibit C-9 | Parameter Estimates for Boston ................................................................................................... C5

Exhibit C-10 | Parameter Estimates for New York ............................................................................................. C5

Exhibit C-11 | Parameter Estimates for Philadelphia ......................................................................................... C6

Exhibit C-12 | Parameter Estimates for Chicago ................................................................................................ C6

Exhibit C-13 | Parameter Estimates for Detroit ................................................................................................. C7

Exhibit C-14 | Parameter Estimates for Washington, DC ................................................................................... C7

Exhibit C-15 | Parameter Estimates for Baltimore ............................................................................................. C8

Exhibit C-16 | Parameter Estimates for Miami .................................................................................................. C8

Exhibit C-17 | Parameter Estimates for Atlanta ................................................................................................. C9

Exhibit C-18 | Parameter Estimates for Dal as ................................................................................................... C9

Exhibit C-19 | Parameter Estimates for Los Angeles ......................................................................................... C10

Exhibit C-20 | Parameter Estimates for San Francisco ...................................................................................... C10

Exhibit C-21 | Parameter Estimates for Seattle ................................................................................................ C11

iv

Figures

Exhibit 3-2 | Residual Correlation Diagnostic for First-Differenced National Rent Series ..................................... 12

Exhibit 3-3 | Residual Correlation Diagnostic for First-Differenced National Utility Series ................................... 13

Exhibit 4-9 | Forecasts of National Quarterly Rent of Primary Residence ............................................................ 23

Exhibit 4-10 | Forecasts of National Quarterly Housing Fuels and Utilities .......................................................... 24

Appendix Figures

Exhibit B-1 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Northeast—Al Classes

........................................................................................................................................................................ B1

Exhibit B-2 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Midwest—Al Classes

........................................................................................................................................................................ B2

Exhibit B-3 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, South—Al Classes ... B2

Exhibit B-4 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, West—Al Classes .... B3

Exhibit B-5 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Northeast—Class A . B3

Exhibit B-6 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Boston .................... B4

Exhibit B-7 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, New York ................ B4

Exhibit B-8 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Philadelphia ............ B5

Exhibit B-9 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Midwest—Class A ... B5

Exhibit B-10 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Chicago ................. B6

Exhibit B-11 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Detroit .................. B6

Exhibit B-12 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, South—Class A ...... B7

Exhibit B-13 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Washington, DC .... B7

Exhibit B-14 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Miami ................... B8

Exhibit B-15 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Atlanta .................. B8

Exhibit B-16 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Baltimore .............. B9

Exhibit B-17 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Dal as .................... B9

Exhibit B-18 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, West—Class A ......B10

Exhibit B-19 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Los Angeles ..........B10

Exhibit B-20 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, San Francisco .......B11

Exhibit B-21 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Seattle .................B11

Exhibit D-1 | Forecasts of Quarterly Rent and Utility, Northeast—Al Classes .................................................... D1

Exhibit D-2 | Forecasts of Quarterly Rent and Utility, Midwest—Al Classes ...................................................... D2

Exhibit D-3 | Forecasts of Quarterly Rent and Utility, South—Al Classes .......................................................... D2

Exhibit D-4 | Forecasts of Quarterly Rent and Utility, West—Al Classes ........................................................... D3

v

Exhibit D-5 | Forecasts of Quarterly Rent and Utility, Northeast—Class A ......................................................... D3

Exhibit D-6 | Forecasts of Quarterly Rent and Utility, Boston ............................................................................ D4

Exhibit D-7 | Forecasts of Quarterly Rent and Utility, New York ........................................................................ D4

Exhibit D-8 | Forecasts of Quarterly Rent and Utility, Philadelphia .................................................................... D5

Exhibit D-9 | Forecasts of Quarterly Rent and Utility, Midwest—Class A ........................................................... D5

Exhibit D-10 | Forecasts of Quarterly Rent and Utility, Chicago ......................................................................... D6

Exhibit D-11 | Forecasts of Quarterly Rent and Utility, Detroit .......................................................................... D6

Exhibit D-12 | Forecasts of Quarterly Rent and Utility, South—Class A .............................................................. D7

Exhibit D-13 | Forecasts of Quarterly Rent and Utility, Washington, DC ............................................................ D7

Exhibit D-14 | Forecasts of Quarterly Rent and Utility, Miami ........................................................................... D8

Exhibit D-15 | Forecasts of Quarterly Rent and Utility, Atlanta ......................................................................... D8

Exhibit D-16 | Forecasts of Quarterly Rent and Utility, Baltimore ...................................................................... D9

Exhibit D-17 | Forecasts of Quarterly Rent and Utility, Dal as ............................................................................ D9

Exhibit D-18 | Forecasts of Quarterly Rent and Utility, West—Class A ............................................................. D10

Exhibit D-19 | Forecasts of Quarterly Rent and Utility, Los Angeles ................................................................. D10

Exhibit D-20 | Forecasts of Quarterly Rent and Utility, San Francisco .............................................................. D11

Exhibit D-21 | Forecasts of Quarterly Rent and Utility, Seattle ........................................................................ D11

vi

ACKNOWLEDGMENTS

The study team would like to thank Gail Clark, Joshua Townley, MacKenzie Regier, and Cindy Romero for providing invaluable editing and formatting support throughout the study. The study team would also like to acknowledge our partners at the U.S. Department of Housing and Urban Development’s Office of Policy Development and Research, with specific assistance from Ransford Osafo-Danso, Chalita Brandly, Peter Kahn, Adam Bibler, and Marie Lihn.

Disclaimer

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

vii

EXECUTIVE SUMMARY

The U.S. Department of Housing and Urban Development (HUD) estimates Fair Market Rents (FMRs) for most geographic areas in the United States to set reimbursement values for public programs that address the housing needs of low-income families and households. Because reliable data on rents lag behind the dates that FMRs are needed by, HUD must forecast the FMRs based on trends in the available data. HUD’s current methodology includes using a “trend factor” to forecast values several quarters ahead of the most current data from the Bureau of Labor Statistics (BLS). The trend factor is the same for all locations, meaning that areas with relatively high (low) rent inflation will be assigned FMRs that are too low (high). This study examines two ways that HUD could add geographic resolution to the trend factors and, perhaps, reduce concern about the accuracy of FMRs.

The first method relies on BLS data on 22 geographic areas and Autoregressive Integrated Moving Average (ARIMA) modeling to estimate different trend factors that can be compared to HUD’s current estimates. Using standard time-series modeling metrics, most of the area-specific estimates appear to be more accurate than the current trend factor. However, the overall impact of using this method is relatively small. In other words, FMRs calculated from HUD’s current method are reasonably close to the FMRs that would be calculated from the 22 areas.

The second method uses commercial data from Axiometrics—which are shown to be more appropriate than data from Zillow—to estimate ARIMA models for up to 254 Core-Based Statistical Areas (CBSAs).

The findings from this approach suggest that FMRs in several areas may be understated by using HUD’s single national trend factor.

The study also highlights the limitations of available data sources, which wil need to be considered if HUD migrates toward localization of trend factors. In particular, standardized public data are less available for smaller metropolitan areas, forcing more reliance on commercial data sources.

viii

1 INTRODUCTION

1.1 Understanding of the Problem

The U.S. Department of Housing and Urban Development (HUD) estimates Fair Market Rents (FMRs) for more than 600 metropolitan areas and almost 2,000 non-metropolitan county areas. FMRs are used to determine reimbursement rates in several federal housing programs and must be published at least 30

days before the start of each fiscal year (FY), making FMR estimates of the gross rent (the sum of shelter rent and necessary utilities) for the upcoming FY.

Within HUD’s current methodology, FMRs reflect three components: (1) the base rent, as estimated with data from the American Community Survey (ACS), (2) an inflation factor that is applied to the base rent, and (3) a trend factor. The inflation factor uses actual values on inflation to bring the base rent to current values, while the trend factor uses forecasted values to bring the current values of the base rent to the future values, which will be published as the FMRs. HUD currently uses components of the Consumer Price Index (CPI) to determine inflation factors in the 22 geographic areas where the components are produced and to determine regional values for the remaining areas of the country.

Therefore, the inflation factors experience some “localization.” In contrast, HUD currently uses a national estimate of the trend factor for all geographic areas.

The national trend factor is used to adjust local area rents applied over seven quarters. In areas where rent growth varies significantly from the national trend, FMRs may not accurately explain current local market conditions and may not capture market rent difference across regions. To address this limitation, HUD is considering the consequences of implementing localized trend factors. This consideration explains why “the goal of this research is to explore and provide a set of alternative FMR trend factor methodologies, analyze the strengths and weaknesses of each, and recommend a methodology that is most in line with current local market conditions and can improve accuracy of trend forecasting for FMR

calculations” (MDRT Task Order: Local Trend Factor Research Project: 5). Both the current inflation factor and the current trend factor methodologies face similar constraints, mainly that reliable, relevant public data—such as those produced by the Bureau of Labor Statistics (BLS)—are not available for all geographic areas of interest to HUD. This means that research on localizing the trend factor may be useful for further localizing the inflation factor. For example, assume that the study team finds that trends from a readily available series from Zillow closely match trends in BLS’s Rent of Primary Residence series in the geographic areas where both series exist. In that case, HUD may wish to consider using Zillow trends to further localize the inflation factor in the geographic areas covered by Zillow but not BLS. Therefore, while the scope of this Task Order (TO) is primarily confined to the trend factor, the research may spill over to issues related to the inflation factor.

1.2 HUD’s Recent Efforts to Improve Trend Factor Calculations

To address the limitations with the national trend factor, HUD has explored options to improve calculations of the trend factor. HUD has explored alternative methodologies, such as (1) using national 1

input variables to forecast local rent and utilities of primary residence from CPI and (2) adding measures of local supply and demand into the rent component of the national model. To account for delayed changes in residential rent due to changes in housing supply and demand, variables like local and national construction permits, oil and gas prices, and a utility index are also included in the model by differencing them across 10 years and adding their lags by two quarters. The two components of the gross rent index (GRI) are forecasted at the smallest geographic area available from CPI data. Local rent and utility costs are used as dependent variables in the model’s run to capture differences in market rents and current market conditions across FMRs.

The phrase “gross rent” is used to describe the theoretically correct value for the FMR. As noted earlier, HUD uses a base value for gross rent from ACS, which is usual y at least 2 years old, and then applies inflation and trend factors. In HUD’s work-to-date, the trend factors were derived from the GRI, which was the weighted average of the Rent of Primary Residence Index and the Fuels and Utilities Index (both components of CPI), for which the weights are based on the rent and utility shares of gross rent. Rent and utility shares are obtained from ACS and applied to the forecasted rent and utility estimates. With actual and forecast values of the GRI, HUD then computed some hypothetical local trend factors as the ratio of the forecast (next year) values to the actual (current year) values. 1

HUD’s initial analyses dealing with localizing the trend factor provide the point of departure for this TO.

To thoroughly understand and assess the prior work conducted by HUD, the study team received HUD’s background information and the data used to conduct the estimations. Exhibit 1-1 presents a description of the received files.

Exhibit 1-1 | Documents Received from HUD

#

File

Description

1

Approach FY19

HUD’s recent work on local trend factor

2

PUC Forecast

Documentation on voucher per-unit cost (PUC) forecast, from which

the national model is derived

3

Report Example

Example of desired template of the final report

4

Subscriptions

Information on data to which HUD’s Office of Policy Development and

Folder

Research has a subscription

Proposals to

Summary of the main issues in estimating FMRs

5

Update the FMR

https://www.huduser.gov/portal/sites/default/files/pdf/Proposals-To-

Formula

Update-the-Fair-Market-Rent-Formula.pdf

6

Gross_Rent_Al

A table of all national, regional, and local parameters used in models,

by primary sampling unit (PSU) and quarter

7

Gross Rent

Component Shares A table of FY19 gross rent component weights

8

Cpi_psu2

Crosswalk of CPI PSU geographies to 2010 Federal Information

Processing Standards (FIPS) codes

9

Local Trend Factor

FY19

SAS code documenting processes used in calculating local trend factor

Note: CPI denotes “Consumer Price Index,” FMR denotes “Fair Market Rent,” and FY denotes “fiscal year.”

1 From the HUD background paper, “Forecasting Regional and Metro FMR Trend Factor,” September 2018.

2

1.3 Study Objectives

The purpose of this report is to present the results and rationale for a set of models that can be used to compute locally based FMRs. More specifically, this report presents alternative approaches to augmenting local market conditions into calculations of FMRs, starting with extensions of HUD’s existing methodology and followed by consideration of models that use alternative sources of data and empirical frameworks. Consequently, this report presents the results of a two-phase research approach with multiple research questions in each phase. The objective of the first phase was to derive a local trend factor using a model similar to the model HUD used in forecasting gross rents national y. The study team identified six research questions that address the objective of the first phase—

1. What is the best model form for forecasting gross rents locally?

2. What lags are deemed appropriate for illustrating delayed responses in gross rent changes to changes in variable inputs?

3. How do the errors of local forecast estimates compare to the errors of the national forecast estimates?

4. What is the maximum range of error considered acceptable for a local estimate before reverting to the use of a national trend or regional factor?

5. What is the best approach for measuring the accuracy of local trend factors?

6. What is the best approach for calculating estimates for non-metropolitan areas?

In the second phase, the objective was to identify alternative approaches (other than forecasting gross rents) that derive a local trend factor that result in a more accurate FMR calculation. There are six research questions associated with the Phase II objective—

1. What variables best describe supply and demand factors that can be used as predictors of changing gross rents for metropolitan areas?

2. What is the possibility of utilizing forecasted data series from state or metropolitan entities?

3. How are local trend factors calculated using the variables to be identified in question 1 of Phase II?

4. What is the best approach for measuring the accuracy of estimates?

5. What is the accuracy of these local trend factors compared with the accuracy of trend factors derived from local and national gross rent forecasting?

6. Are there metropolitan areas where using a local trend factor consistently works better than using a national trend factor?

The findings in this report address the development of a locally based trend factor that may enhance HUD’s recent modeling on forecasting gross rents with BLS CPI components on rent and utilities in Phase I. The findings also address the determination of an ideal dataset that is not confined by the geographies produced by BLS, which may provide opportunities to expand the localization of trend factors and explore alternative geographic definitions in Phase II.

3

1.4 Contents of This Report

The outline of this report is as follows: following this introduction, section two provides a summary of the data used in the study. Section three provides an overview of the methodologies and model specification used in the study. Section four presents a set of selected models for computing a local trend factor. Section four also describes the rationale for the models’ selection and compares their forecasting accuracy against that of HUD’s hypothetical model and presents results for the selected models in computing a local trend factor for 22 geographic areas. 2 Using commercial data, an expanded set of geographies is analyzed in section five, as wel as a comparison of data from Axiometrics and Zillow. Section six includes comparisons of the accuracy of various models with comparisons that show the implications for FY19 FMRs of the various models. Finally, section seven offers our conclusions.

2 The areas are defined by the availability of quarterly rent indices from BLS.

4

2 DATA SOURCES AND ISSUES

Data on supply and demand factors of gross rents are available from a wide variety of sources, including government agencies and private research entities, as identified by HUD. The study team used the data obtained from HUD, including the Rent of Primary Residence Index and the Fuels and Utilities Index, residential fixed investment, local permit data, and local and national employment data, among others.

We also evaluated additional local supply and demand variables to augment the analysis in Phase I, as well as to address the research questions in Phase II. This section describes the data the study team used to test the different model specifications that we evaluated in deriving local trend factors.

Specifically, in this section, we summarize the data on the response variables that we used as dependent variables, and we summarize data on exogenous variables that we used as independent variables in the model specifications. This section also discusses the considerations that were relevant in choosing additional variables (other than the variables that were used in the HUD model) to augment the specified models.

2.1 Dependent Variables

The dependent variables in the study were the Rent of Primary Residence Consumer Price Index (CPI) and the Fuels and Utilities CPI. These two components of CPI for the time interval of January 1995

through January 2018 were obtained from HUD for (1) the United States as a whole, (2) four regions (regional data were used for areas where metropolitan data were not available in the CPI, examined at size class levels—All Classes and Class A3), and (3) 13 metropolitan areas of the United States. Given that these indices are available in the CPI only on a monthly basis for some areas and bimonthly for other areas, HUD converted them to quarterly averages for regions and metropolitan areas. For regions and the three metropolitan areas (New York, Chicago, and Los Angeles) where monthly CPI data are available, HUD averaged the monthly data for each quarter. For the remaining 10 metropolitan areas (Philadelphia; Detroit; Washington, DC; Miami; Atlanta; Boston; Baltimore; Dallas; San Francisco; and Seattle) where CPI data are available bimonthly, HUD used the PROC EXPAND function in Statistical Analysis Software (SAS) to interpolate quarterly estimates for these series. The Rent of Primary Residence CPI is based on repeat surveys of a sample of rental housing units over 6-month periods, with adjustments for aging of the units and vacancies. For this reason, the Rent of Primary Residence CPI is considered a strongly reliable measure of changes in rents, particularly in those areas for which the Bureau of Labor Statistics (BLS) produces local estimates.

2.2 Exogenous Variables

As part of our efforts to improve the forecast accuracy of the estimated models, the study team evaluated and incorporated additional variables that were found to have relationships with the dependent variables. Exogenous variables were selected based on a priori analysis to discern the 3 Based on the BLS definition, “Al Classes” represent al metropolitan and non-metropolitan areas in each region; “Class A”

represents al metropolitan areas with populations of more than 2.5 mil ion.

5

response and a stepwise regression approach. First, variables identified to enhance the rent and utility series forecast models were evaluated based on their impact on the response series, mathematical y termed as correlation. Using the Pearson’s correlation coefficient, the study team listed the identified variables that had a significant impact on the response series. Second, we evaluated the correlation between the chosen independent variables. If one independent variable is excessively correlated with another (multicollinearity), determining their separate influences would be difficult and could result in regression coefficients being sensitive to model specification when both variables are included. For a forecast using exogenous variables, future values of the input variables are estimated using its own past values (such as an Autoregressive Integrated Moving Approach process). To reduce the sensitivity of models, individual variables are added to each model, and the accuracy statistics are calculated based on the coefficient estimation set or training and calibration sets. If the inclusion of a variable enhances model accuracy, the variable is retained. In the ensuing sections, we have described the exogenous variables used for the development of the multivariate models. A summary of the description of these variables is provided in exhibit 2-1.

2.2.1 RENT MODEL VARIABLES

To improve the forecast model for rent, the study team explored additional variables. These variables included seasonally adjusted and unadjusted values of occupancy rate and employment data for metros and U.S. Census Bureau (Census) regions. To capture changes in rent with fluctuations in the labor market, we explored employment data like total employment, unemployment rate, manufacturing employment, labor force participation, and weekly wages. Since these variables are highly correlated to each other and explain similar mechanisms in the labor market, the study team individually included weekly wages and the unemployment rate to forecast rent. The employment data span 21 years (or 84

quarters), while occupancy rates span more than 40 quarters. The model specifications included occupancy rate, unemployment rate, and weekly wages as individual inputs in the forecast model. Using a correlation matrix, the study team based these variables on their impact on the response series. The resulting model specifications were built on the principle explained below and did not contribute to explaining the underlying patterns in the response series; hence, they were not included in further analysis.

2.2.2 UTILITY MODEL VARIABLES

Residential prices for electricity at the state level were used to forecast utilities. The study team seasonally adjusted the series and assessed its impact on the response series. Since the data for electricity are only available at the state level, our forecast models are built for the 13 metro areas that could be mapped to states. CPI indices for energy services and commodities in metros and Census regions were explored and their correlation was examined. Since these indices were significantly4

correlated with the dependent variable and were derived from the dependent variable, these indices were dropped from further analysis.

4 They are considered significant at 95-percent confidence level.

6

Exhibit 2-1 | Exogenous Variables Considered in the Development of Models

Data

Source

Geographic Unit

Data Frequency Data Span

Phase I

State; Core-Based

Total Employment

BLS

Statistical Area

Monthly

1997 to

(CBSA)

2018

Manufacturing

Employment

BLS

State; CBSA

Monthly

1997 to

2008*

Unemployment Rate BLS

State; CBSA

Monthly

1997 to

2018**

Occupancy Rate

Axiometrics (from

HUD)

County; Tract

Monthly

2008 to

2018

Vacancy Index

U.S. Postal Service

State; County;

(from HUD)

Tract

Quarterly

2008 to

2018

State; Metropolitan

Weekly Wages

BLS

Statistical Area

Quarterly

1995 to

(MSA)

2016

Energy CPI

BLS

State; MSA

Quarterly

1997 to

2018

Electricity CPI

BLS

State; MSA

Quarterly

1997 to

2018

Utility (Piped) Gas

Service CPI

BLS

State; MSA

Quarterly

1997 to

2018

Gasoline (All Types)

CPI

BLS

State; MSA

Quarterly

1997 to

2018

Residential Electricity Energy Information

Prices

Administration

State

Quarterly

2001 to

2018

Phase II

Asking Rent and

Axiometrics (from

Concessions

HUD)

County; Tract

Monthly

2008 to

2018

Median Rental Price

(All Home Types)

Zillow

MSA

Monthly

2010 to

2018

Median Home Values Zillow

MSA

Monthly

1996 to

2018

Housing Price Index

Federal Housing

State***; MSA;

Finance Agency

CBSA

Quarterly

1975 to

2018

* Non-adjusted “al employees” data available back to 1939, at the state level.

** Metro data begins in 1990.

*** For non-metropolitan areas, data are available by state and between 1995 and 2008.

Note: BLS denotes “Bureau of Labor Statistics” and CPI denotes “Consumer Price Index.”

2.3 Alternate Rent Series Using Commercial Data

The objective of Phase II (research question 2) is to determine alternative approaches for adding more localization to Fair Market Rent (FMR) estimates. To this end, the study team focused primarily on 7

expanding the number of metro areas (core-based statistical areas, CBSAs) by using extant data from alternative sources to create quarterly rent series for several geographic areas.

8

3 METHODOLOGY

As stated previously, the primary objective of this study is to evaluate methodologies that improve trend forecasting for Fair Market Rent (FMR) calculations. Given that the trend factor component of the FMR

uses forecast values to bring the current values of the base rent to the future values, it is important to use a procedure that produces reliable local rent forecasts to derive the needed improvements in the trend factor. In this study, we used the Autoregressive Integrated Moving Average (ARIMA) model to estimate the future values of local rent and utilities, based on which trend factors for each area were calculated. ARIMA models apply to stationary time series and assume that a time series is a linear combination of its own past values and current and past values of an error term (Box and Jenkins, 1976).

Consequently, ARIMA models are generally specified in terms of three different parameters ( p, d, and q)

where p is the order of the autoregressive term (AR term), d denotes the degree of integration to achieve stationarity, and q is the order of the moving average term (MA term).

The remainder of this section outlines and discusses the types of ARIMA models that were specified and the rationale for choosing the best model used for forecasting. As a prelude to this discussion, section 3.1 presents the structure of the ARIMA model that HUD used in recent efforts to improve trend factor calculations. We use the HUD model as a baseline with which we compared other models under consideration. By making this comparison, we intend to identify the areas where HUD’s model is working or not.

3.1 The HUD Model

HUD used an ARIMA (0,1,1) model for the rent series in all areas and an ARIMA (1,1,0) model for all the utility series. Therefore, the current rent depends on the previous error term (shocks), not the previous period rent. To be specific, an MA (1) model is estimated for the rent growth rates or the first-differencing of rent series. Similarly, an AR (1) model is estimated for the utility growth rate. HUD

estimated the rent model using two approaches. The first approach was to estimate the model based on national exogenous variables. 5 In the second approach, HUD estimated the rent model based on local exogenous variables. 6 However, the utility model was estimated using only national exogenous variables. 7 To reflect the time delay of the response to the identified exogenous variables, each of these variables was lagged two quarters based on theoretical considerations. In this study, we focused on extending HUD’s recent work to find alternative time series models to the current national model.

5 The national exogenous variables used in HUD’s rent model are the Bureau of Economic Analysis (BEA) National Residential Fixed Investment and the Bureau of Labor Statistics’ (BLS) National Civilian Employment data.

6 The local exogenous variables for the rent model comprised the U.S. Census Bureau’s Local Permit data and BLS Local Employment data.

7 The exogenous variables used in HUD’s utility model are the national quarterly average spot price in dol ars per barrel of West Texas Intermediate crude oil, the quarterly national average price in dol ars per short ton of bituminous coal, the quarterly average Henry Hub price of natural gas in dol ars per mil ion BTUs, and the Consumer Price Index for All Urban Consumers.

9

3.2 Model Building Strategy

Building on HUD’s recent work, the study team estimated a set of models for forecasting gross rents for all 22 geographic areas. We extended HUD’s specification in two different directions: first, we estimated different ARIMA models for the gross rent of each area in the belief that local rent series have a different dynamic (trend) than the national counterpart. Second, we also estimated rent growth in the framework of a multivariate time series model by augmenting exogenous variables. The selection of a proper model is extremely important because it reflects the underlying structure of the series. This fitted model, in turn, is used for future forecasting. We followed the three-stage iterative model-building procedure developed by Box and Jenkins (1976): identification, estimation, and diagnostic checking. The identification stage involves transforming the data (if necessary) to improve the normality and the stationarity of the time series and to determine the general form of the model to be estimated.

During the estimation stage, the model parameters are estimated using the method of moments, least square methods, or maximum likelihood methods. Finally, diagnostic checks of the model are performed to reveal possible model inadequacies and to assist in selecting the best model. These stages are discussed in more detail below.

3.2.1 MODEL IDENTIFICATION

The study team began the modeling process by identifying the appropriate model for the rent and utility series, as well as the series for the exogenous variables for each geographic area. Identification of the general form of an ARIMA model involves two steps. First, the data series is analyzed for stationarity and normality. If necessary, to achieve stationarity and normality, the study team performs appropriate differencing of the series. This step is also necessary to determine the integration d order of the time series. All variables used in this study were tested for stationarity (against unit roots) so that the mean, variance, and autocovariances were independent of time. We employed two formal testing tools, the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test and the Dickey-Fuller Generalized Least Squares (DF-GLS) test, to examine stationarity of time series under study. The KPSS test is adopted mainly as a confirmatory tool to the more popular tests under the opposite nul hypothesis. The KPSS test is constructed under the nul of stationarity, in contrast to the nul hypothesis of unit root for the DF-GLS

test. Both tests suggest that the original series of all the variables under study are non-stationary, regardless of the geographic areas. This conclusion led us to transform the original series by taking a first-difference—that is, the growth rates of each series.

We then apply the autocorrelation function (ACF) and partial autocorrelation function (PACF) in the spirit of Box and Jenkins (1976) to the growth rate of each series to identify their underlying dynamics.

The ACF and PACF are helpful in confirming the stationarity of the series and identifying whether the underlying model follows an AR, MA, or ARIMA model. Once the series is differenced and the underlying model is identified, we use an information criterion method to determine the appropriate lag length of the input series used in the forecast model. The results of lag selection for the exogenous variables are presented in appendix A. We compute the information criteria using various ARIMA model orders, with an automated function used to determine the range of the autoregressive model orders. To be specific, we use the Schwarz Bayesian information criterion (SBC) for the lag length selection in each model. The basic idea of the SBC rule is to find an optimal model specification based on the trade-off between performance in fitting available data and minimum model complexity. Hence, lower SBC implies either 10

fewer explanatory variables, better fit, or both. Given any two estimated models, the model with the lowest value of SBC is preferred. Exhibit 3-1 reports the results from this exercise for the rent and utility series at the national level. As shown in the exhibit, the SBC rule suggests that the AR (1) model is best fitting for the national rent growth series and the MA (1) model best fits national utility growth series.

Given that the growth series are the first-difference of the original series, this outcome implies that the national rent is best approximated by an ARIMA (1,1,0) model and the national utility series is best captured by an ARIMA (0,1,1) model. Note that the order “d” in ARIMA (p,d,q) denotes the order of integration; hence ARIMA (p,1,q) model for the original series is equivalent to the ARMA (p,q) model for its first-difference. These parameters were then used at the next stage of estimation.

Exhibit 3-1 | Model Selection for the United States Using Minimum Schwarz Bayesian Information Criterion

Rent Model (NIM)

Utility Model (PTS)

AR Order

MA Order

SBC

AR Order

MA Order

SBC

0

1

– 881.75

0

1

– 486.52

0

2

– 890.62

0

2

– 484.10

1

0

– 924.27

1

0

– 486.19

1

1

– 920.04

1

1

– 483.55

1

2

– 915.70

1

2

– 479.66

2

0

– 920.03

2

0

– 484.70

2

1

– 917.28

2

1

– 482.36

Notes: AR denotes “autoregressive,” MA denotes “moving average,” SBC denotes “Schwarz Bayesian information criterion,”

NIM denotes “National Input Model,” a multivariate model with national exogenous variables, and PTS denotes “Pure Time Series” model, a univariate time series model without exogenous variables. Details of the various model specifications are discussed in section 4.

3.2.2 PARAMETER ESTIMATION

After candidate models have been determined from the identification stage, the study team used the parameters ( p and q) of these models in the estimate statement of PROC ARIMA in SAS to obtain estimates using maximum likelihood estimation.

3.2.3 MODEL DIAGNOSTIC CHECK

Having identified and estimated the parameters of the model in the previous stages, our next step was to verify the adequacy of the fitted models. The study team performed validation tests on the residual series to determine if any patterns remained unaccounted for. A model is said to be adequate if the residuals are white noise—that is, if no significant correlation is observed among the residuals of the fitted model. We tested for the adequacy of our estimated models by graphical y inspecting the ACF and the PACF of the models’ residuals. The residuals of the ACF more importantly provide information about the independence of the residuals; ACF residuals that are significantly different from zero indicate that the model does not adequately represent the data. The residual correlation for the first-differenced rent and utility series for the nation are shown in exhibits 3-2 and 3-3. The plots for the remaining 22

geographic areas are presented in appendix B. The plots indicate no significant correlation between residuals. We conclude that the rent and utility models selected in the parameter estimation stage for 11

the entire nation appear to be adequate and representative of the data. Consequently, we propose using the national model, especial y in cases—such as Boston, Chicago, and Seattle—where there are still significant lags.

Exhibit 3-2 | Residual Correlation Diagnostic for First-Differenced National Rent Series 12

Exhibit 3-3 | Residual Correlation Diagnostic for First-Differenced National Utility Series 3.3 Testing and Measuring Model Performance

From the foregoing procedures, the study team developed three competing models for both rent and utility series for all 22 geographic areas including the nation as a whole. The first model is a univariate time series model using its own past values and error terms. The second is a multivariate time series model that includes other national variables in addition to its own lagged terms. The third is a multivariate model that includes just local explanatory variables in addition to its own lagged terms.

Prior to forecasting, the estimation results of univariate models were compared to those of the multivariate approaches, as well as to HUD’s model to evaluate each model’s forecast accuracy. The response series of each model was divided into two sets: the sample period and the validation period.

The study team estimated each specification over seven overlapping sample periods, where each period included 15 years’ worth of data. The seven sample periods are: 1996 to 2011, 1997 to 2012, 1998 to 2013, 1999 to 2014, 2000 to 2015, 2001 to 2016, and 2002 to 2017. Estimation over multiple time periods provides a better basis for comparing the model’s predictive power, as the results from a particular time period are determined by the specific economic conditions in effect during that period, which may not represent the long-term conditions. To compare the different models, we developed statistics reflecting each model’s post-sample forecast accuracy for the validation period subsequent to the sample periods. The length of the validation period varied depending on the sample period that allowed us to develop forecasts that could be compared to the actual data points. For example, for the estimation period 1996 to 2011, we used a longer period 2012:1 to 2018:1 (21 quarters) as the validation, while for the estimation period 2002 to 2017, the validation period comprised just one quarter. The forecasts of the validation period were compared with the actual data for each of the three approaches and were also compared with HUD’s forecasts. We conducted this comparison to reveal 13

how closely the rent and utility predictions of the validation period tracked the corresponding historical data. We used the Root Mean Square Error (RMSE) statistic to compare the forecast performance of each model. RMSE, a widely used criterion for evaluating forecasting performance (Armstrong and Collopy, 1992), provides a measure of the deviation of the true from the forecasted values. Another measure that can be used to evaluate model performance is the Mean Absolute Percentage Error (MAPE). MAPE provides the difference between true and forecast values divided by the true value. We compute MAPE and RMSE to validate the forecast models. However, RMSE is more suited for our analysis, since the series are comparable (Chai and Draxler, 2014). Hence, further analysis is restricted to using RMSE as the measure of forecast model performance. We computed the errors for each time path, from which we calculated average RMSE across the validation period for each model and for each geographical area. Lower RMSE values denote a better forecasting performance. Based on this information, ex-ante forecasts from April 2018 to October 2020 for rents and utilities for each geographic area were made for the validation period.

14

4 PHASE I RESULTS

This section presents results of analyses related to research question 1. The analyses were conducted with HUD’s master data file and can be thought of as testing model specifications against HUD’s initial work. First, we estimated univariate time series models in which the dynamics of a series are solely driven by its own lagged terms. Second, we estimated multivariate time series models by augmenting other housing market fundamental variables. The utility component of rent is forecasted using as exogenous variables the national quarterly average spot price in dollars per barrel of West Texas Intermediate crude oil, the quarterly national average price in dollars per short ton of bituminous coal, the quarterly average Henry Hub price of natural gas in dollars per million BTUs, and the Consumer Price Index (CPI) for All Urban Consumers (CPI-U).

4.1 Selecting Competing Rent and Utility Models

As stated in the previous section, the first step in the specification of each Autoregressive Integrated Moving Average (ARIMA) model is to assess the stationarity of each series. Unit root tests were estimated for each rent and utility series, with the appropriate degree of differencing then adopted in the estimation and the assessment of the best-fitting model. We run two statistical unit root tests, the Dickey-Fuller Generalized Least Squares (DF-GLS) test and the Kwiatkowski-Phillips-Schmidt-Shin (KPSS) test, to increase the accuracy of the results about the stationarity of the series. All tests were performed with an added trend. Notably, all rent and utility series under consideration were non-stationary in their levels. Exhibits 4-1 and 4-2 show the summary of the DF-GLS and KPSS test results for first-differenced rent series and utility series, respectively.

Exhibit 4-1 | Test for Unit Root and Stationarity for Rent Series

Geographic

Code

Area

DF-GLS Test

KPSS Test

Test Statistic

p-value

Test Statistic

p-value

0000

National

– 2.7992

0.0000

0.2155

0.2396

0100

North

– 1.7622

0.0000

0.8162

0.0066

0200

Midwest

– 1.9350

0.0000

0.6275

0.0186

0300

South

– 2.8169

0.0000

0.1227

0.4849

0400

West

– 1.8456

0.0000

0.2108

0.2478

S100

North Class A

– 2.0872

0.0000

0.8935

0.0040

S200

Midwest Class A

– 1.7502

0.0000

0.7187

0.0113

S300

South Class A

– 2.5890

0.0000

0.1793

0.3122

S400

West Class A

– 1.7448

0.0000

0.2128

0.2443

S11A

Boston

– 1.7935

0.0000

0.7836

0.0081

S12A

New York

– 2.7245

0.0000

0.6746

0.0142

S12B

Philadelphia

– 1.6747

0.0000

0.5399

0.0313

S23A

Chicago

– 3.2610

0.0000

0.7802

0.0082

S23B

Detroit

– 2.6555

0.0000

0.2327

0.2124

15

Geographic

Code

Area

DF-GLS Test

KPSS Test

Test Statistic

p-value

Test Statistic

p-value

S35A

Washington, DC

– 2.0489

0.0000

1.1058

0.0014

S35B

Miami

– 2.1736

0.0000

0.2375

0.2055

S35C

Atlanta

– 2.4856

0.0000

0.3269

0.1135

S35E

Baltimore

– 3.5300

0.0000

0.3936

0.0747

S37A

Dallas

– 2.4950

0.0000

0.4457

0.0547

S49A

Los Angeles

– 1.2421

0.0000

0.3685

0.0870

S49B

San Francisco

– 2.1647

0.0000

0.3396

0.1046

S49D

Seattle

– 2.1044

0.0000

0.3621

0.0906

Notes: DF-GLS denotes “Dickey-Ful er Generalized Least Squares,” and KPSS denotes “Kwiatkowski-Phil ips-Schmidt-Shin.” The nul hypothesis for the DF-GLS test is that there is a unit root present in the series (series is non-stationary). The nul hypothesis for the KPSS test is that there is no unit root present in the series (series is stationary). A coefficient is statistical y significant if its p-value is less than or equal to 5 percent.

Exhibit 4-2 | Test for Unit Root and Stationarity for Utility Series

Geographic

Area

DF-GLS Test

KPSS Test

Code

Test Statistic

p-value

Test Statistic

p-value

0000

National

– 4.6698

0.0000

0.1202

0.4950

0100

North

– 4.1135

0.0000

0.2152

0.2402

0200

Midwest

– 5.3648

0.0000

0.0672

0.7689

0300

South

– 4.9885

0.0000

0.0749

0.7223

0400

West

– 3.5299

0.0000

0.0921

0.6257

S100

North Class A

– 4.0968

0.0000

0.2041

0.2601

S200

Midwest Class A

– 3.9723

0.0000

0.0502

0.8753

S300

South Class A

– 4.3826

0.0000

0.0922

0.6252

S400

West Class A

– 3.7105

0.0000

0.0742

0.7266

S11A

Boston

– 1.7261

0.0000

0.0457

0.9023

S12A

New York

– 3.5986

0.0000

0.1759

0.3204

S12B

Philadelphia

– 2.8424

0.0000

0.2244

0.2251

S23A

Chicago

– 5.5570

0.0000

0.0207

0.9961

S23B

Detroit

– 4.4371

0.0000

0.2466

0.1930

S35A

Washington, DC

– 2.3577

0.0000

0.2624

0.1731

S35B

Miami

– 4.8792

0.0000

0.0875

0.6504

S35C

Atlanta

– 2.7632

0.0000

0.0593

0.8189

S35E

Baltimore

– 2.3565

0.0000

0.0799

0.6929

S37A

Dallas

– 3.0921

0.0000

0.0937

0.6175

S49A

Los Angeles

– 4.4497

0.0000

0.0604

0.8116

S49B

San Francisco

– 4.7253

0.0000

0.0613

0.8060

S49D

Seattle

– 2.7826

0.0000

0.1050

0.5614

Notes: DF-GLS denotes “Dickey-Ful er Generalized Least Squares,” and KPSS denotes “Kwiatkowski-Phil ips-Schmidt-Shin.” The nul hypothesis for the DF-GLS test is that there is a unit root present in the series (series is non-stationary). The nul hypothesis for the KPSS test is that there is no unit root present in the series (series is stationary). A coefficient is statistical y significant if its p-value is less than or equal to 5 percent.

Once we achieved stationarity for the rent and utility series by first-order differencing, the next step was to determine the general form of the models to be estimated. For each geographic area, the study team 16

formulated initial model specifications by examining the autocorrelation and partial autocorrelation functions. For each metro area, we estimated several rent and utility models with varying ARIMA terms.

Guided by the Schwarz Bayesian information criterion (SBC), we settled on final ARIMA specifications.

Exhibits 4-3 and 4-4 present the structure for three different ARIMA rent and utility models, respectively. We estimated the different models by varying the type of ARIMA specifications and the set of exogenous variables included. The study team developed the first model, Pure Time Series (PTS), without exogenous variables; the second, National Input Model (NIM), with national exogenous variables (National Residential Fixed Investment and Civilian Employment Data for rent and spot price, in dollars per barrel of West Texas intermediate crude oil, price in dollars per short ton of bituminous coal; Henry Hub price of natural gas, in dollars per million BTUs; and CPI-U for utilities); and the third, Local Input Model (LIM), based on local exogenous variables (Local Permit Data and Employment Data for rent, and electricity for utilities).

Exhibit 4-3 | ARIMA Model for Rent by Geographic Area

Geographic Code Area

PTS

NIM

LIM

0000

National

1,1,0

1,1,0

NA

0100

North

1,1,2

5,1,0

2,1,1

0200

Midwest

0,1,1

3,1,0

3,1,0

0300

South

1,1,0

1,1,0

1,1,0

0400

West

0,1,2

3,1,0

3,1,0

S100

North Class A

0,1,1

3,1,0

0,1,1

S200

Midwest Class A

1,1,2

4,1,0

4,1,0

S300

South Class A

1,1,0

1,1,0

1,1,0

S400

West Class A

1,1,2

3,1,0

3,1,0

S11A

Boston

0,1,1

0,1,1

0,1,1

S12A

New York

0,1,1

0,1,1

1,1,1

S12B

Philadelphia

0,1,1

0,1,1

0,1,1

S23A

Chicago

0,1,1

0,1,1

0,1,1

S23B

Detroit

0,1,1

0,1,2

0,1,2

S35A

Washington, DC

1,1,0

1,1,0

1,1,0

S35B

Miami

1,1,0

1,1,0

1,1,0

S35C

Atlanta

1,1,0

0,1,1

0,1,1

S35E

Baltimore

0,1,1

0,1,1

0,1,1

S37A

Dallas

1,1,0

1,1,0

1,1,0

S49A

Los Angeles

1,1,0

1,1,1

3,1,0

S49B

San Francisco

1,1,0

1,1,0

1,1,0

S49D

Seattle

3,1,0

0,1,1

3,1,0

Note: ARIMA denotes “Autoregressive Integrated Moving Average,” LIM denotes “Local Input Model,” NIM denotes “National Input Model,” and PTS denotes “Pure Time Series” model.

In exhibit 4-3, the NIM for rent produced more consistent and generally smaller SBC results for the 22

areas than either the PTS or the LIM. Based on the SBC rule, we find that the NIM model dominates the other two competing models in most cases.

17

Exhibit 4-4 | ARIMA Model Utility by Geographic Area

Geographic Code Area

PTS

NIM

LIM

0000

National

0,1,1

0,1,1

NA

0100

North

2,1,1

0,1,1

NA

0200

Midwest

4,1,0

4,1,0

NA

0300

South

4,1,0

4,1,0

NA

0400

West

2,1,0

2,1,0

NA

S100

North Class A

1,1,0

0,1,1

NA

S200

Midwest Class A

4,1,0

3,1,0

NA

S300

South Class A

4,1,0

3,1,0

NA

S400

West Class A

2,1,2

2,1,0

NA

S11A

Boston

4,1,0

4,1,0

2,1,0

S12A

New York

4,1,0

4,1,0

2,1,1

S12B

Philadelphia

4,1,1

4,1,0

1,1,2

S23A

Chicago

0,1,1

0,1,1

3,1,0

S23B

Detroit

0,1,1

0,1,1

0,1,1

S35A

Washington, DC

4,1,0

4,1,0

4,1,0

S35B

Miami

0,1,1

0,1,1

0,1,1

S35C

Atlanta

4,1,0

4,1,0

0,1,1

S35E

Baltimore

4,1,0

4,1,0

4,1,0

S37A

Dallas

4,1,0

4,1,0

4,1,0

S49A

Los Angeles

0,1,1

0,1,1

0,1,1

S49B

San Francisco

0,1,2

0,1,2

0,1,2

S49D

Seattle

0,1,1

0,1,1

0,1,2

Note: ARIMA denotes “Autoregressive Integrated Moving Average,” LIM denotes “Local Input Model,” NIM denotes “National Input Model,” and PTS denotes “Pure Time Series” model.

In exhibit 4-4, the PTS for utility produced generally smaller SBC results for the geographic areas compared to the alternative NIM. Although the NIM for utility has comparable performance to the PTS

for utility in 16 cases, the overall performance of PTS is better than NIM. For cases in which the PTS and NIM ARIMA models differed, the PTS areas had slightly more parsimonious models compared to their NIM counterparts. As in the rent specification, in the minority of cases for utility in which the SBC for a PTS area was not superior to NIM, the results do not outweigh the benefits of the most parsimonious model possible (PTS) across areas.

4.2 Forecast Error Comparison

Using the model with lowest SBC in exhibits 4-3 and 4-4, the study team conducted a test of the model’s forecast performance by splitting each series (dataset for each metro area) into an out-of-sample period. We compared the difference between the forecast estimates and the actual values to determine the forecast accuracy, using the Root Mean Square Error (RMSE). For comparison, we duplicated HUD’s hypothetical model with the national inputs (for rent and utility models) and local inputs (for the rent model) and calculated the RMSE for the same validation period. The lower the value of the measure of 18

accuracy, the better the performance of the forecast model. The results are presented in exhibits 4-5

and 4-6.

Exhibit 4-5 | Average Root Mean Square Errors Across Validation Period by ARIMA

Specification and Geographic Area, Rent Model

Geographic

Our Model

HUD’s Hypothetical

Code

Area

Model

PTS

NIM

LIM

NIM

LIM

0000

National

1.629

1.486

NA

2.013

NA

0100

North

1.088

1.562

1.921

2.742

2.761

0200

Midwest

1.324

0.726

0.726

1.159

1.139

0300

South

2.409

2.085

2.116

1.828

1.858

0400

West

5.785

2.814

2.813

5.050

5.115

S100

North Class A

2.957

3.027

3.177

3.716

3.747

S200

Midwest Class A

0.832

0.607

0.598

0.751

0.734

S300

South Class A

4.672

4.488

4.490

4.431

4.428

S400

West Class A

2.918

2.092

2.101

4.928

4.995

S11A

Boston

1.796

1.259

2.962

1.120

1.143

S12A

New York

4.684

4.726

4.733

5.633

5.671

S12B

Philadelphia

3.194

3.450

3.451

3.818

3.776

S23A

Chicago

0.909

1.565

1.646

1.570

1.631

S23B

Detroit

1.756

1.465

1.091

1.382

1.236

S35A

Washington, DC

2.844

3.331

3.178

4.245

4.131

S35B

Miami

4.370

4.402

4.382

4.164

4.227

S35C

Atlanta

9.426

8.929

9.013

9.642

9.607

S35E

Baltimore

4.216

5.442

5.157

3.661

3.647

S37A

Dallas

9.716

9.605

9.640

10.076

10.033

S49A

Los Angeles

2.321

2.062

2.111

3.278

3.252

S49B

San Francisco

2.113

1.921

1.908

6.467

6.450

S49D

Seattle

6.710

11.901

7.204

11.568

11.871

Notes: ARIMA denotes “Autoregressive Integrated Moving Average,” LIM denotes “Local Input Model,” NA denotes “data not available,” NIM denotes “National Input Model,” PTS denotes “Pure Time Series” model, and RMSE denotes “Root Mean Square Error.” The best RMSE for each geographical region is highlighted.

On comparing the RMSEs across models, 2M’s NIM outperforms the PTS and LIM specifications for 10 of the 22 areas. The exogenous variables included in NIM explain the underlying dynamics of the response series better than those in the LIM. Of the remaining 12 areas, the PTS specification performed better than the LIMs in 9 areas. For these regions, we observe that the PTS specification captures the trend and seasonality embedded in the data, without additional input variables. In exhibit 4-5, with a few exceptions, two of the study team’s rent models, NIM and LIM, produced lower-average RMSEs across local and regional areas compared to their HUD rent model counterparts. Compared to HUD’s hypothetical model (NIM), the study team’s model (NIM) improved RMSEs in 15 areas and fared worse in seven others. These 15 areas had an aggregate reduction in RMSE of 17.185 compared to a total reduction of 2.888 in the seven areas where HUD’s hypothetical model performed better than the study model. For the LIMs, the total reduction in RMSEs between the study model (LIM) and HUD’s 19

hypothetical model (LIM) was 20.853 (in 15 areas) compared to a total reduction of 3.819 (in six areas) where HUD’s hypothetical model fared better than the study model.

Exhibit 4-6 | Average Root Mean Square Errors Across Validation Period by ARIMA

Specification and Geographic Area, Utility Model

Geographic

Our Model

HUD’s Hypothetical

Code

Area

Model

PTS

NIM

LIM*

NIM

LIM

0000

National

2.567

1.008

NA

1.639

NA

0100

North

5.534

7.871

NA

4.561

NA

0200

Midwest

4.175

3.831

NA

4.950

NA

0300

South

4.088

1.555

NA

3.811

NA

0400

West

3.007

6.648

NA

4.314

NA

S100

North Class A

4.477

9.139

NA

2.427

NA

S200

Midwest Class A

4.132

4.537

NA

4.712

NA

S300

South Class A

5.205

2.589

NA

3.356

NA

S400

West Class A

3.846

5.464

NA

5.888

NA

S11A

Boston

36.464

28.714

15.059

10.858

NA

S12A

New York

13.489

9.284

10.283

2.113

NA

S12B

Philadelphia

1.496

2.299

2.154

5.845

NA

S23A

Chicago

7.713

4.201

3.125

8.201

NA

S23B

Detroit

4.401

9.012

6.416

4.665

NA

S35A

Washington, DC

7.806

3.934

8.150

5.547

NA

S35B

Miami

5.096

4.024

2.589

5.276

NA

S35C

Atlanta

10.370

14.276

5.347

18.296

NA

S35E

Baltimore

11.820

13.329

14.926

17.338

NA

S37A

Dallas

3.562

7.992

3.092

4.118

NA

S49A

Los Angeles

2.706

6.054

3.525

3.958

NA

S49B

San Francisco

8.864

10.931

8.470

12.693

NA

S49D

Seattle

4.082

6.035

3.247

5.930

NA

* Data were only available for the 13 metro areas.

Notes: ARIMA denotes “Autoregressive Integrated Moving Average,” LIM denotes “Local Input Model,” NA denotes “data not available,” NIM denotes “National Input Model,” PTS denotes “Pure Time Series” model, and RMSE denotes “Root Mean Square Error.” The best RMSE for each geographical region is highlighted.

For the utility models, the PTS specification performs better than the competing NIM and LIM

specifications for 9 of the 22 areas. The exogenous variables included in NIM and LIM do not contribute to explaining the underlying patterns in the response series for utilities (exhibit 4-6). Of the remaining 13

areas, the LIM specification performed better than the NIMs in 7 areas. For these metro areas, we observe that the exogenous variables included in the LIM explain the underlying dynamics of the response series better than those in the NIM. Compared to HUD’s hypothetical model (NIM) for utility, the study team’s utility model (PTS) improved RMSEs in 14 areas and fared worse in 8 areas.

The RMSEs are comparable across the three model specifications since they are all measured with the same units. One way to think about choosing a model is to compare the models in percentage terms.

20

Suppose one model’s RMSE is 30 percent less than another—that may be a significant difference.

However, if the RMSE is about 5 percent different, the tradeoff would be model complexity to error measures. While there is no absolute criterion for a “good value” of RMSE, which would make one model preferable over another, the objective of the study team is to choose the model with the lowest RMSE.

4.3 Coefficient Significance of Selected Model

Based on the results in the previous section, the study team selected the model determined to provide the most accurate estimate for rent and utility in-sample among the three competing models. While we deemed NIM to be the best fit for the rent series, we selected PTS for estimating and forecasting utility.

In this section, we present results of the parameter estimate of the selected models, as well as their statistical significance. Exhibit 4-7 presents the parameter estimates of NIM for rent and PTS for utility, as well as the exogenous variables used to augment the specified model. For each parameter, the exhibit shows the estimated value, standard error, and t- value for the estimate. The exhibit also indicates the lag at which the parameter appears in the model. For the United States, there are four parameters in the rent model and two parameters in the utility model. The mean term is labeled MU.

The autoregressive (AR) parameter is labeled AR1,1; this is the coefficient of the lagged value of the change in rent. The moving average (MA) parameter is labeled MA1,1; this is the coefficient of the previous error term in the utility series. The p- value provides significance tests for the parameter estimates and indicates whether some terms in the model might be unnecessary. We show only the results for the nation in exhibit 4-7, and present the results for the remaining geographic areas in appendix C. However, we summarize the statistical significance of the parameters across all 22 areas in exhibit 4-8.

Exhibit 4-7 | Parameter Estimates for the United States

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0076

0.0000

0

AR1,1

rpr_sa

0.8488

0.0000

1

NUM1

RES_FIXED_INV

– 0.0074

0.0980

2

NUM2

Civilian_Employment

0.0780

0.0203

3

Utility Model

MU

fu_sa

0.0073

0.0038

0

MA1,1

fu_sa

– 0.4653

0.0000

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameter; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value is less than or equal to 5 percent.

21

Exhibit 4-8 | Percentage of Significant Coefficients

Parameter

Number of Areas Using Corresponding Parameters

Percent

Rent Model

MU

22

100

AR1,1

14

100

AR1,2

6

50

AR1,3

6

100

AR1,4

2

50

AR1,5

1

100

MA1,1

9

100

MA1,2

1

100

RES_FIXED_INV

22

6.5

Civilian_Employment

22

68.2

Utility Model

MU

22

63.6

AR1,1

15

40

AR1,2

14

64.3

AR1,3

11

63.6

AR1,4

11

90.9

MA1,1

10

80.0

MA1,2

2

100

Note: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model.

4.4 Forecasting Rent and Utilities

The study team used the model with the best forecasting measure, based on the model giving lowest RMSE, to forecast rents and utilities for all geographic areas. We forecast the two series for eight quarters ahead using the best models selected in the previous section. We estimated the models for each series using the observations up to 2016 (Q1) and then obtained the forecast values for the following eight quarters until 2018 (Q1). We then computed the forecast errors for the period 2016 (Q2) through 2018 (Q1) by comparing the forecast values with actual observations for the corresponding period. To il ustrate how closely the observed data follow the predicted data, we show in exhibits 4-9

and 4-10 the forecast values of rent and utilities respectively for the United States. The results for the remaining geographic areas are presented in appendix D.

22

Exhibit 4-9 | Forecasts of National Quarterly Rent of Primary Residence

The NIM rent forecast for 2018 (Q2) through 2020 (Q4) closely mimic the national trend for CPI-U Rent of Primary Residence series. The validation series shows slight departures late in the forecast period from both the Consumer Price Index (CPI) Rent series around 2016 (Q2) and from the forecast at 2018

(Q2).

23

Exhibit 4-10 | Forecasts of National Quarterly Housing Fuels and Utilities

In exhibit 4-10, the PTS validation series for utility had noticeable deviations between the CPI-U series in several time periods where it exhibits large fluctuations, indicating the presence of more volatile movements in the underlying CPI-U. Other than those time periods, however, the validation values were more consistent with the CPI-U trend. In later quarters, starting in 2016 (Q1), the validation series tracked quite closely with CPI because PTS provided a very precise forecast for CPI-U. For the forecast period 2018 (Q2) through 2020 (Q4), the selected model produces stable forecast values that are consistent with the overall trend direction of CPI-U.

4.5 Estimating a Local Trend Factor

Once the forecast model is estimated and validated, the rent and utility forecasts are used to construct a locally based trend factor for the geographic areas, which could be used in computing Fair Market Rents (FMRs). These trend factors help to account for current market conditions, especially markets where rent prices are escalating rapidly.

24

The trend factor is used to adjust rents from 2 years prior to the current fiscal year (FY) and is calculated using a gross rent index (GRI). The GRI is the sum of two weighted, independently forecasted components of the CPI-U (for example, Rent of Primary Residence and Fuels and Utilities).

Local trend factors for each region are calculated using the change in average quarterly GRI in the previous calendar year to the average quarterly forecasted GRI index of the respective FY. Specifically, for 2019, the local trend factor is a ratio of the average quarterly GRI for 2018 (Q4) through 2019 (Q3) to 2017 (Q1) through 2017 (Q4).

Exhibit 4-11 presents estimated trend factors for each region using the aforementioned forecasts for rent and utility. Column 3 presents the trend factor using the new model specifications and can be compared to the trend factors constructed by HUD in columns 4 and 5. The new forecasts provide a higher trend factor in 17 of the 22 areas, while five areas show no change.

Exhibit 4-11 | Estimated Trend Factors by ARIMA Specification and Geographic Area Geographic Code Area

The Study Team’s

HUD’s Hypothetical Model

Selected Model

NIM

LIM

0000

National

1.061

1.057

NA

0100

North

1.048

1.056

1.056

0200

Midwest

1.055

1.050

1.050

0300

South

1.064

1.059

1.059

0400

West

1.077

1.067

1.067

S100

North Class A

1.052

1.058

1.058

S200

Midwest Class A

1.058

1.051

1.051

S300

South Class A

1.069

1.062

1.062

S400

West Class A

1.082

1.072

1.071

S11A

Boston

1.084

NA*

NA*

S12A

New York

1.053

1.052

1.052

S12B

Philadelphia

1.041

1.041

1.041

S23A

Chicago

1.053

1.049

1.050

S23B

Detroit

1.053

1.052

1.053

S35A

Washington, DC

1.058

1.056

1.055

S35B

Miami

1.050

1.058

1.058

S35C

Atlanta

1.061

1.042

1.042

S35E

Baltimore

1.046

1.047

1.046

S37A

Dallas

1.071

1.069

1.069

S49A

Los Angeles

1.081

1.073

1.073

S49B

San Francisco

1.110

1.089

1.089

S49D

Seattle

1.088

1.084

1.082

Note: *Data not available for Boston in HUD’s hypothetical model. ARIMA denotes “Autoregressive Integrated Moving Average,” LIM denotes “Local Input Model,” NIM denotes “National Input Model.”

HUD’s NIM trend factor was slightly below (0.01) the study team’s NIM. The differences between the trend factors produced by the study team’s selected model and HUD’s hypothetical model ranged from -

0.01 to 0.021. A few area results for the study model were smaller than in the HUD model (North, North 25

Class A, Miami, and Baltimore). Generally, the forecast based on HUD’s hypothetical model tends to be smaller than the study team’s model in many cities and areas, particularly in the West. The study team’s method appears to underestimate the trend only in Miami and Baltimore, although the extent of underestimation looks mild.

In order to determine if the use of a local trend factor is warranted, as opposed to the current methodology of using a national estimate of the trend factor, we compared our local forecast estimates with our national forecast estimate. Specifically, we used the forecast period between 2018 (Q2) and 2019 (Q4) to examine whether the errors of estimates for each geographic area significantly differ from the errors of the estimate for the United States. 8 The results of this test revealed that all 13 local geographic areas have forecast estimates significantly different from the U.S. estimate, implying that a local trend factor could prove to be beneficial because it may address the varying patterns that exist in each geographical area. To understand the magnitude of this difference and to examine whether using local estimate could improve the accuracy of the trend factor, we compare the FMRs for each area using the local trend factors and the national trend factor in section 6.

8 A paired t-test with equal variance for rent and unequal variance for utilities was conducted to address the significant difference between the estimates of local and national estimates at the 95-percent confidence interval.

26

5 PHASE II RESULTS

The main concern with the analysis in the previous section is that some accuracy may be lost by limiting the number of metro areas. In this section, we look at ways to introduce additional geographic areas with data from Axiometrics and Zillow. 9 The Axiometrics data are from a monthly survey of rental properties starting in April 2008 and include values for asking rent, effective rent, occupancy rate, number of units, average size of the units, and quality grades of the property and local market areas.

The Axiometrics data facilitate the creation of summary measures at various levels of geography.

Zillow offers summary measures, such as their Zillow Rent Indices (ZRI), at various levels of geography.

Zillow measures are created by proprietary algorithms/methods and generally began in 2010 although some indices based on home values are available for longer periods of time. Thus, an important consideration in evaluating the usefulness of these data is the extent to which HUD finds these methods acceptable.

Our general approach was to first evaluate these sources of data by comparing rates of change to rates of change calculated from Bureau of Labor Statistics (BLS) and American Community Survey (ACS) data.

These comparisons provide “face-validity” assessments of the usability of the series. Next, we looked at a case of forecasting values for 277 Core-Based Statistical Areas (CBSAs) and compared them to actual values from ACS.

Exhibit 5-1 | Cases and Geographic Coverage of Axiometrics Data

Year

Properties in Survey

Number of States

Number of CBSAs

2008

22,681

47

322

2009

25,233

47

350

2010

27,244

48

344

2011

30,530

48

388

2012

33,403

50

416

2013

35,335

51

451

2014

37,358

51

446

2015

40,047

51

495

2016

46,334

51

544

2017

49,542

51

604

2018

51,279

51

610

Notes: CBSAs denotes “Core-Based Statistical Areas.” Unique properties = 51,279. The Number of States column includes the 50

states plus Washington, DC.

9 We have looked at the data from the Federal Housing Finance Agency, who produces house price indices (HPIs), starting in 1975, that are based on transactions on the same properties. Indices are produced monthly and quarterly for various geographic regions. This series is of interest to the extent that the relationship between single-family home values is reasonably stable and can be reliably estimated; however, Gallin (2008) is not very encouraging about finding a stable relationship. The main advantage of the HPI series is its lengths but the lack of comparable rent series (in terms of length) hinders the use of the HPIs. Thus, to date we have not identified a relationship between rents and HPI that could be used to forecast trend factors.

27

Exhibit 5-2 | Analysis Variables Created from the Axiometrics Data

Variable

Description

Notes

Geoid

Location of property

Time invariant; incudes lat and lon

Submarket

Submarket name

Time invariant; Axiometrics

defined name

Year Built

Year of construction

Time invariant

Rehab

Year of last rehabilitation

Time invariant

Units

Number of units

Time invariant

Area per Unit

Average square feet per unit

Time invariant

Level

Number of stories

Time invariant

Status

Property status code ("S" = stable, "LU"

= leased up, and so on)

Monthly

Prop

Classification of the property submarket

Submarket_Grade

by letter grade

Monthly; lots of missing values

Propmarket_Grade

Classification of the property market by

letter grade

Monthly; lots of missing values

Submarket_Grade

Classification of the submarket by letter

grade

Monthly

Grent

Asking rent

Monthly

Erent

Asking rent minus concessions

Monthly

Erent per sqft

erent per square foot

Monthly

Occ Rate

Percent of units occupied

Monthly

Con Value

Dollar value of concessions

Monthly

Con Percent

Concessions as a percent of grent

Monthly

5.1 Axiometrics Data

The Axiometrics data are based on a monthly survey of more than 50,000 properties. Some, but not all, properties have been surveyed since 2008 (exhibit 5-1). The main variables available in the Axiometrics data are shown in exhibit 5-2. The Axiometrics data include the asking rent and concessions. The

“effective rent” ( erent) is the asking rent minus the concessions and is the main variable analyzed by the study team. 10 The data also include the precise location of each property, making it possible to create summary measures (for example, mean or 40th percentile erent) for various geographic definitions.

Additionally, the data include some codes that classify the property (for example, “A,” “B”); indicate dates of rehabs; and convey overall property and market assessment (for example, “stable”). (Ful data details will be outlined in an appendix.)

The Axiometrics data offer numerous (almost too many) possibilities for creating time series that are based on summary measures of the individual properties by CBSA (or other geographies). As a result, the study team took an evolutionary approach by starting with the most obvious series. Specifically, we created a set of time series using the properties that responded to the survey in every period. The 10 Concessions would likely increase (decrease) with decreases (increases) in demand for apartment units. Thus, the effective rent should provide a more accurate measure of the current rent when compared to the asking rent.

28

advantage of this approach is that property specific factors that are time-invariant will be eliminated by differencing the data. With this set of responses, the study team then created quarterly and annual series for primary sampling unit (PSU) ( n = 22) and CBSAs ( n = 277).

5.2 Methods to Assess the Data

To assess the Axiometrics data’s usefulness for Fair Market Rent (FMR) trend adjustments, the study team compared the Axiometrics measure of effective rent (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒) to the following: 1. The measure of the seasonally unadjusted “rent of primary residence” (𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆) from the Consumer Price Index for All Urban Consumers (CPI-U) data file produced by BLS.

2. The estimates of median gross rent for all bedroom sizes from ACS (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆). 11

We detail our approach to both BLS and ACS data below.

5.2.1 COMPARISON TO BUREAU OF LABOR STATISTICS DATA

To compare Axiometrics data to BLS data, the study team compared the quarterly and yearly percentage change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each of the 22 PSUs available in both datasets. 12 The steps to create a quarterly and yearly series of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 for each PSU included the following: 1. Removing projects in the Axiometrics data that did not have complete data (in other words, any that had missing values of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 for any month in any year). 13

2. Using the county Federal Information Processing Standards (FIPS) code to merge the Axiometrics data with the crosswalk provided by HUD that links counties to PSUs and creating a PSU variable that indicates the PSU each project is in (any project not in a PSU was removed from the data).

3. Creating a quarterly estimate of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 at the PSU level by averaging the monthly project-level estimates for each quarter (42 in total from 2008 to 2018) in each PSU (23 in total).

4. Creating a yearly estimate of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 at the PSU level by averaging the monthly project-level estimates for each year (11 in total) in each PSU (23 in total).

The steps to create a quarterly and yearly series of 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each PSU included the following: 1. Creating a quarterly estimate of 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 at the PSU level by averaging the monthly PSU-level estimates for each quarter (the same 42 quarters as the Axiometrics data).

2. Creating a yearly estimate of 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 at the PSU level by averaging the monthly PSU-level estimates for each year (the same 11 as the Axiometrics data).

The study team calculated the percentage change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒t and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each PSU( 𝑖𝑖) in each quarter for year (𝑒𝑒) from the preceding quarter or year (𝑒𝑒−1) with the following formula: 11 This is variable B25064—median gross rent.

12 We also did comparisons for each of the four Census regions and can provide these results as an appendix at HUD’s request.

13 We also did comparisons that included projects with incomplete data, as wel as comparisons that only used data from 2010

or later for projects that had complete data. The study team found that the results were best for comparisons that use complete data for al years of the Axiometrics data (2008 to 2018).

29

𝑋𝑋

𝑟𝑟ℎ𝑎𝑎𝑒𝑒𝑎𝑎𝑒𝑒

𝑖𝑖,𝑡𝑡 − 𝑋𝑋𝑖𝑖,𝑡𝑡−1

𝑖𝑖,𝑡𝑡 =

𝑋𝑋

× 100.

𝑖𝑖,𝑡𝑡−1

Thus, the study team created PSU-level percentage changes for 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each quarter and year available in the data. Importantly, nine PSUs only have data from BLS for the fourth quarter of 2017

and later; therefore, the percentage change estimates for these PSUs represent fewer timepoints. We indicate which PSUs have fewer data points in all results below and suggest focusing on the results f Finally, the study team developed several metrics for each PSU to determine the comparability of the percentage change estimates of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 including the following:

 𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵/𝑷𝑷𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵: the number and percentage of timepoints (quarters or years) in which the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 percentage change is higher than the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. One would expect roughly half the timepoints to have a higher 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 percentage change if 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 are comparable measures. If not, then there is evidence that 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 is consistently either higher or lower than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 and that the measures are not very comparable

 𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑵𝑵𝑷𝑷𝑷𝑷: the number and percentage of timepoints in which the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 percentage change is within 10-percentage points (either above or below) of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change.

Since rents tend to be relatively stable, one would expect virtually all timepoints to show the

𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage changes are within 10-percentage points of each other if the measures are comparabl

 𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑵𝑵𝑷𝑷: the number and percentage of timepoints in which the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 percentage change is within 5 percentage points (either above or below) of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. This measure is more stringent than 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑆𝑆𝑃𝑃𝑃𝑃10, but one would still expect many of the timepoints to show the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage changes are within 5-percentage points of each other if the me

 𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑵𝑵𝑷𝑷: the number and percentage of timepoints in which the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 percentage change is within 1 percentage point (either above or below) of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. This measure is very stringent and if timepoints show the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage changes are within 1 percentage point of each other, there is evidence that the measures are comparable.

 𝑷𝑷𝑵𝑵𝑵𝑵𝑷𝑷𝑷𝑷: an indication (either “yes” or “no”) that shows whether an estimated rent value that accounts for the trend in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 is within 10 percent of an estimated rent value that accounts for the trend in 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆. The study team calculated estimated rent values for each measure for each PSU by adding the percentage change for each measure from the earliest timepoint in the data to the latest timepoint to $100. These calculated values are referred to as “simulated” FMRs. For example, if the percentage change was 3.4 percent for 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 5.4 percent for 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆, then the si

𝑆𝑆𝑃𝑃10 indicates whether the

simulated FMR from 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 is within 10 percent of the value from 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆.

We present the results for each of the metrics for each PSU below in exhibit 5-3.

30

5.2.2 COMPARISON TO AMERICAN COMMUNITY SURVEY DATA

To compare Axiometrics data to ACS data, the study team compared the yearly percentage change in

𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 for each of the 249 CBSAs available in both datasets. The steps to create a yearly series of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 for each CBSA included the following:

1. Removing projects in the Axiometrics data that did not have complete data (that is, any that had missing values of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 for any month in any year).

2. Using the county FIPS code to merge the Axiometrics data with the crosswalk provided by HUD

that links counties to CBSAs and creating a CBSA variable that indicates the CBSA each project is located in (any project not in a CBSA was removed from the data).

3. Creating a yearly estimate of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 at the CBSA level by averaging the monthly project-level estimates for each year (10 in total, 2008 to 2017) in each CBSA (249 in total).

The study team downloaded the yearly estimates of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 for each CBSA from American Factfinder using table B25064. The study team then created the yearly percentage change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆

and the metrics comparing the percentage change estimates using the same process that we describe earlier in section 5.2.1.

5.2.3 RESULTS

Exhibit 5-3 shows the results of each of the metrics comparing quarterly estimates of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 from Axiometrics and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 from BLS. The results indicate a high level of comparability between the two datasets. For many PSUs, 𝑃𝑃𝑁𝑁𝑁𝑁𝑆𝑆𝑃𝑃 shows that 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 is larger than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 about half (close to 50 percent) of the time. For some PSUs, 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 is larger more often (75 percent of the time) or less often (25 percent of the time), but in all cases these are PSUs that have limited data in BLS. The New York-Newark-Jersey City, NY-NJ-PA PSU and Baltimore-Columbia-Towson, MD PSU are slightly more likely to have lower values of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒t than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 was only larger than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 38 and 36 percent of the time, respectively).

Another finding that indicates the comparability of the measures is that the difference in the quarterly percentage change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and the percentage change in 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 was within 5 percent for all timepoints in all PSUs with the exception of the Seattle-Tacoma-Bellevue, WA PSU, where the difference was still within 5 percent for 93 percent of the time. In addition, the difference was within 1 percent for several quarters across all PSUs, which indicates the measures are highly comparable.

Finally, in all PSUs except Atlanta-Sandy Springs-Roswell, GA, the artificial FMR value (based on an initial value of 100) using the trend from 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 was within 10 percent of the estimated value using 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆.

Again, this shows the two measures are highly comparable.

The comparison of the Axiometrics data to the ACS measure of median gross rent (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆) also produced results that indicate the Axiometrics measure 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 is a viable option for estimating trends in FMR. Exhibit 5-4 provides a summary of the results comparing yearly series of the two measures for the 254 CBSAs represented in both datasets. The accompanying Excel workbook provides the ful results for each CBSA. Exhibit 5-4 shows that 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 was higher than 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 about half the time (between 4 to 6

of the 10 years in the analysis) in 196 (79 percent) of the CBSAs. Moreover, the difference in the percentage change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and the change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 was within 10 percent for all years in 156 (63

31

percent) of the CBSAs. Finally, in 182 (73 percent) of the CBSAs, the simulated FMR estimate using the trend in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 was within 10 percent of estimate using the trend in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆.

Exhibit 5-3 | Number (%) of Quarters in Each Primary Sampling Unit for the Metrics Comparing the Axiometrics Rent Measure to the Bureau of Labor Statistics Measure PSU NAME

NOVER (POVER) PWITHIN10 PWITHIN5 PWITHIN1 PER10

Boston-Cambridge-Newton, MA-NH

20 (48%)

41 (100%)

41 (100%)

21 (51%)

Yes

New York-Newark-Jersey City, NY-NJ-PA

16 (38%)

41 (100%)

41 (100%)

22 (54%)

Yes

Philadelphia-Camden-Wilmington, PA-NJ-DE-MD

20 (48%)

41 (100%)

41 (100%)

22 (54%)

Yes

Chicago-Napervil e-Elgin, IL-IN-WI

19 (45%)

41 (100%)

41 (100%)

19 (46%)

Yes

Detroit-Warren-Dearborn, MI

20 (48%)

41 (100%)

41 (100%)

21 (51%)

Yes

Minneapolis-St. Paul-Bloomington, MN-WI*

2 (50%)

3 (100%)

3 (100%)

2 (67%)

Yes

St. Louis, MO-IL*

1 (25%)

3 (100%)

3 (100%)

3 (100%)

Yes

Washington-Arlington-Alexandria, DC-VA-MD-WV

20 (48%)

41 (100%)

41 (100%)

26 (63%)

Yes

Miami-Fort Lauderdale-West Palm Beach, FL

23 (55%)

41 (100%)

41 (100%)

30 (73%)

Yes

Atlanta-Sandy Springs-Roswel , GA

22 (52%)

41 (100%)

41 (100%)

18 (44%)

No

Tampa-St. Petersburg-Clearwater, FL*

3 (75%)

3 (100%)

3 (100%)

2 (67%)

Yes

Baltimore-Columbia-Towson, MD

15 (36%)

41 (100%)

41 (100%)

24 (59%)

Yes

Dal as-Fort Worth-Arlington, TX

20 (48%)

41 (100%)

41 (100%)

23 (56%)

Yes

Houston-The Woodlands-Sugar Land, TX

19 (45%)

41 (100%)

41 (100%)

26 (63%)

Yes

Phoenix-Mesa-Scottsdale, AZ*

2 (50%)

3 (100%)

3 (100%)

3 (100%)

Yes

Denver-Aurora-Lakewood, CO*

2 (50%)

3 (100%)

3 (100%)

1 (33%)

Yes

Los Angeles-Long Beach-Anaheim, CA

23 (55%)

41 (100%)

41 (100%)

31 (76%)

Yes

San Francisco-Oakland-Hayward, CA

21 (50%)

41 (100%)

41 (100%)

14 (34%)

Yes

Riverside-San Bernardino-Ontario, CA*

3 (75%)

3 (100%)

3 (100%)

3 (100%)

Yes

Seattle-Tacoma-Bellevue WA

21 (50%)

41 (100%)

38 (93%)

12 (29%)

Yes

San Diego-Carlsbad, CA*

3 (75%)

3 (100%)

3 (100%)

3 (100%)

Yes

Urban Hawai *

3 (75%)

3 (100%)

3 (100%)

1 (33%)

Yes

Urban Alaska*

2 (50%)

3 (100%)

3 (100%)

1 (33%)

Yes

*These areas were not included in the Phase I analysis data due to lack of published monthly data used to construct quarterly averages of Consumer Price Index data prior to 2018. At the time of this report, published monthly data needed to construct quarterly averages for these primary sampling units (PSUs) are only available for four quarters (the fourth quarter of 2017 to the third quarter of 2018).

Notes: PSU denotes “primary sampling unit.” NOVER ( POVER) is the number (percentage) of quarters that percentage change in erent exceeds the percentage change in SEHA. PWITHIN10(5)(1) is the number of quarters that the absolute value of the difference between the percentage change in erent and the percentage change in SEHA is within 10(5)(1) percent. PER10 is

“Yes” if the simulated Fair Market Rent (FMR) calculated from erent is within 10-percentage points of the simulated FMR

calculated from SEHA.

32

Exhibit 5-4 | Summary of Metrics Comparing the Axiometrics Rent Measure to the American Community Survey Measure for Core-Based Statistical Areas

Summary Metrics

Number (%) of CBSAs

POVER is between 40% and 60%

201 (79%)

WITHIN10 is satisfied in all timepoints

160 (63%)

PER10 is satisfied in all timepoints

186 (73%)

Met all three metrics above

109 (43%)

Notes: CBSAs denotes “Core-Based Statistical Areas.” POVER is the percentage of years that percentage change in erent exceeds the percentage change in rentACS. PWITHIN10 is the number of years that the absolute value of the difference between the percentage change in erent and the percentage change in rentACS is within 10 percent. PER10 is “Yes” if the simulated Fair Market Rent (FMR) calculated from erent is within 10 percentage points of the simulated FMR calculated from rentACS.

5.1.3 ASSESSMENT OF AXIOMETRICS DATA

The results of our analysis indicate that the rent estimate from the Axiometrics data is often comparable to the rent estimate from both BLS and ACS in metro areas. Therefore, we recommend that HUD further examine the potential of the Axiometrics data in improving predictions of FMR in metro areas. 14

5.3 Zil ow Data

The study team examined downloads available from Zillow (https://www.zil ow.com/research/data/)

and decided the best series for capturing changes in rents was ZRI Time Series: Multifamily, SFR, Condo/Co-op (downloaded as: Metro_Zri_AllHomesPlusMultifamily.csv). This is a dollar value monthly index of rents for 661 metropolitan areas (Zillow also provides a crosswalk to CBSAs, which we used to make the data comparable). The only variable in the data is the rent value ( ZRI). The Zillow methodology seeks to control for other factors, such as living area and market conditions.

5.3.1 METHODS TO ASSESS THE DATA

To assess the Zillow data’s usefulness for FMR trend adjustments, the study team compared the metro region Zillow rent index (𝑍𝑍𝑃𝑃𝑃𝑃) to the following:

1. The measure of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 from the CPI-U data file produced by BLS.

2. The estimate of median gross rent from the ACS (𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆).

We detail our approach to both the BLS and ACS data below.

14 Several CBSAs have fewer than 30 properties (or projects) with rent values in every period—a limitation of the Axiometrics data (see column “NOBS” in tab “Yearly AXIO to ACS” in the accompanying Excel workbook). The study team found that by limiting the analysis to the 69 CBSAs with more than 30 Axiometrics projects, the metrics improved substantially. Compared with exhibit 5-4, the percentages are 72, 88, 86, and 62, respectively. Therefore, in section 6, the analysis is confined to these 69 CBSAs.

33

5.3.2 COMPARISON TO BUREAU OF LABOR STATISTICS DATA

To compare Zillow data to BLS data, the study team compared the quarterly and yearly percentage change in 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each of the 22 PSUs available in both datasets. The steps to create a quarterly and yearly series of 𝑍𝑍𝑃𝑃𝑃𝑃 for each PSU included the following:

1. Using a crosswalk provided by Zillow to match the metro region IDs in the Zillow data to CBSAs and creating a variable that indicates the CBSA of each Zillow metro region.

2. Using a crosswalk provided by HUD to match the CBSAs in the Zillow data to PSUs and creating a variable that indicates the PSU of each metro region (any metro region not in a PSU was removed from the data).

3. Creating a quarterly estimate of 𝑍𝑍𝑃𝑃𝑃𝑃 at the PSU level by averaging the monthly estimates for each quarter (33 in total from 2010 to 2018) in each PSU (23 in total).

4. Creating a yearly estimate of 𝑍𝑍𝑃𝑃𝑃𝑃 at the PSU level by averaging the monthly project-level estimates for each year (nine in total) in each PSU (23 in total).

The steps to create a quarterly and yearly series of 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each PSU included the following: 1. Creating a quarterly estimate of 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 at the PSU level by averaging the monthly PSU-level estimates for each quarter (the same 33 quarters as the Zillow data).

2. Creating a yearly estimate of 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 at the PSU level by averaging the monthly PSU-level estimates for each year (the same nine as the Zillow data).

The study team calculated the percentage change in 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each PSU 𝑖𝑖 in each quarter or year 𝑒𝑒 from the preceding quarter or year (𝑒𝑒−1) with the following formula:

𝑋𝑋

𝑟𝑟ℎ𝑎𝑎𝑒𝑒𝑎𝑎𝑒𝑒

𝑖𝑖,𝑡𝑡 − 𝑋𝑋𝑖𝑖,𝑡𝑡−1

𝑖𝑖,𝑡𝑡 =

𝑋𝑋

× 100.

𝑖𝑖,𝑡𝑡−1

Thus, the study team created PSU-level percentage changes for 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 for each quarter and year available in the data. As above, nine PSUs only have data from BLS for the fourth quarter of 2017

and later; therefore, the percentage change estimates for these PSUs represent fewer timepoints. In addition, Zillow data does not have any data for timepoints earlier than November 2011 for the New York-Newark-Jersey City, NY-NJ-PA PSU. We indicate which PSUs have fewer data points in all results below.

Finally, the study team used the same metrics as above for each PSU to determine the comparability of the percentage change estimates of 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆. For reference, these metrics are the following—

 𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵/𝑷𝑷𝑵𝑵𝑵𝑵𝑵𝑵𝑵𝑵: the number and percentage of timepoints (quarters or years) for which the 𝑍𝑍𝑃𝑃𝑃𝑃 percentage change is higher than the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. One would expect roughly half the timepoints to have a higher 𝑍𝑍𝑃𝑃𝑃𝑃 percentage change if 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 are comparable measures. If not, then there is evidence that 𝑍𝑍𝑃𝑃𝑃𝑃 is consistently either higher or lower than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 and that the measures are not very comparable.

 𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑵𝑵𝑷𝑷𝑷𝑷: the number and percentage of timepoints for which the 𝑍𝑍𝑃𝑃𝑃𝑃 percentage change is within 10-percentage points (either above or below) of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. Since rents tend to be relatively stable, one would expect virtually all timepoints to show the 𝑍𝑍𝑃𝑃𝑃𝑃 and 34

𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage changes are within 10-percentage points of each other if the measures are comparable.

 𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑵𝑵𝑷𝑷: the number and percentage of timepoints for which the 𝑍𝑍𝑃𝑃𝑃𝑃 percentage change is within 5 percentage points (either above or below) of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. This measure is more stringent than 𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑃𝑆𝑆𝑃𝑃𝑃𝑃10, but one would still expect many of the timepoints to show the 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage changes are within 5 percentage points of each other if the measures are comparable.

 𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑷𝑵𝑵𝑷𝑷: the number and percentage of timepoints for which the 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 percentage change is within 1 percentage point (either above or below) of the 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage change. This measure is very stringent and if timepoints show the 𝑍𝑍𝑃𝑃𝑃𝑃 and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 percentage changes are within 1 percentage point of each other, there is evidence that the measures are comparable.

 𝑷𝑷𝑵𝑵𝑵𝑵𝑷𝑷𝑷𝑷: an indication (either “yes” or “no”) that shows whether an estimated rent value that accounts for the trend in 𝑍𝑍𝑃𝑃𝑃𝑃 is within 10 percent of an estimated rent value that accounts for the trend in 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆. The study team calculated estimated rent values for each measure for each PSU by adding the percentage change for each measure from the earliest timepoint in the data to the latest timepoint to $100. These calculated values are referred to as “simulated” FMRs. For example, if the percentage change was 3.4 percent for 𝑍𝑍𝑃𝑃𝑃𝑃 and 5.4 percent for SEHA, then the simulated FMRs would be $103.40 and $105.40, respectively. 𝑃𝑃𝑆𝑆𝑃𝑃10 indicates whether the simulated FMR from 𝑍𝑍𝑃𝑃𝑃𝑃 is within 10 percent of the value from 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆.

We present the results for each of the metrics for each PSU below.

5.3.3 COMPARISON TO AMERICAN COMMUNITY SURVEY DATA

To compare Zillow data to ACS data, the study team compared the yearly percentage change in 𝑍𝑍𝑃𝑃𝑃𝑃 and

𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 for each of the 422 CBSAs available in both datasets. The steps to create a yearly series of 𝑍𝑍𝑃𝑃𝑃𝑃

for each CBSA included the following:

1. Using a crosswalk provided by Zillow to match the metro region IDs in the Zillow data to CBSAs and creating a variable that indicates the CBSA of each Zillow metro region.

2. Creating a yearly estimate of 𝑍𝑍𝑃𝑃𝑃𝑃 at the CBSA level by averaging the monthly project-level estimates for each year (eight in total, 2010 to 2017) in each CBSA (420 in total).

The study team downloaded the yearly estimates of 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 for each CBSA from American Factfinder using table B25064. The study team then created the yearly percentage change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 and 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆

and the metrics comparing the percentage change estimates using the same process that we describe above in 5.2.2. Importantly, ACS data did not contain data for all years for two CBSAs, and Zillow data did not contain data for all years for 77 CBSAs. We provide a list of these CBSAs and the years for which both ACS and Zillow have data for these CBSAs in the accompanying Excel spreadsheet.

5.3.4 RESULTS

Exhibit 5-5 shows the results of each of the metrics comparing quarterly estimates of 𝑍𝑍𝑃𝑃𝑃𝑃 from Zillow and 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 from BLS. Unlike the results for the rent measure from the Axiometrics data, the evidence for the comparability of 𝑍𝑍𝑃𝑃𝑃𝑃 to 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 is more mixed. The difference in the percentage change in 𝑍𝑍𝑃𝑃𝑃𝑃 and the percentage change in 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 was within 5 percent for all quarters, and it was within 1 percent for 35

many of the quarters, which indicates a high level of comparability between the two measures. On the other hand, the 𝑃𝑃𝑁𝑁𝑁𝑁𝑆𝑆𝑃𝑃 metric shows that 𝑍𝑍𝑃𝑃𝑃𝑃 was often lower than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆; in many of the PSUs,

𝑍𝑍𝑃𝑃𝑃𝑃 was higher than 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆 less than 40 percent of the time. Moreover, in 6 of the 23 PSUs, the simulated FMR using the trend in 𝑍𝑍𝑃𝑃𝑃𝑃 was not within 10 percent of the simulated FMR using the trend in 𝑆𝑆𝑆𝑆𝑆𝑆𝑆𝑆.

Exhibit 5-6 shows a summary of the results of the metrics comparing yearly estimates of 𝑍𝑍𝑃𝑃𝑃𝑃 to the estimate of median gross rent, 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆, from the ACS data. The ful results are in the Excel spreadsheet. These results also show that the Zillow measure is less comparable than the Axiometrics measure. The Zillow rent estimate ZRI was higher than the ACS measure rentACS about half of the time in 65 percent of CBSAs (a majority), but this percentage is smaller than what the study team found when comparing the Axiometrics data (76 percent). Moreover, the difference in the percentage change in 𝑍𝑍𝑃𝑃𝑃𝑃

and the change in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 was within 10 percent for all years in only 50 percent of CBSAs (compared with 64 percent in the comparison of using 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 from the Axiometrics data).

Exhibit 5-6 also shows that simulated FMR estimate using the rent in 𝑍𝑍𝑃𝑃𝑃𝑃 was within 10 percent of the estimate using the trend in 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑆𝑆𝑟𝑟𝑆𝑆 in only a little more than half of the CBSAs (58 percent). Finally, the percent of CBSAs that met all three of the summary metrics was only 25 percent.

36

Exhibit 5-5 | Number (%) of Quarters in Each Primary Sampling Unit for the Metrics Comparing the Zillow Rent Measure to the Bureau of Labor Statistics Measure

PSU NAME

NOVER ( POVER) PWITHIN10 PWITHIN5 PWITHIN1 PER10

Boston-Cambridge-Newton, MA-NH

17 (52%)

32 (100%)

32 (100%)

16 (50%)

Yes

New York-Newark-Jersey City, NY-NJ-PA**

16 (55%)

28 (100%)

28 (100%)

15 (54%)

Yes

Philadelphia-Camden-Wilmington, PA-NJ-DE-MD

12 (36%)

32 (100%)

32 (100%)

23 (72%)

Yes

Chicago-Napervil e-Elgin, IL-IN-WI

12 (36%)

32 (100%)

32 (100%)

19 (59%)

No

Detroit-Warren-Dearborn, MI

12 (36%)

32 (100%)

32 (100%)

12 (38%)

No

Minneapolis-St. Paul-Bloomington, MN-WI*

1 (20%)

4 (100%)

4 (100%)

4 (100%)

Yes

St. Louis, MO-IL*

0 (0%)

4 (100%)

4 (100%)

2 (50%)

Yes

Washington-Arlington-Alexandria, DC-VA-MD-WV 12 (36%)

32 (100%)

32 (100%)

22 (69%)

Yes

Miami-Fort Lauderdale-West Palm Beach, FL

12 (36%)

32 (100%)

32 (100%)

24 (75%)

Yes

Atlanta-Sandy Springs-Roswel , GA

11 (33%)

32 (100%)

32 (100%)

22 (69%)

No

Tampa-St. Petersburg-Clearwater, FL*

1 (20%)

4 (100%)

4 (100%)

4 (100%)

Yes

Baltimore-Columbia-Towson, MD

9 (27%)

32 (100%)

32 (100%)

17 (53%)

No

Dal as-Fort Worth-Arlington, TX

10 (30%)

32 (100%)

32 (100%)

19 (59%)

No

Houston-The Woodlands-Sugar Land, TX

7 (21%)

32 (100%)

32 (100%)

21 (66%)

No

Phoenix-Mesa-Scottsdale, AZ*

1 (20%)

4 (100%)

4 (100%)

2 (50%)

Yes

Denver-Aurora-Lakewood, CO*

1 (20%)

4 (100%)

4 (100%)

3 (75%)

Yes

Los Angeles-Long Beach-Anaheim, CA

11 (33%)

32 (100%)

32 (100%)

26 (81%)

Yes

San Francisco-Oakland-Hayward, CA

18 (55%)

32 (100%)

32 (100%)

18 (56%)

Yes

Riverside-San Bernardino-Ontario, CA*

3 (60%)

4 (100%)

4 (100%)

4 (100%)

Yes

Seattle-Tacoma-Bellevue, WA

12 (36%)

32 (100%)

32 (100%)

14 (44%)

Yes

San Diego-Carlsbad, CA*

1 (20%)

4 (100%)

4 (100%)

2 (50%)

Yes

Urban Hawai *

1 (20%)

4 (100%)

4 (100%)

2 (50%)

Yes

Urban Alaska*

2 (40%)

4 (100%)

4 (100%)

3 (75%)

Yes

*These areas were not included in the Phase I analysis data due to lack of published monthly data used to construct quarterly averages of Consumer Price Index data prior to 2018. At the time of this report, published monthly data needed to construct quarterly averages for these primary sampling units (PSUs) are only available for four quarters (the fourth quarter of 2017 to the third quarter of 2018).

**PSU data are only available for 29 quarters (the fourth quarter of 2011 and later).

Al other PSUs have data available for 33 quarters.

Notes: PSU denotes “primary sampling unit.” NOVER ( POVER) is the number (percentage) of quarters that percentage change in ZRI exceeds the percentage change in SEHA. PWITHIN 10(5)(1) is the number of quarters that the absolute value of the difference between the percentage change in ZRI and the percentage change in SEHA is within 10(5)(1) percent. PER 10 is “Yes”

if the simulated Fair Market Rent (FMR) calculated from ZRI is within 10 percentage points of the simulated FMR calculated from SEHA.

37

Exhibit 5-6 | Summary of Metrics Comparing the Zillow Rent Measure to the American Community Survey Measure for Core-Based Statistical Areas

Summary Metrics

Number (%) of CBSAs

POVER is between 40% and 60%

274 (65%)

WITHIN10 is satisfied in all timepoints

210 (50%)

PER10 is satisfied in all timepoints

243 (58%)

Met all three metrics above

106 (25%)

Notes: CBSAs denotes “Core-Based Statistical Areas.” POVER is the percentage of years that percentage change in ZRI exceeds the percentage change in rentACS. PWITHIN 10 is the number of years that the absolute value of the difference between the percentage change in ZRI and the percentage change in rentACS is within 10 percent. PER 10 is “Yes” if the simulated Fair Market Rent (FMR) calculated from ZRI is within 10 percentage points of the simulated FMR calculated from rentACS.

5.3.5 ASSESSMENT OF ZILLOW DATA

The results of our analysis indicate that rent estimate from the Zillow data is less comparable than the Axiometrics estimate to the rent estimates from both BLS and ACS in metro areas. Therefore, we recommend that HUD exclude Zillow data from efforts to improve predictions of the FMR in metro areas.

5.4 Axiometrics Forecasts

As a test of the potential usability of the Axiometrics data, the study team used the quarterly CBSA-level series (through Q4 of 2016) to create an Autoregressive Integrated Moving Average (ARIMA) model for each of the 254 CBSAs and then forecast the four quarters of 2017 in each CBSA. 15 Then, we compared the forecasts to the actual values in ACS in the following way:

1. Compute the annual Axiometrics erent in 2016 as the average of the four quarters in 2016 and compute the annual erent in 2017 as the average of the four forecasted values.

2. Compute the annual percentage change in erent in 2017 for each CBSA, using the values from step 1.

3. Compute the annual percentage change in rentACS in 2017 for each CBSA, using the ACS values for 2016 and 2017.

4. Compare the percentage changes expected from the Axiometrics data to the percentage changes in ACS.

We emphasize that we did not necessarily expect to find similar levels of rents. However, we do expect (assuming the Axiometrics data are forecastable) to find similar percentage changes in the rents.

15 Forecasts were made from models estimated with PROC X12 in SAS, using an automatic model selection with seasonality and a second difference to stationarize the growth rate of the series. The statement used to invoke the automatic model selection is based on “time series regression with ARIMA noise, missing values, and outliers” method by Gomez and Maravall (1997a,

1998). This algorithm automatically selects the order of the autoregressive and moving average parameters. The nonseasonal autoregressive (AR) and moving average (MA) orders are given by p, q, while the seasonal AR and MA orders are given by p and q respectively. The model selected using this method is of the form (p,d,q) and (P,D,Q). The number of differences and seasonal differences are given by d and D. The estimated parameters are presented in the accompanying Excel workbook.

38

Moreover, to apply the Axiometrics data to the trend factor simply requires accurate percentage changes.

The comparisons are presented in the accompanying Excel spreadsheet. Exhibit 5-7 gives a summary of the findings. The first metric measures whether the percentage differences in the trends are within 10

percent, while the second measures the differences in the trend that are less than 0.1.

The findings from this test suggest that the Axiometrics forecasts are generally within acceptable limits. 16

Exhibit 5-7 | Comparisons of Axiometrics Data Forecasts to American Community Survey Values

Notes: ACS denotes “American Community Survey” and CBSAs denotes “Core-Based Statistical Areas.” erent is the quarterly average rent calculated from the Axiometrics data. rentACS is the annual median gross rent from the 1-Year ACS. A “hat” above a variable denotes a forecasted value, and a subscript denotes the year of the respective value.

16 Axiometrics produces quarterly forecasts of rents at the project level. The study team compared CBSA-level summaries of Axiometrics’ forecasts to our forecasts of trend factors. Essential y, the forecasts were the same, although Axiometrics did not produce forecasts for as many areas.

39

6 ACCURACY OF ALTERNATIVE MODELS

The study team’s trend factors for fiscal year 2019 (FY19), based on data from the Bureau of Labor Statistics (BLS), illustrate the potential variability in trend factors among the primary sampling units (PSUs) examined in section 4 (exhibit 6-1). Compared with HUD’s constant value (1.0572), the differences range from a high of +5.3 to -1.6 percentage points. However, the empirical implications of this heterogeneity are small. The largest differences in Fair Market Rents (FMRs) are in San Francisco ($138) and Boston ($48). Similarly, compared to HUD’s value, the trend factors for FY19 based on Axiometrics data is different by a maximum of 3.4 percentage points and a minimum of 0.2 percentage points. The largest difference in FMRs is in New York ($40) and the smallest in Philadelphia ($2).

Exhibit 6-1 | Comparison of Estimated Fair Market Rents for Select Primary Sampling Units, FY19

ACS

Recent

Annual

PSU

Area

2016

Mover

2016 to

5-Year

Adj.

2017

Trend Factor for FY19

FY19 2-Bedroom FMR

Est.

Factor

CPI Adj.

2M

2M

HUD

BLS

Axiometrics

HUD*

BLS

Axiometrics

(Phase I)

(Phase 2)

(Phase 1)

(Phase 2)

11A

Boston

1,376

1.260

1.0378

1.084

1.044

1,902

1,950

1,878

12A

New York

1,324

1.273

1.0272

1.053

1.034

1,831

1,823

1,790

12B

Philadelphia

1,031

1.081

1.0179

1.041

1.055

1,200

1,181

1,197

23A

Chicago

980

1.136

1.0297

1.053

1.042

1,212

1,207

1,194

23B

Detroit

832

1.065

1.0319

1.053

1.076

967

963

984

35A

Washington,

DC

1,423

1.078

1.0266

1.058

1.019

1,665

1,666

1,605

35B

Miami

1,114

1.176

1.0495 1.0572 1.050

1.063

1,454

1,444

1,462

35C

Atlanta

901

1.103

1.0527

1.061

1.091

1,106

1,110

1,141

35E

Baltimore

1,131

1.093

1.0269

1.046

1.034

1,342

1,328

1,313

37A

Dallas

934

1.143

1.0634

1.071

1.055

1,201

1,216

1,198

49A

Los Angeles

1,301

1.242

1.0483

1.081

1.062

1,791

1,831

1,799

49B

San

Francisco

1,769

1.415

1.0546

1.110

1.065

2,792

2,930

2,811

49D Seattle

NA

NA

NA

1.088

1.078

1,899

NA

NA

Note: ACS denotes “American Community Survey,” BLS denotes “Bureau of Labor Statistics,” CPI denotes “Consumer Price Index,” FY denotes “fiscal year,” FMR denotes “Fair Market Rent,” NA denotes “data not available,” and PSU denotes “primary sampling units.” The FY19 2-Bedroom FMR for Boston and San Francisco do not reflect rent surveys.

*Source: Fiscal Year 2019 Fair Market Rents Documentation System, HUD

https://www.huduser.gov/portal/datasets/fmr.html#2019_query.

To assess the accuracy of trend factors estimated in Phase I with those estimated in Phase II, the study team compared forecasted trend factors retrospectively by assuming a standard for accuracy was the median gross rent from the American Community Survey (ACS) in 2017 (exhibit 6-2). Usually, the study team’s Phase I models outperform the Phase II models, as the Phase II models are better than Phase I in only three PSUs (New York, Baltimore, and Seattle; exhibit 6-2). In New York, for example, the difference 40

in the Phase I estimate (compared to the ACS value) was 0.011, while the Phase II difference is just 0.006

and was similar for Baltimore and Seattle.

Exhibit 6-2 | Comparison of Change in Rent from 2016 to 2017

PSU

Name

Axiometrics

(Phase I )

ACS

BLS

(Phase I)

Rent

Rent and

Utilities

11A

Boston

1.022

1.035

1.031

1.036

12A

New York

1.031

1.025

1.035

1.036

12B

Philadelphia

1.027

1.008

1.020

1.019

23A

Chicago

1.018

1.027

1.029

1.031

23B

Detroit

1.054

1.014

1.023

1.024

35A

Washington, DC

1.015

1.038

1.026

1.028

35B

Miami

1.022

1.054

1.062

1.062

35C

Atlanta

1.053

1.044

1.039

1.035

35E

Baltimore

1.008

1.012

1.024

1.021

37A

Dallas

1.051

1.061

1.052

1.047

49A

Los Angeles

1.069

1.052

1.052

1.050

49B

San Francisco

0.987

1.054

1.062

1.062

49D

Seattle

1.091

1.089

1.048

1.047

Notes: ACS denotes “American Community Survey,” BLS denotes “Bureau of Labor Statistics,” and PSU denotes “primary sampling units.” The 1-year trend for ACS denotes a ratio of 2017 actuals to 2016 actuals. The 1-year trend is calculated as a ratio of 2017 forecast values to 2016 actuals values for Axiometrics and BLS data.

Using the Axiometrics quarterly series through the third quarter of 2018, the study team used the X12

software to build forecasting models for each of the 69 Core-Based Statistical Areas (CBSAs) with more than 30 projects with a ful set of monthly rent data. The models were then used to forecast four quarters, starting with the fourth quarter of 2018 through the third quarter of 2019. The ratio of the average of the forecasted values to the average of the actual Axiometrics values from the four quarters of 2017 provides an alternative set of trend factors with additional spatial resolution when compared to the PSU estimates (exhibit 6-3). 17 These values provide some data on the consequences of using a single trend factor for all U.S Census Bureau (Census) regions. For example, the aggregate difference in the South Region is 11 percent, while the West is 50 percent. 18 On average, the Axiometrics series suggests a trend factor of approximately 3-percentage points higher than 1.0572 in the West.

17 Results for other areas are available at HUD’s request. The difference values are from HUD’s total trend factors: positive (negative) differences would be greater (less) if an adjustment reflecting utilities were made.

18 These figures are unweighted; hence, Coos Bay completely offsets Albuquerque in the aggregate.

41

Exhibit 6-3 | FY19 Core-Based Statistical Area Estimated Trend Factors by Region Region

CBSA

Trend Factor

Difference From

1.0572

Boston-Cambridge-Newton, MA-NH

1.044

– 0.013

Hartford-West Hartford-East Hartford, CT

1.036

– 0.022

Northeast New York-Newark-Jersey City, NY-NJ-PA

1.034

– 0.023

Philadelphia-Camden-Wilmington, PA-NJ-DE-MD

1.055

– 0.002

Pittsburgh, PA

1.039

– 0.018

Providence-Warwick, RI-MA

1.082

0.025

Boston-Cambridge-Newton, MA-NH

1.044

– 0.013

Atlanta-Sandy Springs-Roswell, GA

1.091

0.034

Austin-Round Rock, TX

1.076

0.019

Baltimore-Columbia-Towson, MD

1.034

– 0.023

Birmingham-Hoover, AL

1.050

– 0.007

Charleston-North Charleston, SC

1.050

– 0.007

Charlotte-Concord-Gastonia, NC-SC

1.039

– 0.018

Chattanooga, TN-GA

1.055

– 0.002

Columbia, SC

1.041

– 0.016

Dallas-Fort Worth-Arlington, TX

1.055

– 0.002

Durham-Chapel Hill, NC

1.042

– 0.015

El Paso, TX

1.075

0.017

Greensboro-High Point, NC

1.082

0.025

Greenville-Anderson-Mauldin, SC

1.084

0.027

Gulfport-Biloxi-Pascagoula, MS

1.027

– 0.030

Houston-The Woodlands-Sugar Land, TX

1.053

– 0.004

Jackson, MS

1.029

– 0.028

South

Jacksonville, FL

1.122

0.065

Lexington-Fayette, KY

1.030

– 0.027

Little Rock-North Little Rock-Conway, AR

1.020

– 0.037

Louisville/Jefferson County, KY-IN

1.044

– 0.013

Memphis, TN-MS-AR

1.087

0.030

Miami-Fort Lauderdale-West Palm Beach, FL

1.063

0.006

Nashville-Davidson-Murfreesboro-Franklin, TN

1.042

– 0.015

Oklahoma City, OK

1.031

– 0.027

Orlando-Kissimmee-Sanford, FL

1.129

0.071

Palm Bay-Melbourne-Titusville, FL

1.188

0.131

Raleigh, NC

1.059

0.001

Richmond, VA

1.079

0.022

San Antonio-New Braunfels, TX

1.034

– 0.023

Tampa-St. Petersburg-Clearwater, FL

1.103

0.046

Tulsa, OK

1.012

– 0.046

Virginia Beach-Norfolk-Newport News, VA-NC

1.046

– 0.011

Washington-Arlington-Alexandria, DC-VA-MD-WV

1.019

– 0.038

Wilmington, NC

1.043

– 0.014

Winston-Salem, NC

1.095

0.038

Albuquerque, NM

1.055

– 0.002

West

Colorado Springs, CO

1.059

0.002

Coos Bay, OR Micro Area

1.055

– 0.002

42

Region

CBSA

Trend Factor

Difference From

1.0572

Denver-Aurora-Lakewood, CO

1.064

0.007

Las Vegas-Henderson-Paradise, NV

1.174

0.117

Los Angeles-Long Beach-Anaheim, CA

1.062

0.005

Oxnard-Thousand Oaks-Ventura, CA

1.098

0.041

Phoenix-Mesa-Scottsdale, AZ

1.147

0.090

Portland-Vancouver-Hillsboro, OR-WA

1.055

– 0.002

Riverside-San Bernardino-Ontario, CA

1.085

0.028

Sacramento-Roseville-Arden-Arcade, CA

1.061

0.004

Salt Lake City, UT

1.101

0.044

San Diego-Carlsbad, CA

1.105

0.048

San Francisco-Oakland-Hayward, CA

1.065

0.008

San Jose-Sunnyvale-Santa Clara, CA

1.109

0.052

Seattle-Tacoma-Bellevue, WA

1.078

0.021

Tucson, AZ

1.118

0.061

Chicago-Naperville-Elgin, IL-IN-WI

1.042

– 0.016

Cincinnati, OH-KY-IN

1.061

0.004

Cleveland-Elyria, OH

1.033

– 0.025

Columbus, OH

1.085

0.027

Dayton, OH

1.048

– 0.009

Midwest Detroit-Warren-Dearborn, MI

1.076

0.019

Indianapolis-Carmel-Anderson, IN

1.084

0.026

Kansas City, MO-KS

1.064

0.007

Minneapolis-St. Paul-Bloomington, MN-WI

1.084

0.027

Omaha-Council Bluffs, NE-IA

1.046

– 0.011

St. Louis, MO-IL

1.023

– 0.034

Note: CBSA denotes “Core-Based Statistical Area.”

Locations highlighted in exhibit 6-3 indicate areas with trend factors more than 5 percentage points greater than HUD’s current trend factor. In these areas, HUD’s estimated FMRs may be too low.

However, the potential gains from using localized trend factors require consideration of a couple of implementation issues. First, HUD’s current methodology inflates ACS values to more current values using elements of the CPI-U—a likely reliable source of information. For most CBSAs, this source of information is not available, meaning that the commercial data will need to be used instead. Also, although the findings in section 5 indicate that this approach is reasonable, it is a significant departure from current policy. Second, HUD will need an expanded pull from the ACS to get baseline values for gross rents and the shares of rent and utilities for the additional areas. Third, numerous geographic areas will not be covered by commercial data. These areas will need FMRs based on other methods—

most likely some version of the current method. Final y, the additional complexity of localization may require updating forecasting models on a regular basis, especially in light of the relatively short length of the commercial series, adding costs to implementing localization.

43

7 CONCLUSION

This research project investigated various avenues to which HUD could add geographic resolution to the trend factors and, perhaps, reduce concern about the accuracy of Fair Market Rents (FMRs). Specifically, the project has investigated alternative approaches to augmenting local market conditions into calculations of FMRs, starting with extensions of HUD’s existing methodology and fol owed by consideration of models that use alternative sources of data and empirical frameworks. The study team approached this project in two phases.

In Phase I, the study team relied on the Bureau of Labor Statistics’ (BLS) data on 22 distinct geographic areas to estimate different trend factors. This approach is different from HUD’s current methodology, which uses a national estimate of the trend factor for al geographic areas. We used the Autoregressive Integrated Moving Average (ARIMA) modeling technique to estimate the forecasts of local rent and utilities, based on which trend factors for each area were calculated. The study team began the modeling process by identifying the best model form for forecasting rent and utility for each geographic area, as wel as determining an appropriate lag length for the exogenous variables. In selecting the best model form, we have used the information criterion method—specifically Schwarz Bayesian information criterion (SBC)—to select from a group of established autoregressive (AR) and moving average (MA) orders. We again used SBC to determine the appropriate lag length of each exogenous variable, in order to illustrate the delayed rent and utility responses to changes in these input variables. We have computed SBC using various AR and MA model orders, with an automated function used to determine the range of orders. Our approach produced three competing ARIMA models—a univariate time series model without exogenous variables (Pure Time Series [PTS]), a multivariate model with national exogenous variables (National Input Model [NIM]), and a multivariate model with local exogenous variables (Local Input Model [LIM])—based on which the identified orders of the AR and MA terms were estimated and forecasted rent and utility.

To evaluate each model’s forecast accuracy, we have compared the forecast errors of the models’

estimates using the Root Mean Square Error (RMSE). The results of this comparison showed that the NIM performs better for the rent models, while the PTS performs better than the competing NIM and LIM specifications for the utility models. The study team has selected model specifications based on lower RMSEs in more than 50 percent of the geographical areas. While there is no absolute criterion for a “good value” of RMSE that would make one model preferable over another, it is possible to compare the RMSEs in percentage terms for each geographic area. Suppose the RMSE of one model is over 10

percent lower than another: one could choose the model with the lower RMSE. However, there exists a tradeoff between model complexity and error measure. The study team needs to consider the complexity of a model before choosing one solely based on the error measures. Based on our preferred model, we have used a paired t-test to examine whether the errors of our local estimates significantly differ from the errors of the national estimate. The results of this test showed that the local geographic areas have forecast estimates significantly different from the national estimate. Following the forecasts for each geographical area, the study team was unable to develop forecasts for non-metropolitan regions in Phase I due to the limited availability of data.

44

In Phase II, the study team first assessed the usability of monthly rent series from Axiometrics and Zillow. The Axiometrics data were found to be more comparable to data from BLS and the American Community Survey (ACS) than with the Zil ow data. The study team did not find reliable and consistent data on exogenous variables that may drive rental markets at the Core-Based Statistical Area (CBSA) level. Similarly, there does not appear to be a consistent source of forecasts already produced by states and/or local metropolitan governmental agencies. Subsequently, the Axiometrics data were used to forecast quarterly and annual trends in rents in 254 CBSAs, using PTS models and the X12 software package, to compare the forecasts to actual data from ACS and BLS. Areas with more than 30 sampled properties in the Axiometrics data were used to demonstrate the differences between estimated trend factors from the Axiometrics models to the existing trend factor for fiscal year 2019 (FY19). The Phase II analysis suggests that localization of trend factors is feasible and, in certain areas, may lead to more accurate trend factors.

The findings in this report provide some evidence that localizing the estimates of trend factors will improve the accuracy of FMRs. Carefully constructed time series models for 22 geographic areas generally reduce forecast error, when compared with a single value for all areas. Models of additional geographic areas, constructed with automated algorithms, also seem to offer more accurate trend factors in most areas; however, a perfect standard for determining accuracy is not available. Therefore, one general conclusion is that if accuracy is a primary concern, migrating away from a single trend factor to trend factors for several areas would be sound policy. The overall empirical impact of moving in this direction is relatively small, however. In other words, the empirical payoff to localization may not be obvious when first encountered.

On the other hand, there are clearly geographic areas where localization has relatively large empirical consequences. This suggests a hybrid approach to migrating toward localization by using the results from both phases of this research. This would involve using forecasts from the well-performing models based on BLS data and those based on Axiometrics data to estimate trend factors for numerous CBSAs.

Areas without well-performing forecasting models will be assigned trend factors from either national or regional models. This process is not unlike what HUD now does in applying inflation factors.

These observations should be tempered by some enumeration of the limitations of the study. In Phase I of the research, the study team found a dearth of local data that may help forecast the relatively volatile utility series. Residential price for electricity was the only input variable used to forecast the utility series, but since the data were available only at the state level, it failed to capture the varying patterns in the response series. Additionally, exogenous variables related to the rent series, such as labor force participation, weekly wages, and employment in the manufacturing sector, were not available for the North, South, Midwest, and West regions. In Phase II, the analysis was confined primarily to data from Axiometrics once the study team eliminated Zillow sources from further consideration. Additionally, the study team did not find a stable relationship between rents and the Federal Housing Finance Agency House Price Index, eliminating another source of data. The Axiometrics data, while appealing, are based on a limited number of apartment sites and the monthly series is relatively short. In Phase II accuracy assessments, the study team used readily available estimates of median gross rent from ACS. A better standard for comparison would be estimates of 40th percentile rents paid by recent movers.

45

APPENDIX A. LAG SELECTION RESULTS FOR EXOGENOUS

VARIABLES

Exhibit A-1 | Lag Selection for Civilian Employment Using Minimum Schwarz Bayesian Information Criterion (Panel A)

PSU

Area

Lags

1

2

3

4

5

6

0000

National

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

0100

North

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

0200

Midwest

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

0300

South

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

0400

West

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

S100

North Class A

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

S11A

Boston

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S12A

New York

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S12B

Philadelphia

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S200

Midwest Class A

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

S23A

Chicago

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S23B

Detroit

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S300

South Class A

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

S35A

Washington, DC

– 11.516

– 11.563

– 11.567

– 11.617

– 11.584

– 11.574

S35B

Miami

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S35C

Atlanta

– 11.511

– 11.515

– 11.555

– 11.595

– 11.596

– 11.592

S35E

Baltimore

– 11.516

– 11.563

– 11.567

– 11.617

– 11.584

– 11.574

S37A

Dallas

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S400

West Class A

– 11.267

– 11.245

– 11.286

– 11.264

– 11.226

– 11.181

S49A

Los Angeles

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S49B

San Francisco

– 11.313

– 11.290

– 11.332

– 11.312

– 11.275

– 11.231

S49D

Seattle

– 11.511

– 11.515

– 11.555

– 11.595

– 11.596

– 11.592

Exhibit A-1 reports the SBC for the optimal lag length of the independent variable (civilian employment) used in estimating the national input rent model.

Exhibit A-2 | Lag Selection for Civilian Employment Using Minimum Schwarz Bayesian Information Criterion (Panel B)

PSU

Area

Lags

7

8

9

10

11

12

0000

National

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

0100

North

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

0200

Midwest

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

0300

South

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

0400

West

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

S100

North Class A

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

S11A

Boston

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S12A

New York

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S12B

Philadelphia

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

A1

PSU

Area

Lags

7

8

9

10

11

12

S200

Midwest Class A

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

S23A

Chicago

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S23B

Detroit

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S300

South Class A

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

S35A

Washington, DC

– 11.527

– 11.540

– 11.608

– 11.562

– 11.530

– 11.495

S35B

Miami

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S35C

Atlanta

– 11.553

– 11.567

– 11.559

– 11.512

– 11.510

– 11.532

S35E

Baltimore

– 11.527

– 11.540

– 11.608

– 11.562

– 11.530

– 11.495

S37A

Dallas

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S400

West Class A

– 11.137

– 11.096

– 11.053

– 11.011

– 11.008

– 10.964

S49A

Los Angeles

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S49B

San Francisco

– 11.187

– 11.147

– 11.104

– 11.061

– 11.065

– 11.021

S49D

Seattle

– 11.553

– 11.567

– 11.559

– 11.512

– 11.510

– 11.532

Exhibit A-1 reports the SBC for the optimal lag length of the independent variable (civilian employment) used in estimating the national input rent model.

Exhibit A-3 | Lag Selection for Residential Fixed Investment Using Minimum Schwarz Bayesian Information Criterion (Panel A)

PSU

Area

Lags

1

2

3

4

5

6

0000

National

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

0100

North

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

0200

Midwest

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

0300

South

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

0400

West

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

S100

North Class A

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

S11A

Boston

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S12A

New York

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S12B

Philadelphia

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S200

Midwest Class A

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

S23A

Chicago

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S23B

Detroit

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S300

South Class A

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

S35A

Washington, DC

– 7.115

– 7.176

– 7.146

– 7.100

– 7.099

– 7.070

S35B

Miami

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S35C

Atlanta

– 7.156

– 7.213

– 7.185

– 7.138

– 7.136

– 7.115

S35E

Baltimore

– 7.115

– 7.176

– 7.146

– 7.100

– 7.099

– 7.070

S37A

Dallas

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S400

West Class A

– 7.216

– 7.269

– 7.242

– 7.197

– 7.205

– 7.182

S49A

Los Angeles

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S49B

San Francisco

– 7.226

– 7.284

– 7.258

– 7.214

– 7.221

– 7.197

S49D

Seattle

– 7.156

– 7.213

– 7.185

– 7.138

– 7.136

– 7.115

A2

Exhibit A-4 | Lag Selection for Residential Fixed Investment Using Minimum Schwarz Bayesian Information Criterion (Panel B)

PSU

Area

Lags

7

8

9

10

11

12

0000

National

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

0100

North

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

0200

Midwest

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

0300

South

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

0400

West

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

S100

North Class A

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

S11A

Boston

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S12A

New York

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S12B

Philadelphia

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S200

Midwest Class A

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

S23A

Chicago

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S23B

Detroit

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S300

South Class A

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

S35A

Washington, DC

– 7.028

– 7.043

– 7.000

– 6.961

– 6.939

– 6.958

S35B

Miami

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S35C

Atlanta

– 7.071

– 7.079

– 7.040

– 6.996

– 6.971

– 6.986

S35E

Baltimore

– 7.028

– 7.043

– 7.000

– 6.961

– 6.939

– 6.958

S37A

Dallas

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S400

West Class A

– 7.141

– 7.150

– 7.107

– 7.066

– 7.043

– 7.041

S49A

Los Angeles

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S49B

San Francisco

– 7.155

– 7.170

– 7.128

– 7.087

– 7.059

– 7.062

S49D

Seattle

– 7.071

– 7.079

– 7.040

– 6.996

– 6.971

– 6.986

Exhibit A-2 reports the SBC for the optimal lag length of the independent variable (residential fixed investment) used in estimating the national input rent model.

A3

APPENDIX B. RESIDUAL CORRELATION DIAGNOSTIC RESULTS

Exhibit B-1 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Northeast—All Classes B1

Exhibit B-2 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Midwest—All Classes Exhibit B-3 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, South—All Classes B2

Exhibit B-4 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, West—All Classes Exhibit B-5 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Northeast—Class A B3

Exhibit B-6 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Boston Exhibit B-7 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, New York B4

Exhibit B-8 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Philadelphia Exhibit B-9 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Midwest—Class A B5

Exhibit B-10 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Chicago Exhibit B-11 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Detroit B6

Exhibit B-12 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, South—Class A Exhibit B-13 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Washington, DC

B7

Exhibit B-14 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Miami Exhibit B-15 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Atlanta B8

Exhibit B-16 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Baltimore Exhibit B-17 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Dallas B9

Exhibit B-18 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, West—Class A Exhibit B-19 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Los Angeles B10

Exhibit B-20 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, San Francisco Exhibit B-21 | Residual Correlation Diagnostic for First-Differenced Rent and Utility Series, Seattle B11

APPENDIX C. PARAMETER ESTIMATE RESULTS FOR THE

NATIONAL INPUT RENT MODEL

Exhibit C-1 | Parameter Estimates for Northeast—All Classes

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0084

0.0000

0

AR1,1

rpr_sa

0.5017

0.0000

1

AR1,2

rpr_sa

– 0.1695

0.1615

2

AR1,3

rpr_sa

0.3278

0.0017

3

NUM1

RES_FIXED_INV

0.0054

0.5860

2

NUM2

Civilian_Employment

– 0.0356

0.6581

3

Utility Model

MU

fu_sa

0.0064

0.0598

0

AR1,1

fu_sa

0.0411

0.6983

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-2 | Parameter Estimates for Northeast—Midwest—All Classes

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0063

0.0000

0

AR1,1

rpr_sa

0.4956

0.0000

1

AR1,2

rpr_sa

– 0.2278

0.0604

2

AR1,3

rpr_sa

0.2572

0.0320

3

AR1,4

rpr_sa

0.2409

0.0286

4

NUM1

RES_FIXED_INV

– 0.0024

0.7442

2

NUM2

Civilian_Employment

0.1409

0.0155

3

Utility Model

MU

fu_sa

0.0072

0.0381

0

AR1,1

fu_sa

0.0455

0.6596

1

AR1,2

fu_sa

– 0.0923

0.3299

2

AR1,3

fu_sa

– 0.3800

0.0001

3

AR1,4

fu_sa

0.2883

0.0054

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C1

Exhibit C-3 | Parameter Estimates for South—All Classes

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0074

0.0000

0

AR1,1

rpr_sa

0.7648

0.0000

1

NUM1

RES_FIXED_INV

0.0075

0.3769

2

NUM2

Civilian_Employment

0.1770

0.0056

3

Utility Model

MU

fu_sa

0.0070

0.0844

0

AR1,1

fu_sa

– 0.0149

0.8608

1

AR1,2

fu_sa

– 0.1931

0.0231

2

AR1,3

fu_sa

– 0.0956

0.2586

3

AR1,4

fu_sa

0.5759

0.0000

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-4 | Parameter Estimates for West—All Classes

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0091

0.0000

0

AR1,1

rpr_sa

0.8904

0.0000

1

AR1,2

rpr_sa

– 0.4612

0.0005

2

AR1,3

rpr_sa

0.4114

0.0000

3

NUM1

RES_FIXED_INV

– 0.0081

0.3186

2

NUM2

Civilian_Employment

0.1777

0.0115

3

Utility Model

MU

fu_sa

0.0087

0.0001

0

MA1,1

fu_sa

– 0.5383

0.0000

1

MA1,2

fu_sa

– 0.9230

0.0000

2

AR1,1

fu_sa

– 0.2403

0.0001

1

AR1,2

fu_sa

– 0.8861

0.0000

2

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C2

Exhibit C-5 | Parameter Estimates for Northeast—Class A

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0076

0.0000

0

AR1,1

rpr_sa

0.3826

0.0004

1

AR1,2

rpr_sa

– 0.2163

0.0669

2

AR1,3

rpr_sa

0.3262

0.0034

3

AR1,4

rpr_sa

– 0.0285

0.8058

4

AR1,5

rpr_sa

0.3349

0.0031

5

NUM1

RES_FIXED_INV

0.0000

0.9956

2

NUM2

Civilian_Employment

0.0692

0.2895

3

Utility Model

MU

fu_sa

0.0071

0.0496

0

MA1,1

fu_sa

– 0.9943

0.0013

1

AR1,1

fu_sa

– 0.7226

0.0000

1

AR1,2

fu_sa

0.2773

0.0000

2

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-6 | Parameter Estimates for Midwest—Class A

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0059

0.0000

0

AR1,1

rpr_sa

0.7060

0.0000

1

AR1,2

rpr_sa

– 0.4679

0.0001

2

AR1,3

rpr_sa

0.4772

0.0000

3

NUM1

RES_FIXED_INV

0.0009

0.8556

2

NUM2

Civilian_Employment

0.1212

0.0080

3

Utility Model

MU

fu_sa

0.0070

0.0404

0

AR1,1

fu_sa

0.0521

0.6032

1

AR1,2

fu_sa

– 0.0814

0.3780

2

AR1,3

fu_sa

– 0.3649

0.0001

3

AR1,4

fu_sa

0.3615

0.0003

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C3

Exhibit C-7 | Parameter Estimates for South—Class A

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0071

0.0000

0

AR1,1

rpr_sa

0.6676

0.0000

1

NUM1

RES_FIXED_INV

– 0.0148

0.0647

2

NUM2

Civilian_Employment

0.2178

0.0004

3

Utility Model

MU

fu_sa

0.0069

0.0486

0

AR1,1

fu_sa

0.0263

0.7667

1

AR1,2

fu_sa

– 0.1684

0.0550

2

AR1,3

fu_sa

– 0.1260

0.1522

3

AR1,4

fu_sa

0.5450

0.0000

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-8 | Parameter Estimates for West—Class A

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0084

0.0000

0

AR1,1

rpr_sa

0.8465

0.0000

1

AR1,2

rpr_sa

– 0.6543

0.0000

2

AR1,3

rpr_sa

0.6459

0.0000

3

NUM1

RES_FIXED_INV

– 0.0139

0.0680

2

NUM2

Civilian_Employment

0.1553

0.0218

3

Utility Model

MU

fu_sa

0.0086

0.0000

0

AR1,1

fu_sa

0.2041

0.0451

1

AR1,2

fu_sa

– 0.3111

0.0023

2

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C4

Exhibit C-9 | Parameter Estimates for Boston

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0080

0.0000

0

MA1,1

rpr_sa

– 0.5871

0.0000

1

NUM1

RES_FIXED_INV

0.0261

0.1490

2

NUM2

Civilian_Employment

0.0651

0.6806

3

Utility Model

MU

fu_sa

0.0099

0.1013

0

AR1,1

fu_sa

0.4380

0.0000

1

AR1,2

fu_sa

– 0.6193

0.0000

2

AR1,3

fu_sa

0.2576

0.0232

3

AR1,4

fu_sa

0.1051

0.3267

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-10 | Parameter Estimates for New York

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0090

0.0000

0

MA1,1

rpr_sa

– 0.5757

0.0000

1

NUM1

RES_FIXED_INV

0.0057

0.5972

2

NUM2

Civilian_Employment

– 0.0556

0.5555

3

Utility Model

MU

fu_sa

0.0062

0.1506

0

AR1,1

fu_sa

– 0.0243

0.8115

1

AR1,2

fu_sa

0.0448

0.6547

2

AR1,3

fu_sa

– 0.1686

0.0928

3

AR1,4

fu_sa

0.3094

0.0024

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C5

Exhibit C-11 | Parameter Estimates for Philadelphia

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0062

0.0000

0

MA1,1

rpr_sa

– 0.3314

0.0016

1

NUM1

RES_FIXED_INV

– 0.0065

0.6796

2

NUM2

Civilian_Employment

0.0827

0.5323

3

Utility Model

MU

fu_sa

0.0058

0.0825

0

MA1,1

fu_sa

– 0.3947

0.0131

1

AR1,1

fu_sa

– 0.1734

0.2139

1

AR1,2

fu_sa

– 0.1236

0.1620

2

AR1,3

fu_sa

– 0.2189

0.0192

3

AR1,4

fu_sa

0.5189

0.0000

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-12 | Parameter Estimates for Chicago

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0065

0.0000

0

MA1,1

rpr_sa

– 0.3554

0.0005

1

NUM1

RES_FIXED_INV

0.0124

0.3297

2

NUM2

Civilian_Employment

0.2430

0.0249

3

Utility Model

MU

fu_sa

0.0062

0.4067

0

MA1,1

fu_sa

– 0.2577

0.0116

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C6

Exhibit C-13 | Parameter Estimates for Detroit

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0048

0.0000

0

MA1,1

rpr_sa

– 0.3014

0.0040

1

MA1,2

rpr_sa

0.3293

0.0017

2

NUM1

RES_FIXED_INV

0.0390

0.0045

2

NUM2

Civilian_Employment

0.3002

0.0192

3

Utility Model

MU

fu_sa

0.0084

0.0066

0

MA1,1

fu_sa

– 0.0992

0.3467

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-14 | Parameter Estimates for Washington, DC

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0087

0.0000

0

AR1,1

rpr_sa

0.4967

0.0000

1

NUM1

RES_FIXED_INV

– 0.0206

0.2064

2

NUM2

Civilian_Employment

0.2896

0.0263

3

Utility Model

MU

fu_sa

0.0071

0.0366

0

AR1,1

fu_sa

– 0.2097

0.0115

1

AR1,2

fu_sa

– 0.2106

0.0079

2

AR1,3

fu_sa

– 0.2314

0.0040

3

AR1,4

fu_sa

0.6900

0.0000

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C7

Exhibit C-15 | Parameter Estimates for Baltimore

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0081

0.0000

0

MA1,1

rpr_sa

– 0.6654

0.0000

1

NUM1

RES_FIXED_INV

– 0.0117

0.5978

2

NUM2

Civilian_Employment

0.5928

0.0021

3

Utility Model

MU

fu_sa

0.0073

0.3891

0

AR1,1

fu_sa

– 0.0641

0.3948

1

AR1,2

fu_sa

– 0.1163

0.1046

2

AR1,3

fu_sa

– 0.1653

0.0229

3

AR1,4

fu_sa

0.7301

0.0000

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-16 | Parameter Estimates for Miami

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0082

0.0000

0

AR1,1

rpr_sa

0.8011

0.0000

1

NUM1

RES_FIXED_INV

0.0211

0.1407

2

NUM2

Civilian_Employment

– 0.0015

0.9891

3

Utility Model

MU

fu_sa

0.0050

0.1202

0

MA1,1

fu_sa

– 0.2410

0.0186

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C8

Exhibit C-17 | Parameter Estimates for Atlanta

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0036

0.0138

0

MA1,1

rpr_sa

– 0.6989

0.0000

1

NUM1

RES_FIXED_INV

0.0628

0.0318

2

NUM2

Civilian_Employment

0.5303

0.0153

3

Utility Model

MU

fu_sa

0.0091

0.0164

0

AR1,1

fu_sa

– 0.1811

0.0519

1

AR1,2

fu_sa

– 0.3307

0.0004

2

AR1,3

fu_sa

– 0.2162

0.0211

3

AR1,4

fu_sa

0.5591

0.0000

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-18 | Parameter Estimates for Dallas

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0075

0.0000

0

AR1,1

rpr_sa

0.6139

0.0000

1

NUM1

RES_FIXED_INV

0.0051

0.8198

2

NUM2

Civilian_Employment

– 0.0328

0.8516

3

Utility Model

MU

fu_sa

0.0060

0.2004

0

AR1,1

fu_sa

0.0189

0.8486

1

AR1,2

fu_sa

– 0.2122

0.0293

2

AR1,3

fu_sa

– 0.1436

0.1407

3

AR1,4

fu_sa

0.3700

0.0002

4

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C9

Exhibit C-19 | Parameter Estimates for Los Angeles

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0082

0.0001

0

MA1,1

rpr_sa

0.3979

0.0013

1

AR1,1

rpr_sa

0.9244

0.0000

1

NUM1

RES_FIXED_INV

0.0161

0.1526

2

NUM2

Civilian_Employment

0.2424

0.0033

3

Utility Model

MU

fu_sa

0.0081

0.0158

0

MA1,1

fu_sa

– 0.3324

0.0008

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

Exhibit C-20 | Parameter Estimates for San Francisco

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0098

0.0000

0

AR1,1

rpr_sa

0.8090

0.0000

1

NUM1

RES_FIXED_INV

0.0028

0.8581

2

NUM2

Civilian_Employment

0.3128

0.0087

3

Utility Model

MU

fu_sa

0.0108

0.0000

0

MA1,1

fu_sa

0.0341

0.7382

1

MA1,2

fu_sa

0.2889

0.0046

2

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C10

Exhibit C-21 | Parameter Estimates for Seattle

Parameter

Variable

Estimate

p-value

Lag/Shift

Rent Model

MU

rpr_sa

0.0090

0.0000

0

MA1,1

rpr_sa

– 0.7814

0.0000

1

NUM1

RES_FIXED_INV

– 0.0752

0.0005

2

NUM2

Civilian_Employment

0.4563

0.0180

3

Utility Model

MU

fu_sa

0.0097

0.0017

0

MA1,1

fu_sa

– 0.3813

0.0003

1

Notes: MU denotes the mean term; AR denotes the autoregressive parameters; MA is the moving average parameter; rpr_sa is the dependent variable on which the rent model was run; fu_sa is the dependent variable on which the utility model was run; RES_FIXED_INV and Civilian_Employment are used as independent variables in the national input rent model. An estimate is statistical y significant if its p-value was less than or equal to 5 percent.

C11

APPENDIX D. FORECAST RESULTS

Exhibit D-1 | Forecasts of Quarterly Rent and Utility, Northeast—All Classes

D1

Exhibit D-2 | Forecasts of Quarterly Rent and Utility, Midwest—All Classes

Exhibit D-3 | Forecasts of Quarterly Rent and Utility, South—All Classes

D2

Exhibit D-4 | Forecasts of Quarterly Rent and Utility, West—All Classes

Exhibit D-5 | Forecasts of Quarterly Rent and Utility, Northeast—Class A

D3

Exhibit D-6 | Forecasts of Quarterly Rent and Utility, Boston

Exhibit D-7 | Forecasts of Quarterly Rent and Utility, New York

D4

Exhibit D-8 | Forecasts of Quarterly Rent and Utility, Philadelphia

Exhibit D-9 | Forecasts of Quarterly Rent and Utility, Midwest—Class A

D5

Exhibit D-10 | Forecasts of Quarterly Rent and Utility, Chicago

Exhibit D-11 | Forecasts of Quarterly Rent and Utility, Detroit

D6

Exhibit D-12 | Forecasts of Quarterly Rent and Utility, South—Class A

Exhibit D-13 | Forecasts of Quarterly Rent and Utility, Washington, DC

D7

Exhibit D-14 | Forecasts of Quarterly Rent and Utility, Miami

Exhibit D-15 | Forecasts of Quarterly Rent and Utility, Atlanta

D8

Exhibit D-16 | Forecasts of Quarterly Rent and Utility, Baltimore

Exhibit D-17 | Forecasts of Quarterly Rent and Utility, Dallas

D9

Exhibit D-18 | Forecasts of Quarterly Rent and Utility, West—Class A

Exhibit D-19 | Forecasts of Quarterly Rent and Utility, Los Angeles

D10

Exhibit D-20 | Forecasts of Quarterly Rent and Utility, San Francisco

Exhibit D-21 | Forecasts of Quarterly Rent and Utility, Seattle

D11

REFERENCES

Armstrong, J. Scott, and Fred Collopy. 1992. “Error Measures for Generalizing About Forecasting Methods: Empirical Comparisons.” International Journal of Forecasting 8 (1): 69–80. DOI:

10.1016/0169-2070(92)90008-W.

Box, George E. P., and Gwilym M. Jenkins. 1976. Time Series Analysis: Forecasting and Control. San Francisco: Holden-Day.

Chai, Tianfeng, and Roland R. Draxler. 2014. “Root Mean Square Error (RMSE) or Mean Absolute Error (MAE)?–Arguments Against Avoiding RMSE in the Literature.” Geoscientific Model Development 7 (3): 1247–1250. DOI: 10.5194/gmd-7-1247-2014.

Gal in, Joshua. 2008. “The Long-Run Relationship Between House Prices and Rents.” Real Estate Economics 36 (4): 635–658. DOI: 10.1111/j.1540-6229.2008.00225.x.

Gómez, Víctor, and Agustín Maravall. 1997. Guide for Using the Programs TRAMO and SEATS: Beta Version: June 1998 . Madrid: Banco de España.

Gómez, Víctor, and Agustín Maraval . 1998. Guide for Using the Programs TRAMO and SEATS: Beta Version: December 1997. Working Paper No. 9805. Madrid: Banco de España.

81

79

Document Outline

Table of Contents

List of Exhibits

Acknowledgements

Executive Summary

1 Introduction

1.1 Understanding of the Problem

1.2 HUD’s Recent Efforts to Improve Trend Factor Calculations

1.3 Study Objectives

1.4 Contents of This Report

2 Data Sources and Issues

2.1 Dependent Variables

2.2 Exogenous Variables

2.2.1 Rent Model Variables

2.2.2 Utility Model Variables

2.3 Alternate Rent Series Using Commercial Data

3 Methodology

3.1 The HUD Model

3.2 Model Building Strategy

3.2.1 Model Identification

3.2.2 Parameter Estimation

3.2.3 Model Diagnostic Check

3.3 Testing and Measuring Model Performance

4 Phase 1 Results

4.1 Selecting Competing Rent and Utility Models

4.2 Forecast Error Comparison

4.3 Coefficient Significance of Selected Model

4.4 Forecasting Rent and Utilities

4.5 Estimating a Local Trend Factor

5 Phase II Results

5.1 Axiometrics Data

5.2 Methods to Assess the Data

5.2.1 Comparison to Bureau of Labor Statistics Data

5.2.2 Comparison to American Community Survey Data

5.2.3 Results

5.1.3 Assessment of Axiometrics Data

5.3 Zillow Data

5.3.1 Methods to Assess the Data

5.3.2 Comparison to Bureau of Labor Statistics Data

5.3.3 Comparison to American Community Survey Data

5.3.4 Results

5.3.5 Assessment of Zillow Data

5.4 Axiometrics Forecasts

6 Accuracy of Alternative Models

7 Conclusion

Appendix A. Lag Selection Results for Exogenous Variables

Appendix B. Residual Correlation Diagnostic Results

Appendix C. Parameter Estimate Results for the National Input Rent Model

Appendix D. Forecast Results

References