81 FR 87949 - HUD Program Evaluation Policy-Policy Statement

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

Federal Register Volume 81, Issue 234 (December 6, 2016)

This policy statement of HUD's Office of Policy Development and Research articulates the core principles and practices of the office's evaluation and research activities. This policy reconfirms the Department's commitment to conducting rigorous, relevant evaluations and to using evidence from evaluations to inform policy and practice.

[Federal Register Volume 81, Number 234 (Tuesday, December 6, 2016)]
[Notices]
[Pages 87949-87951]
From the Federal Register Online  [www.thefederalregister.org]
[FR Doc No: 2016-29215]


-----------------------------------------------------------------------

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

[Docket No. FR-5985-N-01]


HUD Program Evaluation Policy--Policy Statement

AGENCY: Office of the Assistant Secretary for Policy Development and 
Research, HUD.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: This policy statement of HUD's Office of Policy Development 
and Research articulates the core principles and practices of the 
office's evaluation and research activities. This policy reconfirms the 
Department's commitment to conducting rigorous, relevant evaluations 
and to using evidence from evaluations to inform policy and practice.

DATES: December 6, 2016.

FOR FURTHER INFORMATION CONTACT: Mark D. Shroder, Associate Deputy 
Assistant Secretary, Office of Research, Evaluation, and Monitoring, 
Office of Policy Development and Research, U.S. Department of Housing 
and Urban Development, 451 7th Street SW., Washington, DC 20410, 
telephone (202) 402-5922. The listed telephone number is not a toll-
free number. Persons with hearing- or speech-impairments may access 
this number through TTY by calling Federal Relay Service at 1-800-877-
8339 (this is a toll-free number).

SUPPLEMENTARY INFORMATION:

I. Background

    The mission of HUD's Office of Policy Development and Research 
(PD&R) is to inform HUD policy development and implementation to 
improve life in American communities through conducting, supporting, 
and sharing research, surveys, demonstrations,

[[Page 87950]]

program evaluations, and best practices. Within HUD, PD&R is 
responsible for nearly all program evaluations. The office provides 
reliable and objective data and analysis to help inform policy 
decisions. Program evaluation has been a core activity of PD&R since 
its formation in 1974.
    In July 2016, the Government Accountability Office (GAO) issued a 
report entitled ``Department of Housing and Urban Development: Actions 
Needed to Incorporate Key Practices into Management Functions and 
Program Oversight,'' (GAO 16-497) in which GAO presented a broad 
assessment of HUD's management of its operations and programs.\1\ In 
the report, GAO examined HUD efforts to: (1) Meet Federal requirements 
and implement key practices for management functions, including 
performance planning and reporting, human capital, financial, 
acquisition, and information technology (IT) management; and (2) 
oversee and evaluate programs.
---------------------------------------------------------------------------

    \1\ See http://www.gao.gov/assets/680/678551.pdf.
---------------------------------------------------------------------------

    PD&R is the primary office within HUD responsible for data 
analysis, research, program evaluations, and policy studies that inform 
the development and implementation of programs and policies across HUD 
offices. PD&R undertakes program evaluations, often by using a process 
that includes convening expert panels. However, GAO found that PD&R had 
not developed agency-wide, written policies for its program 
evaluations, nor documented the criteria used to select the expert 
panels and review the quality of program evaluations.
    This policy statement responds to the GAO report by setting out the 
core principles and practices of PD&R's evaluation and research 
activities. This statement incorporates some language from a policy 
statement by the Office of Policy, Research, and Evaluation of the 
Administration for Children and Families of the U.S. Department of 
Health and Human Services.

II. HUD Program Evaluation Policy

    PD&R has identified the following core principles and practices as 
fundamental to ensuring high-quality and consistent evaluation results: 
rigor, relevance, transparency, independence, ethics, and technical 
innovation. This policy applies to all PD&R-sponsored evaluations and 
economic analyses of regulations; they apply as well to the selection 
of projects, contractors, and PD&R staff that is involved in 
evaluations.

Rigor

    PD&R is committed to using the most rigorous methods that are 
appropriate to the evaluation questions and feasible within budget and 
other constraints. Rigor is not restricted to impact evaluations, but 
is also necessary in implementation or process evaluations, descriptive 
studies, outcome evaluations, and formative evaluations; and in both 
qualitative and quantitative approaches. Rigor requires ensuring that 
inferences about cause and effect are well founded (internal validity); 
requires clarity about the populations, settings, or circumstances to 
which results can be generalized (external validity); and requires the 
use of measures that accurately capture the intended information 
(measurement reliability and validity).
    In assessing the effects of programs or services, PD&R evaluations 
use methods that isolate to the greatest extent possible the impacts of 
the programs or services from other influences such as trends over 
time, geographic variation, or pre-existing differences between 
participants and non-participants. For such causal questions, 
experimental approaches are preferred. When experimental approaches are 
not feasible, PD&R uses the most rigorous approach that is feasible. 
PD&R ensures that contractors and grantees conducting evaluations have 
appropriate expertise through emphasizing the capacity for rigor in 
requests for proposal and funding opportunity announcements.
    PD&R also employs a strategic human capital development plan to 
hire, train, and retain a workforce that ensures the staff has the 
tools and resources to accomplish the mission.

Relevance

    The PD&R evaluation agenda reflects the legislative requirements 
and policy issues related to HUD's mission. PD&R solicits input from 
stakeholders, both internal and external, on the selection of programs 
to be evaluated, initiatives, demonstrations, and research questions. 
For new initiatives and demonstrations in particular, evaluations will 
be more feasible and useful when planned in advance, in concert with 
the development of the initiative or demonstration, rather than as an 
afterthought.
    PD&R disseminates findings in ways that are accessible and useful 
to policy-makers and practitioners. PD&R partners with other HUD 
program offices to inform internal and external stakeholders through 
disseminating evidence from PD&R-sponsored evaluations.

Transparency

    PD&R will release methodologically valid evaluations without regard 
to the findings. Evaluation reports must describe the methods used, 
including strengths and weaknesses, and discuss the generalizability of 
the findings. Evaluation reports must present comprehensive results, 
including favorable, unfavorable, and null findings.
    PD&R publishes a 5-year Research Roadmap that outlines the research 
and evaluation that we believe would be of greatest value to public 
policy. PD&R lists all ongoing evaluation projects at the HUDUSER.gov 
Web site, and updates it monthly. PD&R will release evaluation results 
timely, usually within 4 months of receiving the final report.
    PD&R will, where possible, archive evaluation data for secondary 
use by interested researchers. PD&R typically builds requirements into 
contracts to prepare data sets for secondary use.

Independence

    Independence and objectivity are core principles of evaluation. 
Agency and program leadership, program staff, service providers, and 
others participate actively in setting evaluation priorities, 
identifying evaluation questions, and assessing the implications of 
findings. However, it is important to insulate evaluation functions 
from undue influence and from both the appearance and the reality of 
bias. To promote objectivity, PD&R protects independence in the design, 
conduct, and analysis of evaluations. To this end:
     PD&R conducts evaluations through the competitive award of 
grants and contracts to external experts who are free from conflicts of 
interest.
     PD&R also conducts evaluations in-house and supports 
unsolicited external evaluation proposals with funding, data, or both.
     The Assistant Secretary for PD&R has authority to approve 
the design of evaluation projects and analysis plans; and has authority 
to approve, release, and disseminate evaluation reports. The Assistant 
Secretary does so, in consultation with career staff.

Ethics

    PD&R-sponsored evaluations must be conducted in an ethical manner 
and safeguard the dignity, rights, safety, and privacy of participants. 
PD&R-sponsored evaluations must comply with both the spirit and the 
letter of relevant requirements such as regulations governing research 
involving human subjects. In particular, PD&R protects the privacy of 
HUD-assisted households

[[Page 87951]]

and HUD-insured borrowers through the Rule of Eleven; that is, PD&R 
allows no disclosure of information about the characteristics of any 
group of individuals or households numbering less than eleven by PD&R 
staff, contractors, grantees, or licensees.

Technical Innovation

    PD&R supports and employs new methods of data collection and 
analysis that more reliably and efficiently answer research questions 
than old methods do.

Application of These Principles to Economic Analysis of Regulations

    Economic analysis of regulations, properly conducted, is a critical 
tool in improving public policy. In any PD&R Regulatory Impact 
Analysis:
     PD&R analyzes whether the issues addressed by the 
regulation stem from a market failure, government failure, or other 
systemic problem, and whether the regulation addresses the root causes 
of those problems.
     PD&R uses and as necessary produces the best objective 
estimates of the benefits, costs, and transfers resulting from the 
regulation, taking into account gaps and uncertainties in the available 
data.
     Where clear alternatives to the regulatory actions exist, 
PD&R objectively estimates the benefits, costs, and transfers of those 
alternatives as well.

    Dated: November 30, 2016.
Katherine O'Regan,
Assistant Secretary for Policy Development and Research.
[FR Doc. 2016-29215 Filed 12-5-16; 8:45 am]
 BILLING CODE 4210-67-P