81 FR 70586 - Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys; Addendum: Standards and Guidelines for Cognitive Interviews

OFFICE OF MANAGEMENT AND BUDGET

Federal Register Volume 81, Issue 197 (October 12, 2016)

Page Range		70586-70587
FR Document		2016-24607

Under the Budget and Accounting Procedures Act of 1950 (31 U.S.C. 1104 (d)) and the Paperwork Reduction Act of 1995 (44 U.S.C. 3504 (e)), the Office of Management and Budget (OMB) is issuing Statistical Policy Directive No. 2 Addendum: Standards and Guidelines for Statistical Surveys. This Addendum provides Standards and Guidelines for Conducting Cognitive Interviews. The Addendum may be accessed at www.WhiteHouse.gov/sites/default/ files/omb/inforeg/directive2/final_addendum_to_stat_policy_dir_2.pdf. Additional discussion of the Addendum may be found in the SUPPLEMENTARY INFORMATION section below.

Federal Register, Volume 81 Issue 197 (Wednesday, October 12, 2016)

[Federal Register Volume 81, Number 197 (Wednesday, October 12, 2016)]
[Notices]
[Pages 70586-70587]
From the Federal Register Online  [www.thefederalregister.org]
[FR Doc No: 2016-24607]



[[Page 70585]]

Vol. 81

Wednesday,

No. 197

October 12, 2016

Part IV





Office of Management and Budget





-----------------------------------------------------------------------





Statistical Policy Directive No. 2: Standards and Guidelines for 
Statistical Surveys; Addendum: Standards and Guidelines for Cognitive 
Interviews; Notice

Federal Register / Vol. 81 , No. 197 / Wednesday, October 12, 2016 / 
Notices

[[Page 70586]]


-----------------------------------------------------------------------

OFFICE OF MANAGEMENT AND BUDGET


Statistical Policy Directive No. 2: Standards and Guidelines for 
Statistical Surveys; Addendum: Standards and Guidelines for Cognitive 
Interviews

AGENCY: Executive Office of the President, Office of Management and 
Budget.

ACTION: Notice of final decision.

-----------------------------------------------------------------------

SUMMARY: Under the Budget and Accounting Procedures Act of 1950 (31 
U.S.C. 1104 (d)) and the Paperwork Reduction Act of 1995 (44 U.S.C. 
3504 (e)), the Office of Management and Budget (OMB) is issuing 
Statistical Policy Directive No. 2 Addendum: Standards and Guidelines 
for Statistical Surveys. This Addendum provides Standards and 
Guidelines for Conducting Cognitive Interviews.
    The Addendum may be accessed at www.WhiteHouse.gov/sites/default/files/omb/inforeg/directive2/final_addendum_to_stat_policy_dir_2.pdf. 
Additional discussion of the Addendum may be found in the SUPPLEMENTARY 
INFORMATION section below.

DATES: Effective Date: This Addendum is effective immediately.

ADDRESSES: Please send any questions about this Directive to: Katherine 
K. Wallman, Chief Statistician, Office of Management and Budget, 10201 
New Executive Office Building, Washington, DC 20503, telephone number: 
(202) 395-3093, FAX number: (202) 395-7245. You may also send questions 
via email to [email protected]. Because of delays in the receipt 
of regular mail related to security screening, use of electronic 
communications is encouraged.
    Electronic Availability: This document is available on the Internet 
on the OMB Web site at www.WhiteHouse.gov/sites/default/files/omb/inforeg/directive2/FRN_stat_policy_dir_2_addendum.pdf.

FOR FURTHER INFORMATION CONTACT: Jennifer Park, 10201 New Executive 
Office Building, Washington, DC 20503, email address: [email protected] 
with subject Directive No. 2 Addendum: Standards and Guidelines for 
Cognitive Interviews, telephone number: (202) 395-9046, FAX number: 
(202) 395-7245.

SUPPLEMENTARY INFORMATION: The Nation relies on the flow of credible 
statistics to support the decisions of governments, businesses, 
individuals, households, and other organizations. Federal surveys 
collect much of the information available about the United States' 
economy, population, natural resources, environment, and public and 
private institutions. It is essential that these surveys collect 
information in a clear, straight-forward manner so as to maximize the 
accuracy of data while minimizing respondent burden.
    Scope: The Addendum applies to all Federal agencies subject to the 
Paperwork Reduction Act of 1995. Agencies not subject to the PRA also 
benefit from this Addendum, and are therefore encouraged to apply this 
guidance as well.
    Background: Consistent with the Information Quality Act (Pub. L. 
106-554, Division C, title V, Sec. 515, Dec. 21, 2000; 114 Stat. 2763A-
153 to 2763A-154) and in accordance with Statistical Policy Directive 
No. 1: Fundamental Responsibilities of Federal Statistical Agencies and 
Recognized Statistical Units (79 FR 71610, Dec. 2, 2014), it is the 
responsibility of Federal agencies engaging in statistical work to 
continuously review statistical methodologies and implement 
improvements as they are identified so as to better ensure the 
relevance, accuracy, and objectivity of the statistical products our 
Nation uses to monitor and assess performance, progress, and needs. 
Further, these responsibilities must be achieved in a manner that 
protects the confidentiality of information acquired solely for 
statistical purposes.
    In its role as coordinator of the Federal statistical system under 
the Paperwork Reduction Act, OMB, among other responsibilities, is 
required to ensure the efficiency and effectiveness of the system as 
well as the relevance, accuracy, objectivity, and confidentiality of 
information collected for statistical purposes. OMB is also charged 
with developing and overseeing the implementation of Government-wide 
principles, policies, standards, and guidelines concerning the 
development, presentation, and dissemination of statistical 
information.
    Statistical Policy Directive No. 2: Standards and Guidelines for 
Statistical Surveys was issued in 2006 to ensure implementation of 
improved survey methods across Federal agencies engaged in statistical 
activities. It remains a robust and comprehensive source of guidance. 
Nonetheless, periodic review, updates, and addenda are required to 
ensure that this Directive remains most useful to Federal agencies.
    Cognitive interviewing is a key method used to pretest survey 
questions and questionnaires. This method investigates whether 
respondents understand survey questions according to their intended 
design and whether respondents can provide accurate answers based on 
that intent. Cognitive interviews determine respondent interpretations 
and detail the phenomena considered by respondents in forming their 
answer. Findings from cognitive interviews can indicate whether a 
survey question captures the intended construct; cognitive interview 
findings can also identify difficulties that respondents experience 
when formulating responses. Ultimately, rigorous cognitive interviews 
support the efficient production of useful statistics since the 
findings from cognitive interviews can be used to help minimize costs 
and burden while ensuring the accuracy of the information collected.
    Although cognitive interviews are broadly used across a variety of 
Federal agencies engaged in statistical activities, to date no specific 
Federal guidance has established the manner in which this particular 
method should be conducted. As such, the term ``cognitive interview,'' 
when applied to Federal statistical surveys could refer to any of an 
array of related research techniques that differ in their appropriate 
use for statistical surveys. Accordingly, in 2014, OMB requested 
members of the Federal Interagency Council on Statistical Policy (ICSP) 
to nominate representatives for a new subcommittee formed under the 
aegis of the Federal Committee on Statistical Methodology (FCSM). The 
Question Evaluation Methodology Subcommittee was asked to identify best 
practices for conducting cognitive interviews so as to improve the 
resulting data quality. Subcommittee members reviewed relevant 
scientific literature, the Paperwork Reduction Act, the Information 
Quality Act, Government-wide Information Quality Guidelines 
(Information Quality Guidelines) (67 FR 8453, Jan. 3, 2002), 
Statistical Policy Directive No. 1 and Directive No. 2, and other 
information quality standards currently used by Federal statistical 
agencies. Through this careful process, the subcommittee reached a 
consensus in identifying seven standards for cognitive interviews. The 
subcommittee provided draft standards and guidelines for review by the 
FCSM and the ICSP in 2015. The subcommittee addressed the comments it 
received at each stage and provided its final recommendations to the 
FCSM in 2015.
    Accordingly, OMB requested comments on the recommendation that it 
received from the Federal Committee on Statistical Methodology (FCSM) 
Subcommittee on Question Evaluation Methodology to publish an addendum 
to OMB Statistical Policy Directive No.

[[Page 70587]]

2: Standards and Guidelines for Statistical Surveys. The Addendum, 
Standards and Guidelines for Cognitive Interviews, is intended to 
ensure that the results of statistical surveys sponsored by the Federal 
Government are as reliable and useful as possible while minimizing 
respondent burden. Specifically, the Addendum establishes rigorous 
standards for the use of cognitive interviews in Federal information 
collections. These standards allow data collectors and users alike to 
better evaluate the quality of survey questions and the resulting 
survey statistics.
    Disposition of Comments Received: On May 10, 2016, OMB published in 
the Federal Register (81 FR 29107, May 10, 2016) a notice seeking 
comments on a draft of this Addendum. No comments were received. 
Accordingly, OMB hereby adopts and issues the final Directive No. 2 
Addendum: Standards and Guidelines for Cognitive Interviews. The 
Addendum can be accessed at www.WhiteHouse.gov/sites/default/files/omb/inforeg/directive2/final_addendum_to_stat_policy_dir_2.pdf.

Howard Shelanski,
Administrator. Office of Information and Regulatory Affairs.
[FR Doc. 2016-24607 Filed 10-11-16; 8:45 am]
BILLING CODE P

70586 Federal Register / Vol. 81, No. 197 / Wednesday, October 12, 2016 / Notices

OFFICE OF MANAGEMENT AND information available about the United to their intended design and whether
BUDGET States’ economy, population, natural respondents can provide accurate
resources, environment, and public and answers based on that intent. Cognitive
Statistical Policy Directive No. 2: private institutions. It is essential that interviews determine respondent
Standards and Guidelines for these surveys collect information in a interpretations and detail the
Statistical Surveys; Addendum: clear, straight-forward manner so as to phenomena considered by respondents
Standards and Guidelines for maximize the accuracy of data while in forming their answer. Findings from
Cognitive Interviews minimizing respondent burden. cognitive interviews can indicate
Scope: The Addendum applies to all whether a survey question captures the
AGENCY: Executive Office of the Federal agencies subject to the intended construct; cognitive interview
President, Office of Management and Paperwork Reduction Act of 1995. findings can also identify difficulties
Budget. Agencies not subject to the PRA also that respondents experience when
ACTION: Notice of final decision. benefit from this Addendum, and are formulating responses. Ultimately,
therefore encouraged to apply this rigorous cognitive interviews support
SUMMARY: Under the Budget and guidance as well. the efficient production of useful
Accounting Procedures Act of 1950 (31 Background: Consistent with the statistics since the findings from
U.S.C. 1104 (d)) and the Paperwork Information Quality Act (Pub. L. 106– cognitive interviews can be used to help
Reduction Act of 1995 (44 U.S.C. 3504 554, Division C, title V, Sec. 515, Dec. minimize costs and burden while
(e)), the Office of Management and 21, 2000; 114 Stat. 2763A–153 to ensuring the accuracy of the information
Budget (OMB) is issuing Statistical 2763A–154) and in accordance with collected.
Policy Directive No. 2 Addendum: Statistical Policy Directive No. 1: Although cognitive interviews are
Standards and Guidelines for Statistical Fundamental Responsibilities of Federal broadly used across a variety of Federal
Surveys. This Addendum provides Statistical Agencies and Recognized agencies engaged in statistical activities,
Standards and Guidelines for Statistical Units (79 FR 71610, Dec. 2, to date no specific Federal guidance has
Conducting Cognitive Interviews. 2014), it is the responsibility of Federal established the manner in which this
The Addendum may be accessed at agencies engaging in statistical work to particular method should be conducted.
www.WhiteHouse.gov/sites/default/ continuously review statistical As such, the term ‘‘cognitive interview,’’
files/omb/inforeg/directive2/final_ methodologies and implement when applied to Federal statistical
addendum_to_stat_policy_dir_2.pdf. improvements as they are identified so surveys could refer to any of an array of
Additional discussion of the Addendum as to better ensure the relevance, related research techniques that differ in
may be found in the SUPPLEMENTARY accuracy, and objectivity of the their appropriate use for statistical
INFORMATION section below. statistical products our Nation uses to surveys. Accordingly, in 2014, OMB
DATES: Effective Date: This Addendum monitor and assess performance, requested members of the Federal
is effective immediately. progress, and needs. Further, these Interagency Council on Statistical Policy
ADDRESSES: Please send any questions responsibilities must be achieved in a (ICSP) to nominate representatives for a
about this Directive to: Katherine K. manner that protects the confidentiality new subcommittee formed under the
Wallman, Chief Statistician, Office of of information acquired solely for aegis of the Federal Committee on
Management and Budget, 10201 New statistical purposes. Statistical Methodology (FCSM). The
Executive Office Building, Washington, In its role as coordinator of the Question Evaluation Methodology
DC 20503, telephone number: (202) Federal statistical system under the Subcommittee was asked to identify
395–3093, FAX number: (202) 395– Paperwork Reduction Act, OMB, among best practices for conducting cognitive
7245. You may also send questions via other responsibilities, is required to interviews so as to improve the resulting
email to DirectiveNo2@omb.eop.gov. ensure the efficiency and effectiveness data quality. Subcommittee members
Because of delays in the receipt of of the system as well as the relevance, reviewed relevant scientific literature,
regular mail related to security accuracy, objectivity, and the Paperwork Reduction Act, the
screening, use of electronic confidentiality of information collected Information Quality Act, Government-
communications is encouraged. for statistical purposes. OMB is also wide Information Quality Guidelines
Electronic Availability: This charged with developing and overseeing (Information Quality Guidelines) (67 FR
document is available on the Internet on the implementation of Government- 8453, Jan. 3, 2002), Statistical Policy
the OMB Web site at wide principles, policies, standards, and Directive No. 1 and Directive No. 2, and
www.WhiteHouse.gov/sites/default/ guidelines concerning the development, other information quality standards
files/omb/inforeg/directive2/FRN_stat_ presentation, and dissemination of currently used by Federal statistical
policy_dir_2_addendum.pdf. statistical information. agencies. Through this careful process,
Statistical Policy Directive No. 2: the subcommittee reached a consensus
FOR FURTHER INFORMATION CONTACT:
Standards and Guidelines for Statistical in identifying seven standards for
Jennifer Park, 10201 New Executive
Surveys was issued in 2006 to ensure cognitive interviews. The subcommittee
Office Building, Washington, DC 20503,
implementation of improved survey provided draft standards and guidelines
email address: jpark@omb.eop.gov with
methods across Federal agencies for review by the FCSM and the ICSP in
subject Directive No. 2 Addendum:
engaged in statistical activities. It 2015. The subcommittee addressed the
Standards and Guidelines for Cognitive
remains a robust and comprehensive comments it received at each stage and
Interviews, telephone number: (202)
source of guidance. Nonetheless, provided its final recommendations to
395–9046, FAX number: (202) 395–
mstockstill on DSK3G9T082PROD with NOTICES4

periodic review, updates, and addenda the FCSM in 2015.
7245. Accordingly, OMB requested
are required to ensure that this Directive
SUPPLEMENTARY INFORMATION: The remains most useful to Federal agencies. comments on the recommendation that
Nation relies on the flow of credible Cognitive interviewing is a key it received from the Federal Committee
statistics to support the decisions of method used to pretest survey questions on Statistical Methodology (FCSM)
governments, businesses, individuals, and questionnaires. This method Subcommittee on Question Evaluation
households, and other organizations. investigates whether respondents Methodology to publish an addendum
Federal surveys collect much of the understand survey questions according to OMB Statistical Policy Directive No.

VerDate Sep<11>2014 17:50 Oct 11, 2016 Jkt 238001 PO 00000 Frm 00002 Fmt 4701 Sfmt 4703 E:\FR\FM\12OCN4.SGM 12OCN4

$Federal Register / Vol. 81, No. 197 / Wednesday, October 12, 2016 / Notices 70587 2: Standards and Guidelines for information collections. These Directive No. 2 Addendum: Standards Statistical Surveys. The Addendum, standards allow data collectors and and Guidelines for Cognitive Interviews. Standards and Guidelines for Cognitive users alike to better evaluate the quality The Addendum can be accessed at Interviews, is intended to ensure that the of survey questions and the resulting www.WhiteHouse.gov/sites/default/ results of statistical surveys sponsored survey statistics. files/omb/inforeg/directive2/final_ by the Federal Government are as Disposition of Comments Received: addendum_to_stat_policy_dir_2.pdf. reliable and useful as possible while On May 10, 2016, OMB published in the Federal Register (81 FR 29107, May 10, Howard Shelanski, minimizing respondent burden. Administrator. Office of Information and 2016) a notice seeking comments on a Specifically, the Addendum establishes Regulatory Affairs. draft of this Addendum. No comments rigorous standards for the use of were received. Accordingly, OMB [FR Doc. 2016–24607 Filed 10–11–16; 8:45 am] cognitive interviews in Federal hereby adopts and issues the final BILLING CODE P mstockstill on DSK3G9T082PROD with NOTICES4 VerDate Sep<11>2014 17:50 Oct 11, 2016 Jkt 238001 PO 00000 Frm 00003 Fmt 4701 Sfmt 9990 E:\FR\FM\12OCN4.SGM 12OCN4$

Document Created: 2016-10-12 00:55:29
Document Modified: 2016-10-12 00:55:29

Category		Regulatory Information
Collection		Federal Register
sudoc Class		AE 2.7: GS 4.107: AE 2.106:
Publisher		Office of the Federal Register, National Archives and Records Administration
Section		Notices
Action		Notice of final decision.
Contact		Jennifer Park, 10201 New Executive
FR Citation		81 FR 70586