82 FR 4284 - Generic Clearance for Proposed Information Collection; Comment Request; Generic Clearance for Internet Nonprobability Panel Pretesting and Qualitative Survey Methods Testing

DEPARTMENT OF COMMERCE
Census Bureau

Federal Register Volume 82, Issue 9 (January 13, 2017)

The Department of Commerce, as part of its continuing effort to reduce paperwork and respondent burden, invites the general public and other Federal agencies to take this opportunity to comment on proposed and/or continuing information collections, as required by the Paperwork Reduction Act of 1995.

[Federal Register Volume 82, Number 9 (Friday, January 13, 2017)]
[Notices]
[Pages 4284-4286]
From the Federal Register Online  [www.thefederalregister.org]
[FR Doc No: 2017-00584]


=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF COMMERCE

 Census Bureau


Generic Clearance for Proposed Information Collection; Comment 
Request; Generic Clearance for Internet Nonprobability Panel Pretesting 
and Qualitative Survey Methods Testing

AGENCY: U.S. Census Bureau, Commerce.

ACTION: Notice.

-----------------------------------------------------------------------

SUMMARY: The Department of Commerce, as part of its continuing effort 
to reduce paperwork and respondent burden, invites the general public 
and other Federal agencies to take this opportunity to comment on 
proposed and/or continuing information collections, as required by the 
Paperwork Reduction Act of 1995.

DATES: To ensure consideration, written comments must be submitted on 
or before March 14, 2017.

ADDRESSES: Direct all written comments to Jennifer Jessup, Departmental 
Paperwork Clearance Officer, Department of Commerce, Room 6616, 14th 
and Constitution Avenue NW., Washington, DC 20230 (or via the Internet 
at [email protected]).

FOR FURTHER INFORMATION CONTACT: Requests for additional information or 
copies of the information collection instrument(s) and instructions 
should be directed to Jennifer Hunter Childs, U.S. Census Bureau, 4600 
Silver Hill Road, Center for Survey Measurement, Washington, DC 20233 
or (202)603-4827.

SUPPLEMENTARY INFORMATION:

I. Abstract

    The Census Bureau is committed to conducting research in a cost 
efficient manner. Prior to this generic clearance, several stages of 
testing occurred in research projects at the Census Bureau. As a first 
stage of research, the Census Bureau pretests questions on surveys or 
censuses and evaluates the usability and ease of use of Web sites using 
a small number of subjects during focus groups, usability and cognitive 
testing. These projects are in-person and labor-intensive, but 
typically only target samples of 20 to 30 respondents. This small-scale 
work is done through another existing OMB generic clearance. Often the 
second stage is a larger-scale field test with a split-panel design of 
a survey or a release of a Census Bureau data dissemination product 
with a feedback mechanism. The field tests often involve a lot of 
preparatory work and often are limited in the number of panels tested 
due to the cost considerations. They are often targeted at very large 
sample sizes with over 10,000 respondents per panel. These are 
typically done using stand-alone OMB clearances.
    Cost efficiencies can occur by testing some research questions in a 
medium-scale test, using a smaller number of participants than what we 
typically use in a field test, yet a larger and more diverse set of 
participants than who we recruit for cognitive and usability tests. 
Using Internet panel pretesting, we can answer some research questions 
more thoroughly than in the small-scale testing, but less expensively 
than in the large-scale field test. This clearance established a 
medium-scale (defined as having sample sizes from 100-2000 per study), 
cost-efficient method of testing

[[Page 4285]]

questions and contact strategies over the Internet through different 
types of nonprobability samples.
    This research program will be used by the Census Bureau and survey 
sponsors to test alternative contact methods, including emails and text 
messages (via an opt-in strategy), improve online questionnaires and 
procedures, reduce respondent burden, and ultimately increase the 
quality of data collected in the Census Bureau censuses and surveys. We 
will use the clearance to conduct pretesting of decennial and 
demographic census and survey questionnaires prior to fielding them as 
well as communications and/or marketing strategies and data 
dissemination tools for the Census Bureau. The primary method of 
identifying measurement problems with the questionnaire or survey 
procedure is split panel tests. This will encompass both methodological 
and subject matter research questions that can be tested on a medium-
scale nonprobability panel.
    This research program will also be used by the Census Bureau for 
remote usability testing of electronic interfaces and to perform other 
qualitative analyses such as respondent debriefings. An advantage of 
using remote, medium-scale testing is that participants can test 
products at their convenience using their own equipment, as opposed to 
using Census Bureau-supplied computers. A diverse participant pool 
(geographically, demographically, or economically) is another 
advantage. Remote usability testing would use click through rates and 
other paradata, accuracy and satisfaction scores, and written 
qualitative comments to determine optimal interface designs and to 
obtain feedback from respondents.
    The public is currently offered an opportunity to participate in 
this research remotely, by signing up for an online research panel. If 
a person opts in, the Census Bureau will occasionally email (or text, 
if applicable) the person an invitation to complete a survey for one of 
our research projects. Invited respondents will be told the topic of 
the survey, and how long it will take to complete it. Under this 
clearance, we will also conduct similar-scale and similarly designed 
research using other email lists to validate preliminary findings and 
expand the research.

II. Method of Collection

    Split sample experiments. This involves testing alternative 
versions of questionnaires, invitations to questionnaires (e.g., emails 
or text messages), or Web sites, at least some of which have been 
designed to address problems identified in draft versions or versions 
from previous waves. The use of multiple questionnaires, invitations, 
or Web sites, randomly assigned to permit statistical comparisons, is 
the critical component here; data collection will be via the Internet. 
Comparison of revised questionnaires (or invitations) against a control 
version, preferably, or against each other facilitates statistical 
evaluation of the performance of alternative versions of the 
questionnaire (or invitation or Web site).
    The number of versions tested and the number of cases per version 
will depend on the objectives of the test. We cannot specify with 
certainty a minimum panel size, although we would expect that no 
questionnaire versions would be administered to less than fifty 
respondents.
    Split sample tests that incorporate methodological questionnaire 
design experiments will have a larger maximum sample size (up to 
several hundred cases per panel) than other pretest methods. This will 
enable the detection of statistically significant differences, and 
facilitate methodological experiments that can extend questionnaire 
design knowledge more generally for use in a variety of Census Bureau 
data collection instruments.
    Usability Interviews: This method involves getting respondent input 
to aid in the development of automated questionnaires and Web sites and 
associated materials. The objective is to identify problems that keep 
respondents from completing automated questionnaires accurately and 
efficiently with minimal burden, or that prevent respondents from 
successfully navigating Web sites and finding the information they 
seek. Remote usability testing may be conducted under this clearance, 
whereby a user would receive an invitation to use a Web site or survey, 
then answer targeted questions about that experience.
    Qualitative Interviews: This method involves one-on-one (or 
sometimes group) interviews in which the respondent is typically asked 
questions about survey content areas, survey questions or the survey 
process. A number of different techniques may be involved, including 
cognitive interviews and focus groups. The objective is to identify 
problems of ambiguity or misunderstanding, or other difficulties 
respondents may have answering survey questions in order to improve the 
information ultimately collected in large scale surveys and censuses.
    Data collection for this project is authorized under the 
authorizing legislation for the questionnaire being tested. This 
authorization may be Title 13, United States Code (U.S.C.), Sections 
131, 141, 161, 181, 182, 193, and 301 for Census Bureau-sponsored 
surveys, and Title 13 and 15 for surveys sponsored by other Federal 
agencies. We do not now know what other titles will be referenced, 
since we do not know what survey questionnaires will be pretested 
during the course of the clearance.
    Literature on and considerations about the use of nonprobability 
samples for this type of work have recently been thoroughly covered by 
a Task Force commissioned by the American Association for Public 
Opinion Research and are well documented there (Baker, et al., 2013).
    The information collected in this program of developing and testing 
questionnaires will be used by staff from the Census Bureau and 
sponsoring agencies to evaluate and improve the quality of the data in 
the surveys and censuses that are ultimately conducted. Because the 
questionnaires being tested under this clearance are still in the 
process of development, the data that result from these collections are 
not considered official statistics of the Census Bureau or other 
Federal agencies. Data will be included in research reports prepared 
for sponsors inside and outside of the Census Bureau. The results may 
also be prepared for presentations related to survey methodology at 
professional meetings or publications in professional journals.

III. Data

    OMB Control Number: 0607-0978.
    Form Number(s): TBD.
    Type of Review: Extension of a Currently Approved Collection.
    Affected Public: Individuals and households.
    Number of Respondents: 60,000.
    Average Hours per Response: 0.167
    Burden Hours: 10,000.
    Estimated Total Annual Cost to Public: None.
    Respondent's Obligation: Voluntary.
    Legal Authority: Data collection for this project is authorized 
under the authorizing legislation for the questionnaire being tested. 
This may be Title 13, U.S.C., Sections 131, 141, 161, 181, 182, 193, 
and 301 for Census Bureau-sponsored surveys, and Title 13 and 15 for 
surveys sponsored by other Federal agencies. We do not now know what 
other titles will be referenced, since we do not know what survey 
questionnaires will be pretested during the course of the clearance.

IV. Request for Comments

    Comments are invited on: (a) Whether the proposed collection of 
information

[[Page 4286]]

is necessary for the proper performance of the functions of the agency, 
including whether the information shall have practical utility; (b) the 
accuracy of the agency's estimate of the burden (including hours and 
cost) of the proposed collection of information; (c) ways to enhance 
the quality, utility, and clarity of the information to be collected; 
and (d) ways to minimize the burden of the collection of information on 
respondents, including through the use of automated collection 
techniques or other forms of information technology.
    Comments submitted in response to this notice will be summarized 
and/or included in the request for OMB approval of this information 
collection; they also will become a matter of public record.

Sheleen Dumas,
PRA Departmental Lead, Office of the Chief Information Officer.
[FR Doc. 2017-00584 Filed 1-12-17; 8:45 am]
 BILLING CODE 3510-07-P