80_FR_38845 80 FR 38716 - National Registry of Evidence-Based Programs and Practices

80 FR 38716 - National Registry of Evidence-Based Programs and Practices

DEPARTMENT OF HEALTH AND HUMAN SERVICES
Substance Abuse and Mental Health Services Administration

Federal Register Volume 80, Issue 129 (July 7, 2015)

Page Range38716-38718
FR Document2015-16573

The mission of SAMHSA is to reduce the impact of substance abuse and mental illness on America's communities. Established in 1992, the agency was directed by Congress to target effective substance abuse and mental health services to the people most in need, and to translate research in these areas more effectively and more rapidly into the general health care system. NREPP is a key public resource SAMHSA has developed to help meet this directive. This notice announces the redesign of NREPP to better align the registry with the standards and processes of other evidence-based repositories. A re-launch of the Web site with revised content is anticipated in late fall/early winter 2015. The notice explains the changes in how programs and practices will be identified for NREPP, how submissions will be screened and reviewed, and provides guidance on accessing updated information on the NREPP site. Potential applicants should be aware that this notice includes updated information relating to the eligibility of interventions for inclusion in NREPP and changes in the program and practice review process that supersedes guidance provided in earlier Federal Register notices.

Federal Register, Volume 80 Issue 129 (Tuesday, July 7, 2015)
[Federal Register Volume 80, Number 129 (Tuesday, July 7, 2015)]
[Notices]
[Pages 38716-38718]
From the Federal Register Online  [www.thefederalregister.org]
[FR Doc No: 2015-16573]


-----------------------------------------------------------------------

DEPARTMENT OF HEALTH AND HUMAN SERVICES

Substance Abuse and Mental Health Services Administration


National Registry of Evidence-Based Programs and Practices

AGENCY: Substance Abuse and Mental Health Services Administration 
(SAMHSA), HHS.

ACTION: Notice regarding SAMHSA's NREPP: Redesign of NREPP.

-----------------------------------------------------------------------

SUMMARY: The mission of SAMHSA is to reduce the impact of substance 
abuse and mental illness on America's communities. Established in 1992, 
the agency was directed by Congress to target effective substance abuse 
and mental health services to the people most in need, and to translate 
research in these areas more effectively and more rapidly into the 
general health care system. NREPP is a key public resource SAMHSA has 
developed to help meet this directive. This notice announces the 
redesign of NREPP to better align the registry with the standards and 
processes of other evidence-based repositories. A re-launch of the Web 
site with revised content is anticipated in late fall/early winter 
2015.
    The notice explains the changes in how programs and practices will 
be identified for NREPP, how submissions will be screened and reviewed, 
and provides guidance on accessing updated information on the NREPP 
site. Potential applicants should be aware that this notice includes 
updated information relating to the eligibility of interventions for 
inclusion in NREPP and changes in the program and practice review 
process that supersedes guidance provided in earlier Federal Register 
notices.

FOR FURTHER INFORMATION CONTACT: Carter Roeber, Ph.D., Social Science 
Analyst, Center for Behavioral Health Statistics and Quality, SAMHSA, 1 
Choke Cherry Road, Room 2-1050, Rockville, MD 20857, telephone 240-276-
1488.

SUPPLEMENTARY INFORMATION: 

Advancing Evidence-Based Programs and Practices Through Improved 
Decision Support Tools: Reconceptualizing NREPP

Introduction

    SAMHSA's NREPP is an evidence-based repository and review system 
designed to provide the public with reliable information about 
behavioral health interventions in the areas of mental health and 
substance abuse. Programs and practices that are accepted for inclusion 
in NREPP undergo a review process that provides information on the 
quality of research and the magnitude and direction of program or 
practice impact on individual outcomes. Materials for dissemination are 
reviewed to determine the type and extent of information available to 
support implementation. The results of these reviews are published on 
the NREPP Web site (http://nrepp.samhsa.gov).
    It should be noted that inclusion in NREPP indicates that some, but 
not necessarily all, of the evidence for a program or practice has been 
reviewed. In some cases, the quality of the research supporting the 
program or practice may have been determined to be poor or insufficient 
to earn a rating. Inclusion in NREPP does not constitute endorsement of 
an intervention as effective by SAMHSA. Moreover, since NREPP has not 
reviewed all behavioral health interventions, the use of NREPP as an 
exclusive or exhaustive list of interventions is not appropriate. 
Policymakers and funders in particular are discouraged from limiting 
providers and/or potential grantees to selecting exclusively from among 
NREPP interventions and from funding NREPP interventions regardless of 
the ratings the interventions receive.
    This notice announces changes to (1) the process for identifying 
new programs and practices for NREPP review, (2) the process for 
announcing open submission periods, (3) the minimum requirements to be 
considered for NREPP review, and (4) the review process. This notice 
also announces the intent to re-review currently posted NREPP programs 
and practices to comport with new review criteria and ratings. The re-
review of programs and practices currently posted will take place over 
the course of the next several years, depending on available resources. 
A re-launch of the NREPP Web site will take place in phases and the 
first phase is planned for late fall/early winter of 2015.

Identifying New Programs and Practices for NREPP

    Open submissions periods, during which applicants may submit 
materials for review, will continue to be used to identify new programs 
and practices for review (see below). Programs and practices addressing 
specific SAMHSA priorities may also be identified by SAMHSA or through 
environmental scans (including literature reviews, focus groups, public 
input, and interviews), as time and resources permit. Programs and 
practices related to priority areas may be reviewed before programs and 
practices identified through the open submission period. SAMHSA will be 
consulting with subject matter experts and leadership in underserved 
groups and populations, including American Indian/Alaska Native Tribes, 
regarding ways to incorporate traditional and culturally-specific 
interventions into NREPP, in order to better meet the needs of groups 
whose efforts to promote behavioral health may not have not been 
routinely evaluated. Innovative, but perhaps less rigorously tested, 
programs and

[[Page 38717]]

practices will be considered if specific to SAMHSA priority areas, as a 
parallel contribution to the standard review procedures.
    Stand-alone pharmacologic treatments are not eligible for review 
and should not be submitted to NREPP. The evidence base for 
pharmacologic treatments is reviewed and approved through the U.S. Food 
and Drug Administration (FDA). FDA approved pharmacotherapy 
interventions (on-label use) are considered for NREPP review only when 
combined with one or more behavioral or psychosocial treatments.
    SAMHSA reserves the right to reject for review programs and 
practices whose goals or activities are determined to be inconsistent 
with SAMHSA's mission, which is ``to reduce the impact of substance 
abuse and mental illness on American communities.''

Open Submission Periods

    SAMHSA accepts new applications for review during open submission 
periods. SAMHSA generally holds one open submission period a year but, 
depending on the number of reviews in progress and resources available 
may hold more or fewer within a calendar year. All future open 
submission periods will be announced on the NREPP Web site. Emails will 
also be sent announcing the open submission period to those on the 
NREPP listserv. Anyone wishing to be notified of future open submission 
periods can join the NREPP listserv by sending a request to 
[email protected].
    Applications can be submitted at any point during an open period. 
Program and practice developers, researchers, and others interested in 
submitting an intervention should read below for information about the 
new minimum requirements to be considered for an NREPP review. 
Additional future changes to the review process and criteria will be 
posted on the NREPP Web site as they are implemented. Therefore, before 
submitting a program or practice for NREPP review, applicants should 
examine the most recent information posted on the NREPP Web site about 
the review process and criteria and the most recent guidance for 
preparing an intervention for submission (see http://www.nrepp.samhsa.gov/ReviewSubmission.aspx). This guidance will be 
periodically updated to reflect the redesign of NREPP during the re-
launch.
    The selection of interventions will take place after the closing of 
the open submission period, and applicants will be notified whether 
they have been accepted after an initial screening to ensure that the 
application meets minimum requirements. The number of reviews conducted 
will depend on the availability of funds, with the timing of reviews to 
be determined by SAMHSA. In submitting an intervention, applicants 
should understand that if interventions are selected for review, the 
results of NREPP reviews are considered public information and will be 
posted on the NREPP Web site. Once a review is completed, the applicant 
will be provided with a summary document (``the program or practice 
profile'') that presents results of the review, ratings of program 
effectiveness, and descriptive information about the intervention. The 
applicant will have the opportunity to comment on the profile before it 
is posted but they will not have the option to refuse posting.

Minimum Requirements

    To be considered for review, interventions must meet three minimum 
requirements:
    1. Research or evaluation of the intervention has assessed mental 
health or substance use outcomes among individuals, communities, or 
populations OR other behavioral health-related outcomes on individuals, 
communities, or populations with or at risk of mental health issues or 
substance use problems.
    2. Evidence of these outcomes has been demonstrated in at least one 
study using an experimental or quasi-experimental design. Experimental 
designs require random assignment, a control or comparison group, and 
pre- and post-intervention outcome assessments. Quasi-experimental 
designs do not require random assignment, but do require a comparison 
or control group and pre- and post-intervention outcome assessments. 
Comparison/control groups must be a no-treatment control group, a wait-
list control group, a treatment-as-usual comparison group, or an 
intervention that is presumed to be ineffective or substantially less 
effective than the intervention (e.g., a ``placebo'' control or, in 
cases in which providing no treatment might be considered unethical, 
less effective treatments, even if not treatment-as-usual, such as 
``supportive therapy''). Studies with single-group, pretest-posttest 
designs or single-group, longitudinal/multiple time series do not meet 
this requirement, but will be considered to identify emerging programs 
and practices for consideration in the Learning Center. Comparative 
effectiveness trials, in which two interventions, both presumed to be 
equally effective, are compared, and studies in which the effects of 
the same intervention on various subpopulations are compared or in 
which various doses or components of the same intervention are compared 
will be reviewed, but only for purposes of providing readers in the 
Learning Center the opportunity to determine whether a particular 
program or practice may have use or show promise in different settings 
or circumstances.
    3. The results of these studies have been published in a peer-
reviewed journal or other professional publication, or documented in a 
comprehensive evaluation report, published within the previous 25 
years. Comprehensive evaluation reports must include a review of the 
literature, theoretical framework, purpose, methodology, findings/
results with statistical analysis and p values for significant 
outcomes, discussion, and conclusions.
    Applicants are required to provide full-text documents at the time 
of submission that demonstrate the intervention meets these minimum 
requirements. Other research articles, published or unpublished 
evaluation reports, grant final reports, and replication studies may be 
submitted as additional supporting documentation. Note: Abstracts or 
links to partial articles are regarded as incomplete and will not be 
considered.
    NREPP will no longer require programs and practices to have 
developed implementation materials, training and support resources, and 
quality assurance procedures. However, programs and practices with such 
dissemination and/or implementation resources will be considered for 
prioritized review if within the priorities established by the SAMHSA 
review process, and the materials, along with the location of their 
availability, will be listed in the program or practice profile.
    Applicants submitting dissemination and implementation resources 
should include a brief narrative description of the materials that are 
being submitted. These materials may include, but are not limited to, 
treatment manuals, information for administrators, information for 
direct service staff, tested training curricula, mechanisms for ongoing 
supervision and consultation, protocols for gathering process and 
outcome data, ongoing monitoring of intervention fidelity, and 
processes for gathering feedback. Applicants should also provide the 
location of where the materials can be obtained.

[[Page 38718]]

Selection of Interventions for Review

    SAMHSA will select interventions for review from among submissions 
meeting the minimum requirements. In selecting interventions for 
review, SAMHSA may give special consideration to interventions that 
meet one or more of the following conditions:
     More than one research study or evaluation has been 
conducted on the same or a similar target population that meets the 
minimum requirements.
     The intervention targets underserved populations (e.g., 
minority populations, tribal communities or American Indian/Alaska 
Native populations, elderly individuals, young adults, individuals who 
are incarcerated, etc.).
     Dissemination and implementation materials (e.g., program 
or practice manuals, training guides, measurement instruments, 
implementation fidelity tools) are available. Lower costs and no-cost 
materials may be prioritized.
     The intervention contributes to a content area in which 
few evidence-based interventions have been previously identified.
    Interventions that are not selected for review may be resubmitted 
by the applicant in a future open submission period.

The Review Process

    The review process has been revised to improve the quality of the 
reviews and utility of information that NREPP can provide its users. In 
addition to articles and reports submitted by NREPP applicants, 
additional studies, articles, and evaluation reports regarding the 
interventions will be identified through literature searches. Studies 
and outcomes to be reviewed will be determined through the systematic 
application of standardized screening criteria, and the number of 
studies and outcomes to be reviewed will be expanded to more 
comprehensively represent the evidence base for the program or 
practice. Inclusion of studies and outcomes will no longer be limited 
to positive significant outcomes; all studies and outcomes that meet 
the standardized screening criteria will be reviewed, including those 
with negative and non-significant effects. Programs and practices will 
be assessed on the basis of evaluation studies of program or practice 
impact, information related to conceptual framework (that is, program 
or practice goals, theory of change, and program or practice 
components), and information about implementation fidelity (that is, 
whether a study employs quality assurance measures to declare that the 
program or practice is delivered as intended to the program's or 
practice's target population).
    The methodological rigor (that is, internal validity, statistical 
validity, and measurement validity) of the research for each program or 
practice will be reviewed, as it pertains to each outcome examined, 
along with the magnitude and direction of the program's or practice's 
effect on each outcome. Based on this information, the program's or 
practice's effectiveness for each outcome will be rated, along with the 
rigor of the research examining the program or practice, and the 
ratings will be displayed on the NREPP Web site.
    In general, each NREPP evidence review will be conducted by two 
trained and certified reviewers. However, based on funding and 
available resources, SAMHSA use one reviewer for programs and practices 
being re-reviewed. When necessary, NREPP may conduct author queries to 
confirm or gather additional information needed for the review. Program 
and practice profiles will be developed on the basis of the information 
gathered. Applicants will have the opportunity to review the program or 
practice profile before it is posted on the NREPP site, but they will 
not have the option to refuse posting.
    Dissemination and implementation materials will no longer be rated 
as they were historically. Instead, descriptions of available materials 
for each program or practice, highlighting information that may be of 
most interest to NREPP users, will be included in the program or 
practice profile, along with information documenting the extent to 
which materials are available.
    Programs and practices currently posted on NREPP will be re-
reviewed as time and resources permit but the re-reviews of currently 
posted programs and practices will take place over the next few years.
    Detailed information about the revised review process will be 
available at http://www.nrepp.samhsa.gov after the re-launch of the new 
NREPP Web site.

Enhancing the Learning Center

    NREPP's Learning Center is a developing and underutilized component 
of the NREPP Web site. With the evolution and enhancement of the 
registry, SAMHSA seeks to bring greater recognition to both rigorously 
evaluated behavioral health interventions and those interventions that 
have been implemented, demonstrate promise, but have not necessarily 
been evaluated in a rigorous manner. To that end, the Learning Center 
is being significantly revamped to support stakeholder engagement and 
to become a shared learning environment for all stakeholders. SAMHSA 
recognizes that the successful promotion and dissemination of evidence-
based programs and practices requires an environment that promotes 
community assessment, program and practice planning and evaluation, as 
well as guidance on the selection and implementation of programs and 
practices listed on NREPP. There are useful types of evaluation 
research, often conducted among underserved populations, which provide 
valuable insights for practitioners, but do not meet the minimum 
criteria required for experimental or quasi-experimental design. SAMHSA 
intends the Learning Center to be a forum for presenting research on 
emerging programs and practices, and exploring ways that pre-
experimental and qualitative research can complement and enrich 
findings from experimental and quasi-experimental research designs. An 
inventory of such programs and practices will be compiled and 
maintained within the Learning Center and will operate in parallel to 
the listing of reviewed programs and practices with experimental and 
quasi-experimental designs. In this way, SAMHSA intends to support 
programs and practices researched with the most rigorous approaches 
while also supporting the development of practice-based evidence, 
especially for certain populations and emerging practices that are 
critical to learning and improving behavioral health outcomes for 
persons with or at risk of developing behavioral health issues.

Summer King,
Statistician.
[FR Doc. 2015-16573 Filed 7-6-15; 8:45 am]
BILLING CODE 4162-20-P



                                              38716                            Federal Register / Vol. 80, No. 129 / Tuesday, July 7, 2015 / Notices

                                              DEPARTMENT OF HEALTH AND                                 SUMMARY:    The mission of SAMHSA is to               necessarily all, of the evidence for a
                                              HUMAN SERVICES                                           reduce the impact of substance abuse                  program or practice has been reviewed.
                                                                                                       and mental illness on America’s                       In some cases, the quality of the
                                              Substance Abuse and Mental Health                        communities. Established in 1992, the                 research supporting the program or
                                              Services Administration                                  agency was directed by Congress to                    practice may have been determined to
                                                                                                       target effective substance abuse and                  be poor or insufficient to earn a rating.
                                              Center for Substance Abuse                               mental health services to the people                  Inclusion in NREPP does not constitute
                                              Treatment; Notice of Meeting                             most in need, and to translate research               endorsement of an intervention as
                                                 Pursuant to Public Law 92–463,                        in these areas more effectively and more              effective by SAMHSA. Moreover, since
                                              notice is hereby given that the                          rapidly into the general health care                  NREPP has not reviewed all behavioral
                                              Substance Abuse and Mental Health                        system. NREPP is a key public resource                health interventions, the use of NREPP
                                              Services Administration’s (SAMHSA’s)                     SAMHSA has developed to help meet                     as an exclusive or exhaustive list of
                                              Center for Substance Abuse Treatment                     this directive. This notice announces                 interventions is not appropriate.
                                              (CSAT) National Advisory Council will                    the redesign of NREPP to better align the             Policymakers and funders in particular
                                              meet on July 16, 2015, from 3:00 p.m.–                   registry with the standards and                       are discouraged from limiting providers
                                              4:00 p.m. (EDT). The meeting will be                     processes of other evidence-based                     and/or potential grantees to selecting
                                              closed to the public.                                    repositories. A re-launch of the Web site             exclusively from among NREPP
                                                 The meeting will include discussion                   with revised content is anticipated in                interventions and from funding NREPP
                                              and evaluation of grant applications                     late fall/early winter 2015.                          interventions regardless of the ratings
                                              reviewed by Initial Review Groups, and                      The notice explains the changes in                 the interventions receive.
                                              involve an examination of confidential                   how programs and practices will be                       This notice announces changes to (1)
                                              financial and business information as                    identified for NREPP, how submissions                 the process for identifying new
                                              well as personal information concerning                  will be screened and reviewed, and                    programs and practices for NREPP
                                              the applicants. Therefore, the meeting                   provides guidance on accessing updated                review, (2) the process for announcing
                                              will be closed to the public, as                         information on the NREPP site.                        open submission periods, (3) the
                                              determined by the SAMHSA                                 Potential applicants should be aware                  minimum requirements to be
                                              Administrator, in accordance with Title                  that this notice includes updated                     considered for NREPP review, and (4)
                                              5 U.S.C. 552b(c)(4) and (6) and (c)(9)(B)                information relating to the eligibility of            the review process. This notice also
                                              and 5 U.S.C. App. 2, Section 10(d).                      interventions for inclusion in NREPP                  announces the intent to re-review
                                                 The meeting will be held virtually.                   and changes in the program and practice               currently posted NREPP programs and
                                              Meeting information and a roster of                      review process that supersedes guidance               practices to comport with new review
                                              Council members may be obtained                          provided in earlier Federal Register                  criteria and ratings. The re-review of
                                              either by accessing the SAMHSA                           notices.                                              programs and practices currently posted
                                              Council Web site at: http://                             FOR FURTHER INFORMATION CONTACT:                      will take place over the course of the
                                              www.samhsa.gov/about-us/advisory-                        Carter Roeber, Ph.D., Social Science                  next several years, depending on
                                              councils/csat-national-advisory-council                  Analyst, Center for Behavioral Health                 available resources. A re-launch of the
                                              or by contacting LCDR Holly Berilla.                     Statistics and Quality, SAMHSA, 1                     NREPP Web site will take place in
                                                                                                       Choke Cherry Road, Room 2–1050,                       phases and the first phase is planned for
                                                Council Name: SAMHSA’s Center for                                                                            late fall/early winter of 2015.
                                              Substance Abuse Treatment; National                      Rockville, MD 20857, telephone 240–
                                              Advisory Council.                                        276–1488.                                             Identifying New Programs and
                                                Date/Time/Type: July 16, 2015, 3:00 p.m.–              SUPPLEMENTARY INFORMATION:                            Practices for NREPP
                                              4:00 p.m. EDT, Closed.
                                                Place: Virtual—Teleconference.                         Advancing Evidence-Based Programs                        Open submissions periods, during
                                                Contact: LCDR Holly Berilla, Designated                and Practices Through Improved                        which applicants may submit materials
                                              Federal Official, CSAT National Advisory                 Decision Support Tools:                               for review, will continue to be used to
                                              Council, 1 Choke Cherry Road, Rockville,                 Reconceptualizing NREPP                               identify new programs and practices for
                                              Maryland 20857 (mail), Telephone: (240)                                                                        review (see below). Programs and
                                              276–1252, Fax: (240) 276–2252, Email:                    Introduction                                          practices addressing specific SAMHSA
                                              holly.berilla@samhsa.hhs.gov.                              SAMHSA’s NREPP is an evidence-                      priorities may also be identified by
                                              Summer King,                                             based repository and review system                    SAMHSA or through environmental
                                              Statistician, SAMHSA.                                    designed to provide the public with                   scans (including literature reviews,
                                              [FR Doc. 2015–16548 Filed 7–6–15; 8:45 am]               reliable information about behavioral                 focus groups, public input, and
                                              BILLING CODE 4162–20–P
                                                                                                       health interventions in the areas of                  interviews), as time and resources
                                                                                                       mental health and substance abuse.                    permit. Programs and practices related
                                                                                                       Programs and practices that are                       to priority areas may be reviewed before
                                              DEPARTMENT OF HEALTH AND                                 accepted for inclusion in NREPP                       programs and practices identified
                                              HUMAN SERVICES                                           undergo a review process that provides                through the open submission period.
                                                                                                       information on the quality of research                SAMHSA will be consulting with
                                              Substance Abuse and Mental Health                        and the magnitude and direction of                    subject matter experts and leadership in
                                              Services Administration                                  program or practice impact on                         underserved groups and populations,
                                                                                                       individual outcomes. Materials for                    including American Indian/Alaska
                                              National Registry of Evidence-Based                      dissemination are reviewed to                         Native Tribes, regarding ways to
tkelley on DSK3SPTVN1PROD with NOTICES




                                              Programs and Practices                                   determine the type and extent of                      incorporate traditional and culturally-
                                              AGENCY: Substance Abuse and Mental                       information available to support                      specific interventions into NREPP, in
                                              Health Services Administration                           implementation. The results of these                  order to better meet the needs of groups
                                              (SAMHSA), HHS.                                           reviews are published on the NREPP                    whose efforts to promote behavioral
                                                                                                       Web site (http://nrepp.samhsa.gov).                   health may not have not been routinely
                                              ACTION: Notice regarding SAMHSA’s
                                                                                                         It should be noted that inclusion in                evaluated. Innovative, but perhaps less
                                              NREPP: Redesign of NREPP.
                                                                                                       NREPP indicates that some, but not                    rigorously tested, programs and


                                         VerDate Sep<11>2014   20:31 Jul 06, 2015   Jkt 235001   PO 00000   Frm 00056   Fmt 4703   Sfmt 4703   E:\FR\FM\07JYN1.SGM   07JYN1


                                                                               Federal Register / Vol. 80, No. 129 / Tuesday, July 7, 2015 / Notices                                          38717

                                              practices will be considered if specific                 accepted after an initial screening to                compared, and studies in which the
                                              to SAMHSA priority areas, as a parallel                  ensure that the application meets                     effects of the same intervention on
                                              contribution to the standard review                      minimum requirements. The number of                   various subpopulations are compared or
                                              procedures.                                              reviews conducted will depend on the                  in which various doses or components
                                                 Stand-alone pharmacologic treatments                  availability of funds, with the timing of             of the same intervention are compared
                                              are not eligible for review and should                   reviews to be determined by SAMHSA.                   will be reviewed, but only for purposes
                                              not be submitted to NREPP. The                           In submitting an intervention,                        of providing readers in the Learning
                                              evidence base for pharmacologic                          applicants should understand that if                  Center the opportunity to determine
                                              treatments is reviewed and approved                      interventions are selected for review,                whether a particular program or practice
                                              through the U.S. Food and Drug                           the results of NREPP reviews are                      may have use or show promise in
                                              Administration (FDA). FDA approved                       considered public information and will                different settings or circumstances.
                                              pharmacotherapy interventions (on-                       be posted on the NREPP Web site. Once                    3. The results of these studies have
                                              label use) are considered for NREPP                      a review is completed, the applicant                  been published in a peer-reviewed
                                              review only when combined with one or                    will be provided with a summary                       journal or other professional
                                              more behavioral or psychosocial                          document (‘‘the program or practice                   publication, or documented in a
                                              treatments.                                              profile’’) that presents results of the
                                                 SAMHSA reserves the right to reject                                                                         comprehensive evaluation report,
                                                                                                       review, ratings of program effectiveness,
                                              for review programs and practices                                                                              published within the previous 25 years.
                                                                                                       and descriptive information about the
                                              whose goals or activities are determined                                                                       Comprehensive evaluation reports must
                                                                                                       intervention. The applicant will have
                                              to be inconsistent with SAMHSA’s                                                                               include a review of the literature,
                                                                                                       the opportunity to comment on the
                                              mission, which is ‘‘to reduce the impact                                                                       theoretical framework, purpose,
                                                                                                       profile before it is posted but they will
                                              of substance abuse and mental illness                                                                          methodology, findings/results with
                                                                                                       not have the option to refuse posting.
                                              on American communities.’’                                                                                     statistical analysis and p values for
                                                                                                       Minimum Requirements                                  significant outcomes, discussion, and
                                              Open Submission Periods                                                                                        conclusions.
                                                                                                          To be considered for review,
                                                SAMHSA accepts new applications                        interventions must meet three minimum                    Applicants are required to provide
                                              for review during open submission                        requirements:                                         full-text documents at the time of
                                              periods. SAMHSA generally holds one                         1. Research or evaluation of the                   submission that demonstrate the
                                              open submission period a year but,                       intervention has assessed mental health               intervention meets these minimum
                                              depending on the number of reviews in                    or substance use outcomes among                       requirements. Other research articles,
                                              progress and resources available may                     individuals, communities, or                          published or unpublished evaluation
                                              hold more or fewer within a calendar                     populations OR other behavioral health-               reports, grant final reports, and
                                              year. All future open submission                         related outcomes on individuals,                      replication studies may be submitted as
                                              periods will be announced on the                         communities, or populations with or at                additional supporting documentation.
                                              NREPP Web site. Emails will also be                      risk of mental health issues or substance             Note: Abstracts or links to partial
                                              sent announcing the open submission                      use problems.                                         articles are regarded as incomplete and
                                              period to those on the NREPP listserv.                      2. Evidence of these outcomes has                  will not be considered.
                                              Anyone wishing to be notified of future                  been demonstrated in at least one study
                                                                                                       using an experimental or quasi-                          NREPP will no longer require
                                              open submission periods can join the
                                                                                                       experimental design. Experimental                     programs and practices to have
                                              NREPP listserv by sending a request to
                                                                                                       designs require random assignment, a                  developed implementation materials,
                                              nrepp@samhsa.hhs.gov.
                                                Applications can be submitted at any                   control or comparison group, and pre-                 training and support resources, and
                                              point during an open period. Program                     and post-intervention outcome                         quality assurance procedures. However,
                                              and practice developers, researchers,                    assessments. Quasi-experimental                       programs and practices with such
                                              and others interested in submitting an                   designs do not require random                         dissemination and/or implementation
                                              intervention should read below for                       assignment, but do require a comparison               resources will be considered for
                                              information about the new minimum                        or control group and pre- and post-                   prioritized review if within the
                                              requirements to be considered for an                     intervention outcome assessments.                     priorities established by the SAMHSA
                                              NREPP review. Additional future                          Comparison/control groups must be a                   review process, and the materials, along
                                              changes to the review process and                        no-treatment control group, a wait-list               with the location of their availability,
                                              criteria will be posted on the NREPP                     control group, a treatment-as-usual                   will be listed in the program or practice
                                              Web site as they are implemented.                        comparison group, or an intervention                  profile.
                                              Therefore, before submitting a program                   that is presumed to be ineffective or                    Applicants submitting dissemination
                                              or practice for NREPP review,                            substantially less effective than the                 and implementation resources should
                                              applicants should examine the most                       intervention (e.g., a ‘‘placebo’’ control             include a brief narrative description of
                                              recent information posted on the NREPP                   or, in cases in which providing no                    the materials that are being submitted.
                                              Web site about the review process and                    treatment might be considered                         These materials may include, but are
                                              criteria and the most recent guidance for                unethical, less effective treatments, even            not limited to, treatment manuals,
                                              preparing an intervention for                            if not treatment-as-usual, such as                    information for administrators,
                                              submission (see http://                                  ‘‘supportive therapy’’). Studies with                 information for direct service staff,
                                              www.nrepp.samhsa.gov/                                    single-group, pretest-posttest designs or             tested training curricula, mechanisms
                                              ReviewSubmission.aspx). This guidance                    single-group, longitudinal/multiple time              for ongoing supervision and
tkelley on DSK3SPTVN1PROD with NOTICES




                                              will be periodically updated to reflect                  series do not meet this requirement, but              consultation, protocols for gathering
                                              the redesign of NREPP during the re-                     will be considered to identify emerging               process and outcome data, ongoing
                                              launch.                                                  programs and practices for                            monitoring of intervention fidelity, and
                                                The selection of interventions will                    consideration in the Learning Center.                 processes for gathering feedback.
                                              take place after the closing of the open                 Comparative effectiveness trials, in                  Applicants should also provide the
                                              submission period, and applicants will                   which two interventions, both                         location of where the materials can be
                                              be notified whether they have been                       presumed to be equally effective, are                 obtained.


                                         VerDate Sep<11>2014   20:31 Jul 06, 2015   Jkt 235001   PO 00000   Frm 00057   Fmt 4703   Sfmt 4703   E:\FR\FM\07JYN1.SGM   07JYN1


                                              38718                            Federal Register / Vol. 80, No. 129 / Tuesday, July 7, 2015 / Notices

                                              Selection of Interventions for Review                    delivered as intended to the program’s                learning environment for all
                                                SAMHSA will select interventions for                   or practice’s target population).                     stakeholders. SAMHSA recognizes that
                                                                                                          The methodological rigor (that is,                 the successful promotion and
                                              review from among submissions
                                                                                                       internal validity, statistical validity, and          dissemination of evidence-based
                                              meeting the minimum requirements. In
                                                                                                       measurement validity) of the research                 programs and practices requires an
                                              selecting interventions for review,
                                                                                                       for each program or practice will be                  environment that promotes community
                                              SAMHSA may give special
                                                                                                       reviewed, as it pertains to each outcome              assessment, program and practice
                                              consideration to interventions that meet
                                                                                                       examined, along with the magnitude                    planning and evaluation, as well as
                                              one or more of the following conditions:
                                                                                                       and direction of the program’s or                     guidance on the selection and
                                                • More than one research study or
                                                                                                       practice’s effect on each outcome. Based              implementation of programs and
                                              evaluation has been conducted on the
                                                                                                       on this information, the program’s or                 practices listed on NREPP. There are
                                              same or a similar target population that
                                                                                                       practice’s effectiveness for each                     useful types of evaluation research,
                                              meets the minimum requirements.                          outcome will be rated, along with the                 often conducted among underserved
                                                • The intervention targets
                                                                                                       rigor of the research examining the                   populations, which provide valuable
                                              underserved populations (e.g., minority                  program or practice, and the ratings will             insights for practitioners, but do not
                                              populations, tribal communities or                       be displayed on the NREPP Web site.                   meet the minimum criteria required for
                                              American Indian/Alaska Native                               In general, each NREPP evidence                    experimental or quasi-experimental
                                              populations, elderly individuals, young                  review will be conducted by two trained               design. SAMHSA intends the Learning
                                              adults, individuals who are                              and certified reviewers. However, based               Center to be a forum for presenting
                                              incarcerated, etc.).                                     on funding and available resources,                   research on emerging programs and
                                                • Dissemination and implementation                     SAMHSA use one reviewer for programs                  practices, and exploring ways that pre-
                                              materials (e.g., program or practice                     and practices being re-reviewed. When                 experimental and qualitative research
                                              manuals, training guides, measurement                    necessary, NREPP may conduct author                   can complement and enrich findings
                                              instruments, implementation fidelity                     queries to confirm or gather additional               from experimental and quasi-
                                              tools) are available. Lower costs and no-                information needed for the review.                    experimental research designs. An
                                              cost materials may be prioritized.                       Program and practice profiles will be                 inventory of such programs and
                                                • The intervention contributes to a                    developed on the basis of the                         practices will be compiled and
                                              content area in which few evidence-                      information gathered. Applicants will                 maintained within the Learning Center
                                              based interventions have been                            have the opportunity to review the                    and will operate in parallel to the listing
                                              previously identified.                                   program or practice profile before it is              of reviewed programs and practices
                                                Interventions that are not selected for                posted on the NREPP site, but they will               with experimental and quasi-
                                              review may be resubmitted by the                         not have the option to refuse posting.                experimental designs. In this way,
                                              applicant in a future open submission                       Dissemination and implementation                   SAMHSA intends to support programs
                                              period.                                                  materials will no longer be rated as they             and practices researched with the most
                                              The Review Process                                       were historically. Instead, descriptions              rigorous approaches while also
                                                                                                       of available materials for each program               supporting the development of practice-
                                                 The review process has been revised                   or practice, highlighting information                 based evidence, especially for certain
                                              to improve the quality of the reviews                    that may be of most interest to NREPP                 populations and emerging practices that
                                              and utility of information that NREPP                    users, will be included in the program                are critical to learning and improving
                                              can provide its users. In addition to                    or practice profile, along with                       behavioral health outcomes for persons
                                              articles and reports submitted by NREPP                  information documenting the extent to                 with or at risk of developing behavioral
                                              applicants, additional studies, articles,                which materials are available.                        health issues.
                                              and evaluation reports regarding the                        Programs and practices currently
                                              interventions will be identified through                                                                       Summer King,
                                                                                                       posted on NREPP will be re-reviewed as
                                              literature searches. Studies and                         time and resources permit but the re-                 Statistician.
                                              outcomes to be reviewed will be                          reviews of currently posted programs                  [FR Doc. 2015–16573 Filed 7–6–15; 8:45 am]
                                              determined through the systematic                        and practices will take place over the                BILLING CODE 4162–20–P
                                              application of standardized screening                    next few years.
                                              criteria, and the number of studies and                     Detailed information about the
                                              outcomes to be reviewed will be                          revised review process will be available              DEPARTMENT OF HOMELAND
                                              expanded to more comprehensively                         at http://www.nrepp.samhsa.gov after                  SECURITY
                                              represent the evidence base for the                      the re-launch of the new NREPP Web
                                              program or practice. Inclusion of studies                site.                                                 U.S. Customs and Border Protection
                                              and outcomes will no longer be limited
                                              to positive significant outcomes; all                    Enhancing the Learning Center                         Notice of Issuance of Final
                                              studies and outcomes that meet the                          NREPP’s Learning Center is a                       Determination Concerning Wound
                                              standardized screening criteria will be                  developing and underutilized                          Therapy System
                                              reviewed, including those with negative                  component of the NREPP Web site. With                 AGENCY:  U.S. Customs and Border
                                              and non-significant effects. Programs                    the evolution and enhancement of the                  Protection, Department of Homeland
                                              and practices will be assessed on the                    registry, SAMHSA seeks to bring greater               Security.
                                              basis of evaluation studies of program or                recognition to both rigorously evaluated              ACTION: Notice of final determination.
                                              practice impact, information related to                  behavioral health interventions and
tkelley on DSK3SPTVN1PROD with NOTICES




                                              conceptual framework (that is, program                   those interventions that have been                    SUMMARY:   This document provides
                                              or practice goals, theory of change, and                 implemented, demonstrate promise, but                 notice that U.S. Customs and Border
                                              program or practice components), and                     have not necessarily been evaluated in                Protection (‘‘CBP’’) has issued a final
                                              information about implementation                         a rigorous manner. To that end, the                   determination concerning the country of
                                              fidelity (that is, whether a study                       Learning Center is being significantly                origin of the PICO single use negative
                                              employs quality assurance measures to                    revamped to support stakeholder                       pressure wound therapy system
                                              declare that the program or practice is                  engagement and to become a shared                     manufactured and distributed by Smith


                                         VerDate Sep<11>2014   20:31 Jul 06, 2015   Jkt 235001   PO 00000   Frm 00058   Fmt 4703   Sfmt 4703   E:\FR\FM\07JYN1.SGM   07JYN1



Document Created: 2015-12-15 13:27:47
Document Modified: 2015-12-15 13:27:47
CategoryRegulatory Information
CollectionFederal Register
sudoc ClassAE 2.7:
GS 4.107:
AE 2.106:
PublisherOffice of the Federal Register, National Archives and Records Administration
SectionNotices
ActionNotice regarding SAMHSA's NREPP: Redesign of NREPP.
ContactCarter Roeber, Ph.D., Social Science Analyst, Center for Behavioral Health Statistics and Quality, SAMHSA, 1 Choke Cherry Road, Room 2-1050, Rockville, MD 20857, telephone 240-276- 1488.
FR Citation80 FR 38716 

2025 Federal Register | Disclaimer | Privacy Policy
USC | CFR | eCFR