83 FR 29122 - Decisions Related to the Development of a Clearinghouse of Evidence-Based Practices in Accordance With the Family First Prevention Services Act of 2018
DEPARTMENT OF HEALTH AND HUMAN SERVICES
Administration for Children and Families Federal Register Volume 83, Issue 121 (June 22, 2018)
Administration for Children and Families
Federal Register Volume 83, Issue 121 (June 22, 2018)
| Page Range | 29122-29124 | |
| FR Document | 2018-13420 |
[Federal Register Volume 83, Number 121 (Friday, June 22, 2018)] [Notices] [Pages 29122-29124] From the Federal Register Online [www.thefederalregister.org] [FR Doc No: 2018-13420] ----------------------------------------------------------------------- DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children and Families Decisions Related to the Development of a Clearinghouse of Evidence-Based Practices in Accordance With the Family First Prevention Services Act of 2018 AGENCY: Administration for Children and Families, HHS. ACTION: Request for public comment. ----------------------------------------------------------------------- SUMMARY: The Administration for Children and Families, HHS, solicits comments by July 22, 2018 on initial criteria and potential candidate programs and services for review in a Clearinghouse of evidence-based practices in accordance with the Family First Prevention Services Act of 2018. The Clearinghouse will identify promising, supported, and well-supported practices for mental health and substance abuse prevention and treatment programs, in-home parent skill-based programs, and kinship navigator programs appropriate for children who are candidates for foster care pregnant or parenting foster youth, and the parents or kin caregivers of those children and youth. SUPPLEMENTARY INFORMATION: Invitation to Comment: HHS invites comments regarding this Notice. To ensure that your comments have maximum effect, please identify clearly the section of this Notice that your comment addresses. 1.0 Background and Legislative Context The Family First Prevention Services Act (FFPSA) was signed into law as part of the Bipartisan Budget Act (H.R. 1892) on February 9, 2018. FFPSA enables States to use Federal funds available under parts B and E of title IV of the Social Security Act to provide enhanced support to children and families and prevent foster care placements through the provision of evidence-based mental health and substance abuse prevention and treatment services, in-home parent skill-based programs, and kinship navigator services. As described in the statutory language, these services and programs are intended ``for children who are candidates for foster care or who are pregnant or parenting foster youth and the parents or kin caregivers of the children''. FFPSA requires an independent systematic review of evidence to designate programs and services as ``promising,'' ``supported,'' and ``well-supported'' practices, defined as follows in section 471(e)(4)(C):Promising Practice: ``A practice shall be considered to be a `promising practice' if the practice is superior to an appropriate comparison practice using conventional standards of statistical significance (in terms of demonstrated meaningful improvements in validated measures of important child and parent outcomes, such as mental health, substance abuse, and child safety and well-being), as established by the results or outcomes of at least one study that--(1) was rated by an independent systematic review for the quality of the study design and execution and determined to be well-designed and well-executed; and (2) utilized some form of control (such as an untreated group, a placebo group, or a wait list study).'' Supported Practice: ``A practice shall be considered to be a `supported practice' if (I) the practice is superior to an appropriate comparison practice using conventional standards of statistical significance (in terms of demonstrated meaningful improvements in validated measures of important child and parent outcomes, such as mental health, substance abuse, and child safety and well-being), as established by the results or outcomes of at least one study that--(aa) was rated by an independent systematic review for the quality of the study design and execution and determined to be well-designed and well-executed; and (bb) was a rigorous random-controlled trial (or, if not available, a study using a rigorous quasi-experimental research design); and (cc) was carried out in a usual care or practice setting and (II) the study described in sub-clause (I) established that the practice has a sustained effect (when compared to a control group) for at least 6 months beyond the end of treatment.'' Well-supported Practice: ``A practice shall be considered to be a `well-supported practice' if (I) the practice is superior to an appropriate comparison practice using conventional standards of statistical significance (in terms of demonstrated meaningful improvements in validated measures of important child and parent outcomes, such as mental health, substance abuse, and child safety and well-being), as established by the results or outcomes of at least two studies that--(aa) were rated by an independent systematic review for the quality of the study design and execution and determined to be well-designed and well-executed; and (bb) were rigorous random-controlled trials (or, if not available, studies using a rigorous quasi-experimental research design); and (cc) were carried out in a usual care or practice setting and (II) at least one of the studies described in sub-clause (I) established that the practice has a sustained effect (when compared to a control group) for at least 1 year beyond the end of treatment.'' In accordance with FFPSA, practices must also meet the following requirements: Book or manual: The practice has a book, manual, or other available writings that specify the components of the practice protocol and describe how to administer the practice. No empirical risk of harm: There is no empirical basis suggesting that, compared to its likely benefits, the practice constitutes a risk of harm to those receiving it. Weight of evidence supports benefits: If multiple outcome studies have been conducted, the overall weight of evidence supports the benefits of the practice. Reliable and valid outcome measures: Outcome measures are reliable and valid, and are administrated consistently and accurately across all those receiving the practice. No case data for severe or frequent risk of harm: There is no case data suggesting a risk of harm that was probably caused by the treatment and that was severe or frequent (section 471(e)(4)(C)(ii) of the Act). In order to meet these requirements, the Administration for Children and Families (ACF) in the Department of Health and Human Services (HHS) intends to establish and maintain a public Clearinghouse of practices, including culturally specific, or location- or population-based adaptations of practices, identified via a systematic review of evidence on relevant programs and services. In accordance with FFPSA and building from other federal evidence reviews, HHS is developing initial criteria that will be used to designate programs and services as promising, supported, and well-supported practices. HHS will also identify a preliminary list of candidate services and programs that will be considered for systematic review. This Notice (1) identifies and requests comment on potential initial criteria for (a) identifying eligible programs and services for review by the Clearinghouse, (b) prioritizing eligible programs and services for review, (c) identifying eligible studies aligned with prioritized programs and services, (d) prioritizing eligible studies for rating, (e) rating studies, and (f) rating programs and services as promising, supported, and well-supported practices. This Notice (2) requests comment on potential programs and services that may meet the aforementioned criteria and that should be considered as candidates for systematic review. After comments are received, HHS will revise and publish the initial criteria and a preliminary list of candidate programs and services to be considered for review; and begin to conduct reviews. This Notice is one step in ensuring that activities associated with the development of a Clearinghouse are [[Page 29123]] transparent and build from the existing knowledge of States, federal agencies, researchers, evaluators, program and service developers, key stakeholders and experts, and the general public. 2.0 Initial Criteria 2.1 Program or Service Eligibility Criteria. Programs or services may be eligible for inclusion in the Clearinghouse if they meet the following criteria developed in accordance with FFPSA statutory language [sections 471(e)(1)(B) and 471(e)(1)(c)]: 2.1.1 Types of Programs and Services. HHS intends to limit eligibility to mental health and substance abuse prevention and treatment services, in-home parent skill-based programs (including parenting skills training, parent education, and individual and family counseling), and kinship navigator programs. 2.1.2 Book/Manual/Writings Available. HHS intends to limit eligibility to programs or services that have a book, manual, or other available documentation that specifies the components of the practice protocol and describes how to administer the practice. 2.2 Program or Service Prioritization Criteria. Timing and resources may not allow for a detailed review of all programs and services determined to be eligible by the criteria detailed in section 2.1 Program or Service Eligibility Criteria. Programs or services may be prioritized for review on the basis of the following criteria: 2.2.1 Types of Programs and Services. As noted in 2.1.1. Types of Programs and Services, HHS intends to limit eligibility to mental health and substance abuse prevention and treatment services, in- home parent skill-based programs (including parenting skills training, parent education, and individual and family counseling), or kinship navigator programs. This Notice requests comment on the scope of programs and services and topic areas of interest within the aforementioned categories that should be prioritized for inclusion. 2.2.2 Target Population of Interest. HHS intends to prioritize programs or services for review that have been developed or used to target children and families involved in the child welfare system or populations similar to those involved in the child welfare system. This Notice requests comment on populations that may be considered ``similar'' to those involved in the child welfare system. 2.2.3 Target Outcomes. HHS intends to prioritize programs or services for review that aim to impact target outcomes. Target outcomes should be defined in accordance with FFPSA statutory language [section 471(e)(4)(C)] and include those outcomes that ``. . . prevent child abuse and neglect, and reduce the likelihood of foster care placement by supporting birth families and kinship families and improving targeted supports for pregnant and parenting youth and their children.'' These may include, but are not limited to, ``. . . important child and parent outcomes, such as mental health, substance abuse, and child safety and well-being.'' This Notice requests comment on which types of mental health, substance abuse, and child and family outcomes should be considered as `target outcomes' and requests research evidence to support recommendations of `target outcomes'. HHS does not intend to include access to service, satisfaction with programs and services, and referral to programs and services as `target outcomes'. 2.2.4 Number of Impact Studies. HHS intends to prioritize programs or services with at least two studies with non-overlapping analytic samples and distinct implementations examining effectiveness/impact. 2.2.5 In Use/Active. HHS intends to prioritize programs or services currently in use in the U.S. Programs or services that are no longer in operation or have no information available about active implementation will not be prioritized. 2.2.6 Implementation and Fidelity Support. HHS intends to prioritize programs or services that have implementation training and staff support and/or fidelity monitoring tools and resources available to implementers in the United States. 2.2.7 Trauma-Informed. HHS may also prioritize services and programs that have been implemented using a trauma-informed approach. FFPSA statutory language [section 471(e)(4)(B)] states, ``The services or programs to be provided to or on behalf of a child are provided under an organizational structure and treatment framework that involves understanding, recognizing, and responding to the effects of all types of trauma and in accordance with recognized principles of a trauma-informed approach and trauma- specific interventions to address trauma's consequences and facilitate healing.'' This Notice requests comment on the feasibility of prioritizing programs and services based on past implementation in accordance with trauma-informed principles. 2.2.8 Delivery Setting for In-Home Parent Skill-Based Programs and Services. HHS intends to prioritize in-home parent skill-based programs and services where the primary service delivery strategy takes place in the caregivers' place of residence. 2.3 Study Eligibility Criteria. HHS intends to engage in a broad literature search to identify studies examining prioritized programs and services. This search may include databases, websites, existing literature reviews, and meta-analyses. HHS intends to screen studies for eligibility using the following criteria: 2.3.1 Impact Study. HHS intends to limit eligibility to studies included in government reports and peer-reviewed journal articles that assess effectiveness (i.e., impact) using quantitative methods. 2.3.2 Target Outcomes. HHS intends to limit eligibility to studies that examine the impact of the service or program on at least one `target outcome', as described in section 2.2.3. Target Outcomes. As noted above, this Notice requests comment on specific outcomes in accordance with FFPSA statutory language that should be considered `target outcomes' and requests research evidence to support recommendations of `target outcomes'. 2.3.3 Conducted in the U.S., U.K., Canada, New Zealand, or Australia. HHS intends to limit eligibility to studies conducted with samples in the U.S., U.K., Canada, New Zealand, or Australia to ensure that the evidence base reflects the populations where programs and services will be implemented. 2.3.4 Study Published in English. HHS intends to limit eligibility to studies published in English. 2.3.5 Published or Prepared in or after 1990. HHS intends to limit eligibility to studies published or prepared in or after 1990. 2.3.6 Usual Care or Practice Setting. HHS intends to limit eligibility to studies carried out in a usual care or practice setting in accordance with FFPSA [section 471(e)(4)(C)]. This Notice requests comment on the definition of usual care or practice settings. 2.4 Study Prioritization Criteria. Timing and resources may not allow for a detailed rating of all studies determined to be eligible by the criteria identified in section 2.3 Study Eligibility Criteria. HHS intends to conduct a high-level scan of eligible studies to determine which should be prioritized for rating. This Notice requests comment on criteria that can be used to prioritize eligible studies for rating. 2.4.1 Implementation Period: FFPSA [section 471(e)(1)(A) and (B)] states that the Secretary may make a payment to a State for providing services or programs ``for not more than a 12-month period''. This Notice requests comment on whether studies with program or service implementation periods of longer than 12 months should be considered for review and if so, whether any other implementation period cutoff should be included as a study prioritization criterion. 2.4.2 Sample of Interest. HHS intends to prioritize studies that include samples of children and families involved in the child welfare system or populations similar to those involved in the child welfare system. This Notice requests comment on populations that may be considered ``similar'' to those involved in the child welfare system. 2.5 Study Rating Criteria. HHS intends to rate studies on the following criteria: 2.5.1 Favorable Effects. HHS intends to rate studies based on whether they demonstrate at least one meaningful favorable effect (i.e., positive significant effect) on a `target outcome' as specified in section 2.3.2 Target Outcomes. A meaningful effect will be defined using conventional standards of statistical significance (i.e., two-tailed hypothesis test and a specified alpha level of p < .05). This Notice requests comment on whether and how ratings should consider the number or magnitude of favorable effects. 2.5.2 Unfavorable Effects. HHS intends to rate studies based on the number of unfavorable effects (i.e., negative significant effects) on either `target' or non-target outcomes as specified in section 2.3.2 Target Outcomes. Effects will be defined using conventional standards of statistical significance (i.e., two-tailed hypothesis test and a specified alpha level of p < .05). This [[Page 29124]] Notice requests comment on whether and how studies should also be rated on the number of null effects on `target outcomes', and on whether and how ratings should consider the number or magnitude of unfavorable effects. 2.5.3 Sustained Favorable Effect. HHS intends for studies with at least one favorable effect on a `target outcome', as determined by the criteria in 2.5.1 Favorable Effects, to be rated on whether or not they demonstrate a sustained favorable effect. As noted in section 471(e)(4)(C), a `supported practice' must have at least one study that demonstrates ``a sustained effect (when compared to a control group) for at least 6 months beyond the end of treatment'' and a `well-supported practice' must have at least one study that demonstrates ``a sustained effect (when compared to a control group) for at least 1 year beyond the end of treatment.'' HHS intends to classify studies as not demonstrating a sustained favorable effect (i.e., effects are demonstrated for less than 6 months), demonstrating a sustained favorable effect of 6 months or more (but less than 12 months), or demonstrating a sustained favorable effect of 12 months or more. 2.5.4 Rigorous Study Design. HHS intends to rate studies as either high, moderate, or low on the rigor and appropriateness of their study design. Study designs that receive the highest rating will be either Randomized Controlled Trials (RCTs) or rigorous quasi-experimental designs. HHS defines randomized controlled trials as a study design in which sample members are assigned to the program or service and comparison groups by chance. Randomized control designs are often considered the ``gold standard'' of research design because personal characteristics (before the program or service begins) do not affect whether someone is assigned to the program or service or control group. HHS defines a quasi- experimental design as a study design in which sample members are selected for the program or service and comparison groups in a nonrandom way. Similar to criteria considered in other federal evidence clearinghouses, rigorous study designs will be those that are appropriately powered, include an appropriate control group, maintain original assignment to study arms, and are appropriate to combat threats to internal validity. This Notice requests comment on threats to internal validity that should be considered. This Notice requests comment on appropriate thresholds for evaluating and assigning a rating to a study design. 2.5.5 Rigorous Study Analysis. HHS intends to rate studies as either high, moderate, or low on the rigor and appropriateness of their analysis. Study analyses that receive the highest rating may be those that tested and established baseline equivalence, appropriately accounted for overall and differential sample attrition, appropriately accounted for multiple comparisons, and when necessary accounted for clustering. This Notice requests comment on appropriate thresholds for evaluating and assigning a rating to a study analysis. 2.5.6 Reliability, Validity, and Systematic Administration of Outcome Measures. HHS intends to rate studies as either high, moderate, or low on the extent to which `target outcome' measures are reliable (i.e., the extent to which a measure produces the same results when used repeatedly), valid (i.e., the extent to which a measure captures what it is intended to measure), and were administered consistently and accurately across all those receiving the practice in accordance with FFPSA statutory language [section 471(e)(4)(C)] or receiving the appropriate comparison practice. This Notice requests comment on appropriate thresholds for evaluating and assigning a rating to the reliability, validity, and administration of `target outcome' measures. 2.6 Program or Service Rating Criteria. HHS intends for programs or services to be rated as promising, supported, or well-supported practices if they meet the below criteria that collectively assess the strength of evidence for a practice and build from the individual study criteria described in section 2.5 Study Rating Criteria. These criteria were developed in accordance with FFPSA statutory language [section 471(e)(4)(C)]. 2.6.1 Promising Practice: HHS intends to designate a program or service as a `promising practice' if the program or service has at least one study that demonstrates a favorable effect on a target outcome as described by criterion 2.5.1 Favorable Effects and achieves, at a minimum, moderate ratings on criteria 2.5.4 through 2.5.6. 2.6.2 Supported Practice: HHS intends to designate a program or service as a `supported practice' if the program or service has at least one study that demonstrates a favorable effect on a target outcome as described by 2.5.1 Favorable Effects, demonstrates a sustained favorable effect on a target outcome of at least 6 months beyond the end of treatment as described in Section 2.5.3 Sustained Favorable Effect, and achieves the high rating on criteria 2.5.4 through 2.5.6. 2.6.3 Well-Supported Practice: HHS intends to designate a program or service as a `well-supported practice' if the practice has at least two studies with non-overlapping analytic samples and distinct implementations that demonstrate favorable effects as described by 2.5.1 Favorable Effects, demonstrate sustained favorable effects of at least 12 months beyond the end of treatment as described in Section 2.5.3 Sustained Favorable Effect, and achieve the high rating on criteria 2.5.4 through 2.5.6. HHS does not intend to rate a program or service as a `promising', `supported', or `well-supported practice' if there is an empirical basis, as evidenced by multiple unfavorable effects on target or non- target outcomes across reviewed studies, as described in 2.5.2 Unfavorable Effects, that suggest the overall weight of evidence does not support the benefits of the program or service. This Notice requests comment on approaches for determining that promising, supported, and well-supported practices do not constitute a risk of harm. As described in FFPSA [section 471(e)(4)(C)], ``There is no empirical basis suggesting that, compared to its likely benefits, the practice constitutes a risk of harm to those receiving it'', ``If multiple outcome studies are conducted, the overall weight of evidences supports the benefits of the practice'', and ``There is no case data suggesting a risk of harm that was probably caused by the treatment and that was severe or frequent''. 3.0 Recommendations of Potential Candidate Programs and Services for Review This Notice requests comment on potential candidate programs and services to consider for the systematic evidence review. Comments should identify how recommended programs and services meet the criteria described in section 2.1 Program or Service Eligibility Criteria. These criteria include: Types of Programs and Services and Book/Manual/ Writings Available. Comments should also identify how recommended programs and services meet the criteria described in section 2.2 Program or Service Prioritization Criteria. These criteria include: Types of Programs and Services, Target Population of Interest, Target Outcomes, Number of Impact Studies, In Use/Active, Implementation and Fidelity Support, Trauma-Informed, and Delivery Setting for In-Home Parent Skill-Based Programs and Services. In order to leverage new insights from the field, HHS may put forth additional future Notices requesting recommendations of potential candidate programs and services for review. 4.0 Submission of Comments Comments may be submitted until July 22, 2018 by email to [email protected]. Naomi Goldstein, Deputy Assistant Secretary for Planning, Research, and Evaluation. [FR Doc. 2018-13420 Filed 6-21-18; 8:45 am] BILLING CODE 4184-25-P
![29122 Federal Register / Vol. 83, No. 121 / Friday, June 22, 2018 / Notices
Dated: June 19, 2018. who are pregnant or parenting foster In accordance with FFPSA, practices
Seema Verma, youth and the parents or kin caregivers must also meet the following
Administrator, Centers for Medicare & of the children’’. requirements:
Medicaid Services. FFPSA requires an independent • Book or manual: The practice has a
[FR Doc. 2018–13503 Filed 6–20–18; 8:45 am] systematic review of evidence to book, manual, or other available writings that
BILLING CODE 4120–01–P designate programs and services as specify the components of the practice
‘‘promising,’’ ‘‘supported,’’ and ‘‘well- protocol and describe how to administer the
practice.
supported’’ practices, defined as follows • No empirical risk of harm: There is no
DEPARTMENT OF HEALTH AND in section 471(e)(4)(C): empirical basis suggesting that, compared to
HUMAN SERVICES
• Promising Practice: ‘‘A practice shall be its likely benefits, the practice constitutes a
considered to be a ‘promising practice’ if the risk of harm to those receiving it.
Administration for Children and • Weight of evidence supports benefits: If
practice is superior to an appropriate
Families multiple outcome studies have been
comparison practice using conventional
standards of statistical significance (in terms conducted, the overall weight of evidence
Decisions Related to the Development supports the benefits of the practice.
of demonstrated meaningful improvements
of a Clearinghouse of Evidence-Based in validated measures of important child and • Reliable and valid outcome measures:
Practices in Accordance With the parent outcomes, such as mental health, Outcome measures are reliable and valid, and
Family First Prevention Services Act of substance abuse, and child safety and well- are administrated consistently and accurately
2018 being), as established by the results or across all those receiving the practice.
outcomes of at least one study that—(1) was • No case data for severe or frequent risk
AGENCY: Administration for Children rated by an independent systematic review of harm: There is no case data suggesting a
and Families, HHS. for the quality of the study design and risk of harm that was probably caused by the
ACTION: Request for public comment. execution and determined to be well- treatment and that was severe or frequent
designed and well-executed; and (2) utilized (section 471(e)(4)(C)(ii) of the Act).
SUMMARY: The Administration for some form of control (such as an untreated In order to meet these requirements,
Children and Families, HHS, solicits group, a placebo group, or a wait list study).’’ the Administration for Children and
comments by July 22, 2018 on initial • Supported Practice: ‘‘A practice shall be Families (ACF) in the Department of
criteria and potential candidate considered to be a ‘supported practice’ if (I)
the practice is superior to an appropriate
Health and Human Services (HHS)
programs and services for review in a intends to establish and maintain a
comparison practice using conventional
Clearinghouse of evidence-based standards of statistical significance (in terms public Clearinghouse of practices,
practices in accordance with the Family of demonstrated meaningful improvements including culturally specific, or
First Prevention Services Act of 2018. in validated measures of important child and location- or population-based
The Clearinghouse will identify parent outcomes, such as mental health, adaptations of practices, identified via a
promising, supported, and well- substance abuse, and child safety and well- systematic review of evidence on
supported practices for mental health being), as established by the results or relevant programs and services. In
and substance abuse prevention and outcomes of at least one study that—(aa) was accordance with FFPSA and building
treatment programs, in-home parent rated by an independent systematic review
for the quality of the study design and
from other federal evidence reviews,
skill-based programs, and kinship HHS is developing initial criteria that
execution and determined to be well-
navigator programs appropriate for designed and well-executed; and (bb) was a will be used to designate programs and
children who are candidates for foster rigorous random-controlled trial (or, if not services as promising, supported, and
care pregnant or parenting foster youth, available, a study using a rigorous quasi- well-supported practices. HHS will also
and the parents or kin caregivers of experimental research design); and (cc) was identify a preliminary list of candidate
those children and youth. carried out in a usual care or practice setting services and programs that will be
SUPPLEMENTARY INFORMATION: Invitation and (II) the study described in sub-clause (I) considered for systematic review.
established that the practice has a sustained This Notice (1) identifies and requests
to Comment: HHS invites comments
effect (when compared to a control group) for comment on potential initial criteria for
regarding this Notice. To ensure that at least 6 months beyond the end of
your comments have maximum effect, treatment.’’
(a) identifying eligible programs and
please identify clearly the section of this • Well-supported Practice: ‘‘A practice services for review by the
Notice that your comment addresses. shall be considered to be a ‘well-supported Clearinghouse, (b) prioritizing eligible
practice’ if (I) the practice is superior to an programs and services for review, (c)
1.0 Background and Legislative appropriate comparison practice using identifying eligible studies aligned with
Context conventional standards of statistical prioritized programs and services, (d)
The Family First Prevention Services significance (in terms of demonstrated prioritizing eligible studies for rating, (e)
Act (FFPSA) was signed into law as part meaningful improvements in validated rating studies, and (f) rating programs
of the Bipartisan Budget Act (H.R. 1892) measures of important child and parent and services as promising, supported,
outcomes, such as mental health, substance
on February 9, 2018. FFPSA enables abuse, and child safety and well-being), as
and well-supported practices. This
States to use Federal funds available established by the results or outcomes of at Notice (2) requests comment on
under parts B and E of title IV of the least two studies that—(aa) were rated by an potential programs and services that
Social Security Act to provide enhanced independent systematic review for the may meet the aforementioned criteria
support to children and families and quality of the study design and execution and and that should be considered as
prevent foster care placements through determined to be well-designed and well- candidates for systematic review. After
the provision of evidence-based mental executed; and (bb) were rigorous random- comments are received, HHS will revise
daltland on DSKBBV9HB2PROD with NOTICES
health and substance abuse prevention controlled trials (or, if not available, studies and publish the initial criteria and a
and treatment services, in-home parent using a rigorous quasi-experimental research preliminary list of candidate programs
design); and (cc) were carried out in a usual
skill-based programs, and kinship care or practice setting and (II) at least one
and services to be considered for
navigator services. As described in the of the studies described in sub-clause (I) review; and begin to conduct reviews.
statutory language, these services and established that the practice has a sustained This Notice is one step in ensuring that
programs are intended ‘‘for children effect (when compared to a control group) for activities associated with the
who are candidates for foster care or at least 1 year beyond the end of treatment.’’ development of a Clearinghouse are
VerDate Sep<11>2014 17:16 Jun 21, 2018 Jkt 244001 PO 00000 Frm 00035 Fmt 4703 Sfmt 4703 E:\FR\FM\22JNN1.SGM 22JNN1](https://thefederalregister.org/doc/83_FR_29243/page/1.svg)
![Federal Register / Vol. 83, No. 121 / Friday, June 22, 2018 / Notices 29123
transparent and build from the existing being.’’ This Notice requests comment on 2.3.3 Conducted in the U.S., U.K.,
knowledge of States, federal agencies, which types of mental health, substance Canada, New Zealand, or Australia. HHS
researchers, evaluators, program and abuse, and child and family outcomes should intends to limit eligibility to studies
be considered as ‘target outcomes’ and conducted with samples in the U.S., U.K.,
service developers, key stakeholders
requests research evidence to support Canada, New Zealand, or Australia to ensure
and experts, and the general public. recommendations of ‘target outcomes’. HHS that the evidence base reflects the
2.0 Initial Criteria does not intend to include access to service, populations where programs and services
satisfaction with programs and services, and will be implemented.
2.1 Program or Service Eligibility referral to programs and services as ‘target 2.3.4 Study Published in English. HHS
Criteria. Programs or services may be outcomes’. intends to limit eligibility to studies
eligible for inclusion in the 2.2.4 Number of Impact Studies. HHS published in English.
Clearinghouse if they meet the following intends to prioritize programs or services 2.3.5 Published or Prepared in or after
criteria developed in accordance with with at least two studies with non- 1990. HHS intends to limit eligibility to
FFPSA statutory language [sections overlapping analytic samples and distinct studies published or prepared in or after
implementations examining effectiveness/ 1990.
471(e)(1)(B) and 471(e)(1)(c)]: impact. 2.3.6 Usual Care or Practice Setting. HHS
2.1.1 Types of Programs and Services. 2.2.5 In Use/Active. HHS intends to intends to limit eligibility to studies carried
HHS intends to limit eligibility to mental prioritize programs or services currently in out in a usual care or practice setting in
health and substance abuse prevention and use in the U.S. Programs or services that are accordance with FFPSA [section
treatment services, in-home parent skill- no longer in operation or have no 471(e)(4)(C)]. This Notice requests comment
based programs (including parenting skills information available about active on the definition of usual care or practice
training, parent education, and individual implementation will not be prioritized. settings.
and family counseling), and kinship 2.2.6 Implementation and Fidelity 2.4 Study Prioritization Criteria. Timing
navigator programs. Support. HHS intends to prioritize programs and resources may not allow for a detailed
2.1.2 Book/Manual/Writings Available. or services that have implementation training rating of all studies determined to be eligible
HHS intends to limit eligibility to programs and staff support and/or fidelity monitoring by the criteria identified in section 2.3 Study
or services that have a book, manual, or other tools and resources available to implementers Eligibility Criteria. HHS intends to conduct a
available documentation that specifies the in the United States. high-level scan of eligible studies to
components of the practice protocol and 2.2.7 Trauma-Informed. HHS may also determine which should be prioritized for
describes how to administer the practice. prioritize services and programs that have rating. This Notice requests comment on
2.2 Program or Service Prioritization been implemented using a trauma-informed criteria that can be used to prioritize eligible
Criteria. Timing and resources may not allow approach. FFPSA statutory language [section studies for rating.
for a detailed review of all programs and 471(e)(4)(B)] states, ‘‘The services or 2.4.1 Implementation Period: FFPSA
services determined to be eligible by the programs to be provided to or on behalf of [section 471(e)(1)(A) and (B)] states that the
criteria detailed in section 2.1 Program or a child are provided under an organizational Secretary may make a payment to a State for
Service Eligibility Criteria. Programs or structure and treatment framework that providing services or programs ‘‘for not more
services may be prioritized for review on the involves understanding, recognizing, and than a 12-month period’’. This Notice
basis of the following criteria: responding to the effects of all types of requests comment on whether studies with
2.2.1 Types of Programs and Services. As trauma and in accordance with recognized program or service implementation periods
noted in 2.1.1. Types of Programs and principles of a trauma-informed approach of longer than 12 months should be
Services, HHS intends to limit eligibility to and trauma-specific interventions to address considered for review and if so, whether any
mental health and substance abuse trauma’s consequences and facilitate other implementation period cutoff should be
prevention and treatment services, in-home healing.’’ This Notice requests comment on included as a study prioritization criterion.
parent skill-based programs (including the feasibility of prioritizing programs and 2.4.2 Sample of Interest. HHS intends to
parenting skills training, parent education, services based on past implementation in prioritize studies that include samples of
and individual and family counseling), or accordance with trauma-informed principles. children and families involved in the child
kinship navigator programs. This Notice 2.2.8 Delivery Setting for In-Home Parent welfare system or populations similar to
requests comment on the scope of programs Skill-Based Programs and Services. HHS those involved in the child welfare system.
and services and topic areas of interest intends to prioritize in-home parent skill- This Notice requests comment on
within the aforementioned categories that based programs and services where the populations that may be considered
should be prioritized for inclusion. primary service delivery strategy takes place ‘‘similar’’ to those involved in the child
2.2.2 Target Population of Interest. HHS in the caregivers’ place of residence. welfare system.
intends to prioritize programs or services for 2.3 Study Eligibility Criteria. HHS intends 2.5 Study Rating Criteria. HHS intends to
review that have been developed or used to to engage in a broad literature search to rate studies on the following criteria:
target children and families involved in the identify studies examining prioritized 2.5.1 Favorable Effects. HHS intends to
child welfare system or populations similar programs and services. This search may rate studies based on whether they
to those involved in the child welfare system. include databases, websites, existing demonstrate at least one meaningful
This Notice requests comment on literature reviews, and meta-analyses. HHS favorable effect (i.e., positive significant
populations that may be considered intends to screen studies for eligibility using effect) on a ‘target outcome’ as specified in
‘‘similar’’ to those involved in the child the following criteria: section 2.3.2 Target Outcomes. A meaningful
welfare system. 2.3.1 Impact Study. HHS intends to limit effect will be defined using conventional
2.2.3 Target Outcomes. HHS intends to eligibility to studies included in government standards of statistical significance (i.e., two-
prioritize programs or services for review that reports and peer-reviewed journal articles tailed hypothesis test and a specified alpha
aim to impact target outcomes. Target that assess effectiveness (i.e., impact) using level of p < .05). This Notice requests
outcomes should be defined in accordance quantitative methods. comment on whether and how ratings should
with FFPSA statutory language [section 2.3.2 Target Outcomes. HHS intends to consider the number or magnitude of
471(e)(4)(C)] and include those outcomes that limit eligibility to studies that examine the favorable effects.
‘‘. . . prevent child abuse and neglect, and impact of the service or program on at least 2.5.2 Unfavorable Effects. HHS intends to
daltland on DSKBBV9HB2PROD with NOTICES
reduce the likelihood of foster care one ‘target outcome’, as described in section rate studies based on the number of
placement by supporting birth families and 2.2.3. Target Outcomes. As noted above, this unfavorable effects (i.e., negative significant
kinship families and improving targeted Notice requests comment on specific effects) on either ‘target’ or non-target
supports for pregnant and parenting youth outcomes in accordance with FFPSA outcomes as specified in section 2.3.2 Target
and their children.’’ These may include, but statutory language that should be considered Outcomes. Effects will be defined using
are not limited to, ‘‘. . . important child and ‘target outcomes’ and requests research conventional standards of statistical
parent outcomes, such as mental health, evidence to support recommendations of significance (i.e., two-tailed hypothesis test
substance abuse, and child safety and well- ‘target outcomes’. and a specified alpha level of p < .05). This
VerDate Sep<11>2014 17:16 Jun 21, 2018 Jkt 244001 PO 00000 Frm 00036 Fmt 4703 Sfmt 4703 E:\FR\FM\22JNN1.SGM 22JNN1](https://thefederalregister.org/doc/83_FR_29243/page/2.svg)
![29124 Federal Register / Vol. 83, No. 121 / Friday, June 22, 2018 / Notices
Notice requests comment on whether and intends to rate studies as either high, empirical basis suggesting that,
how studies should also be rated on the moderate, or low on the extent to which compared to its likely benefits, the
number of null effects on ‘target outcomes’, ‘target outcome’ measures are reliable (i.e., practice constitutes a risk of harm to
and on whether and how ratings should the extent to which a measure produces the
those receiving it’’, ‘‘If multiple outcome
consider the number or magnitude of same results when used repeatedly), valid
unfavorable effects. (i.e., the extent to which a measure captures studies are conducted, the overall
2.5.3 Sustained Favorable Effect. HHS what it is intended to measure), and were weight of evidences supports the
intends for studies with at least one favorable administered consistently and accurately benefits of the practice’’, and ‘‘There is
effect on a ‘target outcome’, as determined by across all those receiving the practice in no case data suggesting a risk of harm
the criteria in 2.5.1 Favorable Effects, to be accordance with FFPSA statutory language that was probably caused by the
rated on whether or not they demonstrate a [section 471(e)(4)(C)] or receiving the treatment and that was severe or
sustained favorable effect. As noted in appropriate comparison practice. This Notice frequent’’.
section 471(e)(4)(C), a ‘supported practice’ requests comment on appropriate thresholds
must have at least one study that for evaluating and assigning a rating to the 3.0 Recommendations of Potential
demonstrates ‘‘a sustained effect (when reliability, validity, and administration of Candidate Programs and Services for
compared to a control group) for at least 6 ‘target outcome’ measures. Review
months beyond the end of treatment’’ and a 2.6 Program or Service Rating Criteria.
‘well-supported practice’ must have at least HHS intends for programs or services to be This Notice requests comment on
one study that demonstrates ‘‘a sustained rated as promising, supported, or well- potential candidate programs and
effect (when compared to a control group) for supported practices if they meet the below services to consider for the systematic
at least 1 year beyond the end of treatment.’’ criteria that collectively assess the strength of evidence review. Comments should
HHS intends to classify studies as not evidence for a practice and build from the identify how recommended programs
demonstrating a sustained favorable effect individual study criteria described in section and services meet the criteria described
(i.e., effects are demonstrated for less than 6 2.5 Study Rating Criteria. These criteria were in section 2.1 Program or Service
months), demonstrating a sustained favorable developed in accordance with FFPSA
Eligibility Criteria. These criteria
effect of 6 months or more (but less than 12 statutory language [section 471(e)(4)(C)].
months), or demonstrating a sustained 2.6.1 Promising Practice: HHS intends to include: Types of Programs and Services
favorable effect of 12 months or more. designate a program or service as a and Book/Manual/Writings Available.
2.5.4 Rigorous Study Design. HHS ‘promising practice’ if the program or service Comments should also identify how
intends to rate studies as either high, has at least one study that demonstrates a recommended programs and services
moderate, or low on the rigor and favorable effect on a target outcome as meet the criteria described in section 2.2
appropriateness of their study design. Study described by criterion 2.5.1 Favorable Effects Program or Service Prioritization
designs that receive the highest rating will be and achieves, at a minimum, moderate Criteria. These criteria include: Types of
either Randomized Controlled Trials (RCTs) ratings on criteria 2.5.4 through 2.5.6.
Programs and Services, Target
or rigorous quasi-experimental designs. HHS 2.6.2 Supported Practice: HHS intends to
designate a program or service as a Population of Interest, Target Outcomes,
defines randomized controlled trials as a
study design in which sample members are ‘supported practice’ if the program or service Number of Impact Studies, In Use/
assigned to the program or service and has at least one study that demonstrates a Active, Implementation and Fidelity
comparison groups by chance. Randomized favorable effect on a target outcome as Support, Trauma-Informed, and
control designs are often considered the described by 2.5.1 Favorable Effects, Delivery Setting for In-Home Parent
‘‘gold standard’’ of research design because demonstrates a sustained favorable effect on Skill-Based Programs and Services. In
personal characteristics (before the program a target outcome of at least 6 months beyond order to leverage new insights from the
or service begins) do not affect whether the end of treatment as described in Section field, HHS may put forth additional
someone is assigned to the program or 2.5.3 Sustained Favorable Effect, and
future Notices requesting
service or control group. HHS defines a achieves the high rating on criteria 2.5.4
through 2.5.6. recommendations of potential candidate
quasi-experimental design as a study design
in which sample members are selected for 2.6.3 Well-Supported Practice: HHS programs and services for review.
the program or service and comparison intends to designate a program or service as 4.0 Submission of Comments
groups in a nonrandom way. Similar to a ‘well-supported practice’ if the practice has
criteria considered in other federal evidence at least two studies with non-overlapping Comments may be submitted until
clearinghouses, rigorous study designs will analytic samples and distinct July 22, 2018 by email to
be those that are appropriately powered, implementations that demonstrate favorable ffclearinghouse@acf.hhs.gov.
include an appropriate control group, effects as described by 2.5.1 Favorable
Effects, demonstrate sustained favorable Naomi Goldstein,
maintain original assignment to study arms,
and are appropriate to combat threats to effects of at least 12 months beyond the end Deputy Assistant Secretary for Planning,
internal validity. This Notice requests of treatment as described in Section 2.5.3 Research, and Evaluation.
comment on threats to internal validity that Sustained Favorable Effect, and achieve the [FR Doc. 2018–13420 Filed 6–21–18; 8:45 am]
should be considered. This Notice requests high rating on criteria 2.5.4 through 2.5.6. BILLING CODE 4184–25–P
comment on appropriate thresholds for HHS does not intend to rate a program
evaluating and assigning a rating to a study
or service as a ‘promising’, ‘supported’,
design. DEPARTMENT OF HEALTH AND
2.5.5 Rigorous Study Analysis. HHS or ‘well-supported practice’ if there is
an empirical basis, as evidenced by HUMAN SERVICES
intends to rate studies as either high,
moderate, or low on the rigor and multiple unfavorable effects on target or
Food and Drug Administration
appropriateness of their analysis. Study non-target outcomes across reviewed
analyses that receive the highest rating may studies, as described in 2.5.2 [Docket No. FDA–2018–N–2066]
be those that tested and established baseline Unfavorable Effects, that suggest the
equivalence, appropriately accounted for overall weight of evidence does not Tobacco Products Scientific Advisory
daltland on DSKBBV9HB2PROD with NOTICES
overall and differential sample attrition, support the benefits of the program or Committee; Notice of Meeting
appropriately accounted for multiple
service. This Notice requests comment AGENCY: Food and Drug Administration,
comparisons, and when necessary accounted
for clustering. This Notice requests comment on approaches for determining that HHS.
on appropriate thresholds for evaluating and promising, supported, and well- ACTION: Notice.
assigning a rating to a study analysis. supported practices do not constitute a
2.5.6 Reliability, Validity, and Systematic risk of harm. As described in FFPSA SUMMARY: The Food and Drug
Administration of Outcome Measures. HHS [section 471(e)(4)(C)], ‘‘There is no Administration (FDA) announces a
VerDate Sep<11>2014 17:16 Jun 21, 2018 Jkt 244001 PO 00000 Frm 00037 Fmt 4703 Sfmt 4703 E:\FR\FM\22JNN1.SGM 22JNN1](https://thefederalregister.org/doc/83_FR_29243/page/3.svg)
Document Created: 2018-11-06 09:50:07
Document Modified: 2018-11-06 09:50:07
| Category | Regulatory Information | |
| Collection | Federal Register | |
| sudoc Class | AE 2.7: GS 4.107: AE 2.106: | |
| Publisher | Office of the Federal Register, National Archives and Records Administration | |
| Section | Notices | |
| Action | Request for public comment. | |
| FR Citation | 83 FR 29122 |
2025 Federal Register | Disclaimer | Privacy Policy
USC | CFR | eCFR