82 FR 58434 - Family Self-Sufficiency Performance Measurement System (“Composite Score”)

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

Federal Register Volume 82, Issue 237 (December 12, 2017)

This Notice describes and requests comment on a performance measurement system that HUD plans to implement for Public Housing Agencies (PHAs) that receive HUD Family Self-Sufficiency (FSS) program coordinator grants. The Notice also requests comment on whether and, if so, how to develop a performance measurement system for FSS programs that do not receive HUD FSS coordinator funding. The desired effect of this notice is to notify and solicit comments from public housing agencies regarding new proposed criteria for evaluating FSS programs.

[Federal Register Volume 82, Number 237 (Tuesday, December 12, 2017)]
[Notices]
[Pages 58434-58439]
From the Federal Register Online  [www.thefederalregister.org]
[FR Doc No: 2017-26696]



[[Page 58434]]

=======================================================================
-----------------------------------------------------------------------

DEPARTMENT OF HOUSING AND URBAN DEVELOPMENT

[Docket No. FR-6046-N-01]


Family Self-Sufficiency Performance Measurement System 
(``Composite Score'')

AGENCY: Office of Public and Indian Housing, HUD.

ACTION: Notice of Proposed New Performance Measurement System 
(``Composite Score'') for the Family Self-Sufficiency Program.

-----------------------------------------------------------------------

SUMMARY: This Notice describes and requests comment on a performance 
measurement system that HUD plans to implement for Public Housing 
Agencies (PHAs) that receive HUD Family Self-Sufficiency (FSS) program 
coordinator grants. The Notice also requests comment on whether and, if 
so, how to develop a performance measurement system for FSS programs 
that do not receive HUD FSS coordinator funding. The desired effect of 
this notice is to notify and solicit comments from public housing 
agencies regarding new proposed criteria for evaluating FSS programs.

DATES: Comment Due Date: January 26, 2018.

ADDRESSES: HUD invites interested persons to submit comments regarding 
the proposed FSS Performance Measurement System to the Regulations 
Division, Office of General Counsel, Department of Housing and Urban 
Development, 451 Seventh Street SW, Room 10276, Washington, DC 20410-
0001. Communications must refer to the above docket number and title 
and should contain the information specified in the ``Request for 
Comments'' section. There are two methods for submitting public 
comments.
    1. Submission of Comments by Mail. Comments may be submitted by 
mail to the Regulations Division, Office of General Counsel, Department 
of Housing and Urban Development, 451 7th Street SW, Room 10276, 
Washington, DC 20410-0500. Due to security measures at all federal 
agencies, however, submission of comments by mail often results in 
delayed delivery. To ensure timely receipt of comments, HUD recommends 
that comments submitted by mail be submitted at least two weeks in 
advance of the public comment deadline.
    2. Electronic Submission of Comments. Interested persons may submit 
comments electronically through the Federal eRulemaking Portal at 
http://www.regulations.gov. HUD strongly encourages commenters to 
submit comments electronically. Electronic submission of comments 
allows the commenter maximum time to prepare and submit a comment, 
ensures timely receipt by HUD, and enables HUD to make them immediately 
available to the public. Comments submitted electronically through the 
http://www.regulations.gov website can be viewed by other commenters 
and interested members of the public. Commenters should follow 
instructions provided on that site to submit comments electronically.

    Note: To receive consideration as public comments, comments must 
be submitted through one of the two methods specified above. Again, 
all submissions must refer to the docket number and title of the 
notice.

    No Facsimile Comments. Facsimile (FAX) comments are not acceptable.
    Public Inspection of Public Comments. All properly submitted 
comments and communications regarding this notice submitted to HUD will 
be available for public inspection and copying between 8 a.m. and 5 
p.m. weekdays at the above address. Due to security measures at the HUD 
Headquarters building, an advance appointment to review the public 
comments must be scheduled by calling the Regulations Division at 202-
708-3055 (this is not a toll-free number). Individuals with speech or 
hearing impairments may access this number through TTY by calling the 
Federal Relay Service at 800-877-8339. Copies of all comments submitted 
are available for inspection and downloading at http://www.regulations.gov.

FOR FURTHER INFORMATION CONTACT: Questions on this notice may be 
addressed to [email protected] or by contacting Anice Chenault at 502-618-
8163 (email strongly preferred)
    Electronic Data Availability. This Federal Register notice and a 
spreadsheet containing scores using the proposed methodology for FSS 
programs funded in any of the last three years will be available 
electronically from the HUD FSS Web page https://www.hud.gov/program_offices/public_indian_housing/programs/hcv/fss. Federal 
Register notices also are available electronically at https://www.federalregister.gov/, the U.S. Government Printing Office website.

SUPPLEMENTARY INFORMATION: This Notice sets forth a new performance 
measurement system for evaluating the efficacy of FSS programs, 
requests comment on that performance measurement system, and asks 
additional questions regarding these proposed changes.

I. Why has HUD developed the FSS performance measurement system?

    In pursuit of advancing HUD's ability to evaluate the effectiveness 
of the FSS program, per statutory mandate (Section 23(i)(2) of the 
Housing Act of 1937), HUD has developed a new FSS performance 
measurement system to provide HUD, Congress, and public housing 
agencies (PHAs) with information on the performance of individual FSS 
programs. The information will help PHAs determine the extent to which 
PHAs are administering effective and impactful FSS programs that help 
participants to successfully graduate from the program and make 
progress toward economic security. The information will also help HUD 
understand the extent to which individual FSS program performance, and 
the performance of all FSS programs receiving HUD FSS coordinator 
funding as a group, improves or declines over time.
    HUD plans to use the performance measures to identify high 
performing and troubled FSS programs. In the future, HUD will likely 
consider the FSS performance score of an FSS program in determining FSS 
funding awards. HUD may also use the rating system to identify PHAs 
that could benefit from technical assistance to improve their programs. 
At this time, HUD does not envision using this performance measurement 
system for tribes/TDHEs, who do not report into Public and Indian 
Housing Information Center (PIC), or for PHAs with a Moving to Work 
(MTW) designation, as they report differently into PIC, using Form HUD-
50058-MTW. However, HUD is presently exploring a change to the 
reporting processes for MTW agencies in order to include them in the 
FSS performance scoring process.

II. What measures will HUD use to evaluate the performance of FSS 
programs receiving FSS funding?

    HUD developed the approach described in this Notice based in part 
on feedback received on an earlier performance measurement approach 
proposed in the FY 2014 FSS Notice of Funding Availability (NOFA). In 
the FY 2014 NOFA, HUD proposed evaluating FSS programs based on the 
share of FSS participants that experience an increase in earned income 
(also known as ``earnings growth'') over a specified time period. Among 
other feedback, commentators expressed concern that this approach did 
not adequately account for differences in local economic conditions and 
differences in the approach of local FSS programs.

[[Page 58435]]

While some FSS programs encourage participants to increase their 
earnings immediately, others encourage FSS participants to build skills 
and credentials first and then seek higher paying jobs. The new FSS 
performance measurement system addresses these issues, as well as many 
others, allowing for a more nuanced evaluation of the performance of 
local FSS programs.
    Under the planned performance measurement system, at least once per 
year, HUD will analyze data collected through the PIC to calculate FSS 
performance scores for each FSS program for which sufficient data are 
available to calculate the score. A PHA's FSS performance score will be 
calculated based on three measures, weighted as follows:

A. Earnings Performance Measure (50 percent)
B. Graduation Rate (30 percent)
C. Participation Rate (20 percent)

    HUD has selected these measures because they are important 
indicators of program performance and are verifiable using the data HUD 
collects through the PIC data system. No outside or additional 
reporting will be required, ensuring the system does not increase the 
reporting burden of PHAs. No new Paperwork Reduction Act (PRA) 
Information Collection will be required for the scoring, as proposed.
    As described below, the Earnings Performance Measure represents the 
difference between the earnings growth of FSS participants and the 
earnings growth of other similar households within the PHA within a 
specified time frame. This approach helps to control for variations in 
local economic conditions. Earnings growth is one of the primary 
outcomes desired from FSS; the FSS performance score therefore assigns 
the Earnings Performance Measure a high weight. HUD has assigned the 
next highest weight to the Graduation Rate indicator--which represents 
the rate of FSS participants who successfully ``graduate'' from the 
program--to encourage PHAs to work closely with individual FSS 
participants to increase graduation rates. (To graduate from FSS, a 
participant must be employed, be independent of welfare assistance for 
at least one year, and achieve the other goals set forth in the 
participant's contract of participation.) Finally, the FSS performance 
score looks at Participation Rate, which reflects the extent to which a 
PHA exceeds the minimum number of households that HUD requires the PHA 
to serve as a condition of receiving an FSS grant. PHAs with higher 
Participation Rates are serving more households than required, which is 
a desired output, provided the PHAs are serving those households 
effectively. Because the Earnings Performance Measure is weighted more 
heavily than the Participation Rate, however, PHAs should be careful 
not to execute more Contracts of Participation than they can serve 
effectively, because doing so would likely reduce their scores on the 
Earnings Performance Measure.
    Together, the Earnings Performance Measure, Graduation Rate, and 
Participation Rate provide a balanced measurement of the performance of 
an individual FSS program. The three measures are calculated as 
follows:

A. Earnings Performance Measure Calculation

    The Earnings Performance Measure gauges the extent to which the 
earnings \1\ of FSS participants increase over time after joining the 
FSS program. In developing the methodology for this measure, HUD has 
been sensitive to the fact that some FSS programs encourage FSS 
participants to immediately increase their earnings while others 
encourage FSS participants to first build human capital through 
education and training in order to qualify for higher paying jobs. The 
methodology is also sensitive to the fact that the earnings of low-
income workers are often volatile, and that the economic conditions in 
which different FSS programs are operating vary from community to 
community.
---------------------------------------------------------------------------

    \1\ For the purposes of the FSS program and these FSS measures, 
earnings are defined as annual earnings from all wage sources, as 
recorded on the HUD-50058 form. These include the following form 
50058 income codes: B--Own Business, F--Federal Wages, HA--PHA Wage, 
M--Military Wage or W--Other Wage.
---------------------------------------------------------------------------

    To accommodate these different factors and control for variations 
among FSS programs, HUD calculates the Earnings Performance Measure for 
each FSS program using the process outlined below. HUD applies this 
process to the population of FSS participants who enrolled in the FSS 
program 3.5 to 7.5 years prior to the end of the most recent quarter of 
data available through PIC to calculate the latest FSS performance 
scores.
    Controlling for Variations in the Composition of Local FSS 
Programs: While households with elderly heads or heads who are a person 
with disabilities may participate in FSS, such households are not 
included in the calculation of a PHA's earnings performance measure. 
This ensures that PHAs that serve larger shares of such households are 
not disadvantaged in the performance measurement process as compared to 
PHAs that serve smaller shares of such households.
    Controlling for FSS Program Model and Earnings Fluctuations: To 
calculate an Earnings Performance Measure for a PHA, HUD first measures 
the growth in annual household earnings of each household enrolled in 
FSS at the PHA in two ways and selects the higher of the two measures 
for each household:
    1. Earnings Growth Since Enrollment: The difference between (i) 
annual earnings upon enrollment in FSS and (ii) the most recent 
earnings estimate available in PIC for that household from an annual 
reexamination.
    2. Average Annual Earnings While in FSS: The difference between (i) 
earnings upon enrollment in FSS and (ii) the household's average annual 
earnings during the time period between enrollment in FSS and the most 
recent annual reexamination of income available in PIC.
    Controlling for FSS Program Model and Earnings Fluctuations: HUD 
selects the higher of the two measures for each household in order to 
accommodate different approaches to implementing FSS while also 
correcting for variations in year-to-year earnings, which can be 
volatile for low-income households. Some PHAs encourage FSS 
participants to focus immediately on increasing their earnings, while 
others encourage FSS participants to focus on obtaining education and 
building skills first and then seek a higher paying job once they have 
stronger credentials. Other agencies use both approaches, tailoring the 
approach to each individual. Measure 1, Earnings Growth Since 
Enrollment, accommodates programs that encourage participants to focus 
first on education and training, while both measures work acceptably 
for programs that encourage individuals to increase their earnings 
immediately. Measure 2, Average Annual Earnings While in FSS, focuses 
on the difference between starting and average annual earnings, which 
ensures that an FSS participant who has made good progress in 
increasing earnings while in FSS, but who nevertheless has experienced 
a temporary setback of job loss as of the most recent annual 
reexamination, nevertheless has his or her progress recognized. For 
each household, the Earnings Performance Measure focuses on the higher 
of the two measures, maximizing HUD's ability to recognize households' 
progress toward increased earnings while participating in FSS.
    Controlling for Local Economic Conditions: Because economic 
conditions vary from one community to the next, HUD has built in a 
mechanism to control for these differences. HUD

[[Page 58436]]

adjusts for local economic conditions by comparing the average earnings 
growth of FSS participants at a PHA to the average earnings growth for 
nonparticipants with similar characteristics at the same PHA. The 
difference in performance between the two groups represents the 
Earnings Performance Measure for that PHA. Since the earnings of non-
FSS participants would be expected to grow faster at PHAs located in 
stronger job markets than in PHAs located in weaker job markets, this 
comparison helps to account for differences in local economic 
conditions, which facilitates a meaningful comparison of earnings 
growth across FSS programs. Specifically, to calculate an Earnings 
Performance Measure for each PHA, HUD:
     Selects three comparison households for each FSS household 
based on the extent to which the comparison households are similar to 
the FSS household on the following characteristics: Earnings as of the 
time of the FSS household's entry into FSS, age of head of household, 
length of time in the voucher or public housing program, number of 
adults in the household and number of children under age 5.
     Calculates the earnings growth for all of the comparison 
households using the same approach used to calculate the earnings 
growth for FSS households, with the FSS household's enrollment date 
being applied to its comparison households for purposes of calculating 
the comparison households' initial earnings.
     Calculates the difference between the average earnings 
growth for all FSS participants and the average earnings growth for all 
comparison households at each PHA. The difference between the two 
represents the PHA's earnings performance measure.
    HUD applies this measure to all FSS participants with a head of 
household who is neither elderly nor a person with disabilities who 
joined FSS between 3.5 and 7.5 years prior to the end of the quarter of 
the PIC extract used to calculate the score. For example, if the most 
recent PIC data extract ended in March 31, 2017, HUD's calculation of 
earnings performance measures would focus on FSS participants who 
joined the FSS program between October 1, 2009 and September 30, 2013. 
This methodology aggregates information for four years of FSS entrants 
in order to generate a large enough sample to analyze. The methodology 
does not examine data for participants that have entered the FSS 
program more recently than 3.5 years ago to allow sufficient time to 
have passed for FSS participants to have benefitted from the program. 
At the same time, the methodology does not focus only on an older 
sample of FSS participants to ensure that the results reflect recent 
FSS program performance to the maximum extent practicable.
    Technical note: In measuring earnings growth, the methodology 
focuses solely on earnings determined through annual reexaminations, 
disregarding the results of any interim reexaminations. The reason for 
doing this is that not all PHAs require interim reexaminations of 
income when earnings rise in between annual reexaminations. To ensure 
an apples-to-apples comparison of earnings growth across PHAs, HUD 
focuses only on annual reexaminations. An annual progress report is 
required for every FSS participant regardless of the spacing of rental 
re-examinations, so PHAs involved in rent reform demonstrations would 
be included in this scoring.

B. Graduation Rate Calculation

    This measure examines the share of FSS participants at each PHA who 
have ``graduated'' from the FSS program. It is calculated based on the 
graduation rate of FSS participants who entered each PHA's FSS program 
5 to 8 years before the end of the most recent quarter of available PIC 
data. The methodology focuses on these households to allow sufficient 
time for most of the FSS participants who will graduate to have done 
so. HUD considered focusing on an older cohort to capture 100 percent 
of the FSS participants who will graduate, but HUD determined that it 
was more advantageous for the period analyzed to include more recent 
performance by the PHA.
    Controlling for Turnover Rates: Turnover rates at PHAs can vary 
significantly for reasons unrelated to FSS. To avoid penalizing 
programs with higher turnover, HUD excludes non-graduating FSS 
participants who exited the Housing Choice Voucher (HCV) or Public 
Housing programs before the end of the analysis period from both the 
numerator and the denominator in calculating the Graduation Rate.

C. Participation Rate Calculation

    The Participation Rate is the ratio of the number of FSS 
participants being served to the minimum number expected to be served 
under the standards used for awarding funding under the FSS NOFA. 
Agencies that exactly meet the standard will have a ratio of 1.0. 
Agencies that serve more than the required number will have a ratio 
above 1.0. Agencies that serve fewer than the required number will have 
a ratio below 1.0.
    To calculate the Participation Rate, HUD first calculates the 
minimum number of FSS participants that HUD expects each PHA to serve 
for each of the most recent three (3) fiscal years for which both 
funding award and number served data are available. HUD calculates this 
number based on the guidelines in the NOFA and the number of 
coordinators funded in each agency during each year. HUD then sums the 
number of FSS participants actually served in each of the three years 
based on PIC data. Finally, HUD divides the total number of FSS 
participants served in each PHA by the total minimum number expected 
for the PHA's HUD-funded coordinator positions to determine the 
participation rate. If funding is only awarded to the PHA in one or two 
of the three years, the measure only uses data for the years for which 
funding was awarded. Note that this metric, while similar, is different 
from the ``number of participants served,'' which has been used in NOFA 
competitions and assesses only the most recent period of performance.
    Controlling for Annual Variation and PIC Reporting: HUD also 
separately calculates the Participation Rate for the most recent year 
and then grades a PHA's Participation Rate based on the higher of: (a) 
The PHA's three-year average and (b) the most recent year. Looking at 
the higher of the these two values allows HUD to use the most recent 
available data for PHAs that have made progress in increasing the 
number served while avoiding penalizing PHAs for the results of an 
atypical year. It also ensures that PHAs that have improved the quality 
of their PIC reporting on FSS participation can be judged based on the 
FSS participant counts derived from recent PIC reports, rather than 
from reports submitted in earlier years. Given the new guidance that 
HUD issued on PIC reporting for FSS on May 16, 2016 (PIH Notice 2016-
08), HUD expects the quality of FSS reporting to PIC to be improved 
going forward and reminds PHAs of the importance of ensuring accurate 
and timely submissions of FSS Addendums to PIC.
    As calculated using the procedures described above, the 
participation rate is higher if the PHA has served more participants 
relative to its funding level. The ratio required in the NOFA is 25 for 
one full-time coordinator and 50 for each additional full-time 
coordinator. For example, a PHA with 1 funded full-time coordinator is 
expected to serve at least 25 participants during the year, while a PHA 
with 3 funded full-time coordinators is expected to serve at least

[[Page 58437]]

125 participants. If the PHA with 1 coordinator serves 40 FSS 
participants (much more than the minimum required) and the PHA with 3 
coordinators serves 130 participants (only slightly more than the 
minimum expected), the PHA with the smaller number of coordinators and 
participants will have a higher participation rate (40/25 = 1.60 versus 
130/125 = 1.04).
    PHAs that receive funding jointly with other PHAs are evaluated 
together in calculating the participation rate. HUD sums the number of 
FSS participants served by each of the jointly-funded agencies and the 
minimum number of participants the agencies are jointly expected to 
serve and provides the same participation score for each of the PHAs.

III. How will HUD convert the measures into an FSS Performance Score?

    After making the calculations described above, HUD will develop an 
FSS Performance Score for each PHA using a two-step process.

A. Step One: Assigning Scores to Each of the Three Measures

    In Step One, HUD will assign a score of 0 to 10 to each PHA's FSS 
program for each of the three measures. Scores will be assigned using 
the procedures described below. The ranges for awarding points between 
two values include those values as well as all intermediary values.
    For each of the three measures, HUD has selected criteria for 
evaluating PHA performance. For each measure, the highest performers 
are assigned a score of 10, the next-highest performers are assigned a 
score of 7.5, and low performers are assigned a score of 0. HUD will 
award a score of 5 to PHAs whose performance does not satisfy the 
criteria for highest, next-highest, or low performance for that 
measure.
1. Earnings Performance Measure (50 Percent of Final Score)
     10 points: Earnings performance measure of $6,400 or 
higher.
     7.5 points: Earnings performance measure between $4,750 
and $6,399.
     0 points: Earnings performance measure below $1,500 and a 
p-value of .10 on a statistical test measuring the likelihood that a 
PHA's earnings performance measure is significantly lower than the 
median measure of $3,418 (see below for an explanation of this 
statistical test).
     5 points: All PHAs that do not qualify for a 10, 7.5, or a 
0.
    As described above, a PHA's earnings performance measure represents 
the difference between: (a) The average earnings growth for FSS 
participants and (b) the average earnings growth for comparison 
households at the same PHA. A PHA's earnings performance measure is not 
simply a measure of the extent to which FSS participants increased 
their earnings. Instead, a PHA's earnings performance measure reflects 
the relative growth of FSS participants relative to a matched set of 
non-participants at that PHA. HUD assigns a higher score to FSS 
programs that achieve a higher earnings performance score.
    In addition to focusing on the size of the earnings performance 
measure, the scoring for this measure applies a one-tailed test of 
statistical significance, designed to protect FSS programs from being 
scored ``low performer'' due to random variation and low sample size. 
For example, without this protection, an individual FSS program may 
include several anomalous participants or control households that skew 
research results. The statistical test measures the likelihood that a 
PHA's earnings performance measure is significantly lower than the 
median measure. The lower the p-value, the less likely it is that a PHA 
received a below-median earnings performance measure due to random 
variation. To receive 0 points, a PHA must not only have an earnings 
performance measure below $1,500 but also a p-value on this test of 
less than .10, which means there is at least a 90 percent probability 
that the earnings performance measure is truly below the median value 
of $3,418.
    While a similar statistical test could theoretically be applied to 
help identify high performing programs, such a test would make it 
harder for small FSS programs to qualify. To avoid disadvantaging 
smaller FSS programs, p-values are not considered in determining 
whether to award 10 or 7.5 points.
2. Graduation Rate (30 Percent of Final Score)
     10 points: Graduation rate of 38 percent or higher.
     7.5 points: Graduation rate between 27 percent and 37.99 
percent.
     0 points: Graduation rate of 8 percent or lower.
     5 points: All PHAs that do not qualify for a 10, 7.5, or a 
0.
    Under this approach, a higher graduation rate results in a higher 
score.
3. Participation Rate (20 Percent of Final Score)
     10 points: Participation rate of 2.1 or higher.
     7.5 points: Participation rate between 1.7 and 2.09.
     0 points: Participation rate of 0.95 or lower.
     5 points: All PHAs that do not qualify for a 10, 7.5, or a 
0.
    Under this approach, a higher participation rate results in a 
higher score.
Step Two: Developing the Final FSS Performance Score and Grade
    After computing individual scores for each of the three measures, 
HUD aggregates each PHA's scores using the weights noted above to 
develop a final FSS Performance Score from 0 to 10. Based on this 
score, HUD assigns the following ranking to the PHA's performance:
     Excellent: FSS Performance score of 7.25 or higher.
     Standard: FSS Performance score between 4.0 and 7.24.
     Low: FSS Performance score between 3.00 and 3.99.
     Troubled: FSS Performance score of less than 3.00.

IV. How were these thresholds selected?

    The thresholds for converting the three performance measures into 
scores in step one are fixed and will now apply to all future years 
until HUD revises the methodology. These thresholds were selected by 
applying the FSS Performance Score methodology to PIC data from the 
quarter ending December 31, 2016. The thresholds were selected as 
follows:

1. Earnings Performance Measure (50 Percent of Final Score)

     The threshold for awarding a score of 10 points (an 
earnings performance measure of $6,400) represents approximately the 
80th percentile of the distribution of results of the earnings 
performance measure for PHAs whose measures have a p value >.10 on a 
statistical test measuring the likelihood that the earnings performance 
measure is different from $0. HUD calculated the distribution using 
agencies that receive a p value below .10 on this test to reduce the 
likelihood that the results would be affected by random variation.
     The threshold for awarding a score of 7.5 points ($4,750) 
represents approximately the 60th percentile of the distribution of 
results of the earnings performance measure for PHAs whose measures 
have a p value <.10 on the statistical test described above.
     The threshold for awarding a score of 0 points ($1,500) 
represents approximately the 20th percentile of the distribution of 
results of the earnings performance measure for all PHAs.

[[Page 58438]]

2. Graduation Rate (30 Percent of Final Score)

     The threshold for awarding a score of 10 points represents 
approximately the 80th percentile of the distribution of graduation 
rates.
     The threshold for awarding a score of 7.5 points 
represents approximately the 60th percentile of the distribution of 
graduation rates.
     The threshold for awarding a score of 0 points represents 
approximately the 20th percentile of the distribution of graduation 
rates.

3. Participation Rate (20 Percent of Final Score)

     The threshold for awarding a score of 10 points represents 
approximately the 80th percentile of the distribution of participation 
rates.
     The threshold for awarding a score of 7.5 points 
represents approximately the 60th percentile of the distribution of 
participation rates.
     The threshold for awarding a score of 0 points is 0.95, 
which falls below the minimum standard established by HUD. A PHA 
serving the minimum number of FSS participants required to obtain FSS 
funding would normally have a participation rate of 1.0. However, this 
methodology uses a score of 0.95 to give PHAs the benefit of the doubt 
and account for any temporary vacancies in the FSS program.

4. Composite FSS Performance Scores and Grades

     The threshold for awarding a ranking of Excellent 
represents approximately the 80th percentile of the distribution of FSS 
Performance Scores.
     The range for awarding a ranking of Low represents 
approximately the 10th through the 20th percentiles in the distribution 
of FSS Performance Scores.
     Programs falling below approximately the 10th percentile 
in the distribution of FSS Performance Scores are classified as 
Troubled.
     All other FSS programs are classified as ``Standard'' 
performers. The range for awarding a ranking of Standard represents 
approximately the 20th through the 80th percentiles of the distribution 
of FSS Performance Scores.
    As noted above, all thresholds are now fixed and will not be 
recalculated each year. This will facilitate tracking individual PHA 
progress as well as that of all FSS programs over time. Further, this 
framework does not limit how many programs can receive any particular 
ranking. The thresholds are absolute, not relative.

V. What else do PHAs need to know about the FSS performance score 
methodology?

    The following is additional information about how HUD calculates 
FSS performance scores:
    1. For households entering FSS more than one time during the 
analysis period, the methodology focuses only on the FSS Contract of 
Participation that began 5 to 8 years before the end of the most recent 
quarter of available PIC data to calculate the FSS performance score. 
This facilitates appropriate evaluation of each program's graduation 
rate, which focuses on the same group of households. If a participant 
entered more than once during that period, the methodology focuses on 
the older entry.
    2. FSS performance scores are calculated for any PHA that has 
sufficient data in PIC to calculate at least one of the three measures 
used to calculate the score. If there are insufficient data to 
calculate one or two of the measures, that PHA will receive a middle 
(standard) score of ``5'' for the missing measure(s) before calculating 
the FSS performance score.
    3. A PHA for which none of the three scores are available will not 
receive a score.
    4. Because the earnings performance measure and the graduation rate 
are calculated using data that spans a range of years, it will take 
time for a PHA to improve its FSS Performance Score through 
improvements in earnings and graduation outcomes. However, improvements 
in these areas will eventually become apparent in a PHA's FSS 
Performance Score. It is important for PHAs with low scores to begin 
implementing improvements as quickly as possible. PHAs with 
participation rates below 0.95 can quickly improve their FSS 
Performance Scores by increasing participation rates to meet HUD's 
minimum requirements.

VI. How will HUD assess the performance of FSS programs that do not 
receive funding?

    HUD is interested in evaluating the performance of all FSS programs 
administered by PHAs, including programs that do not receive funding 
from HUD. However, there are several concerns with applying the 
methodology described above to the evaluation of the performance of 
non-funded agencies. First, the participation rate cannot be calculated 
using the methodology described in this notice because there are no set 
expectations for program size. Second, such programs tend to be smaller 
than NOFA-funded programs, which means their results are more subject 
to random variation due to the participation of individuals with 
idiosyncratic features. Third, these program participants tend to 
receive less personal attention from FSS coordinators due to the lack 
of dedicated funding from HUD for FSS.
    HUD will continue studying options for measuring the performance of 
such agencies to determine if an approach can be developed for 
evaluating the quality of their FSS programs. To inform HUD's analysis 
of this issue, HUD requests comments on the following questions:
    1. Should HUD evaluate FSS programs that do not receive funding 
from HUD?
    2. Should the performance of an unfunded FSS program be considered 
by HUD in determining whether to award funding? If not, what factors 
should be used in determining whether to award funding to a currently 
unfunded agency?
    3. Should the FSS performance score of an unfunded PHA be compared 
solely with that of other unfunded PHAs or also against the performance 
of funded agencies?
    4. How should the procedures for evaluating the performance of 
funded FSS programs be adapted for purposes of measuring the 
performance of FSS programs that do not receive funding?
    5. Should HUD calculate a participation rate for unfunded FSS 
programs in evaluating their performance and if so, how should it be 
calculated?
    6. In addition to, or instead of a participation rate, should HUD 
limit the evaluation of non-funded agencies to FSS programs over a 
certain size, such as 15 or 25 participants? Focusing only on FSS 
programs of a certain minimum size should help to improve the 
reliability of the evaluation results while also focusing the 
evaluation (and any corresponding preference for funding) on PHAs that 
demonstrate a threshold level of commitment to the FSS program.

VII. Other Questions

    In addition to the questions noted above, HUD requests feedback on 
the following questions:
    1. Has HUD assigned the appropriate weight to each of the three 
measures? The proposed system uses the following weights: Earnings 
performance measure (50 percent); Graduation rate (30 percent); and 
Participation rate (20 percent).
    2. In evaluating earnings growth, HUD focuses on the average of the 
earnings growth of individual households at a PHA, rather than median 
growth. HUD

[[Page 58439]]

takes this approach to recognize the potential life-changing impacts of 
helping individuals move from unemployment to high-paying jobs. Such 
impacts are captured in looking at average earnings growth, but might 
be missed in looking only at the median growth. It is appropriate in 
this context to use averages, or should HUD switch to medians instead?
    3. Has HUD adequately accounted for variations in local economic 
conditions? If not, what further adjustments should be made? The 
earnings performance measure accounts for local economic conditions by 
comparing the earnings growth for FSS participants at a PHA to the 
earnings growth for non-FSS participants at the same PHA with similar 
characteristics. The assumption underlying this approach is that 
earnings growth for non-FSS participants will be higher in areas with 
stronger job markets than in areas with weaker job markets. To attain 
the same earnings performance measure, a PHA in an area with a strong 
job market would thus need to demonstrate a higher level of earnings 
growth among FSS participants than would a PHA in an area with a weaker 
job market. After calculating the difference between earnings growth 
for FSS and non-FSS participants at a PHA, the proposed system makes no 
further adjustments. Should HUD further adjust its system to account 
for variations in local economic conditions, and if so, how should HUD 
make this adjustment? For example, HUD could divide the earnings 
performance measure by the average starting earnings for a PHA's FSS 
participants and then compare the resulting percentages across PHAs. 
Further, HUD could adjust the earnings performance measures by an index 
that accounts for local economic conditions.
    4. HUD currently allows a PHA to count FSS participants living in 
multifamily FSS programs toward the minimum number of participants 
required to be served in order to qualify for FSS funding. The PIC data 
system, however, does not capture information on multifamily FSS 
participants. HUD requests suggestions on how best to capture 
information on multifamily FSS participants being served by a PHA's FSS 
coordinator to determine a PHA's participation rate.
    5. HUD currently permits, and funds, FSS programs in Tribes and 
Tribally Designated Housing Entities (TDHEs). However, Tribes and TDHEs 
do not report into the PIC data system. HUD requests suggestions on how 
to best capture information on tribal FSS participants to determine a 
score.
    6. HUD currently permits, and funds, FSS programs at MTW agencies. 
However, MTW agencies are only required to report select FSS data 
fields into the PIC system. HUD requests suggestions on how to best 
capture information on MTW FSS participants to determine a score.
    7. How should HUD evaluate FSS programs offered by HUD-assisted 
multifamily properties with Section 8 contracts? These programs are 
very new and currently submit quarterly spreadsheets rather than an FSS 
addendum integrated into a HUD data reporting system.

VIII. Environmental Impact

    This notice does not direct, provide for assistance or loan and 
mortgage insurance for, or otherwise govern or regulate, real property 
acquisition, disposition, leasing, rehabilitation, alteration, 
demolition, or new construction, or establish, revise or provide for 
standards for construction or construction materials, manufactured 
housing, or occupancy. Accordingly, under 24 CFR 50.19(c)(1), this 
notice is categorically excluded from environmental review under the 
National Environmental Policy Act of 1969 (42 U.S.C. 4321).

    Dated: December 5, 2017.
Dominique Blom,
General Deputy Assistant Secretary, Office of Public and Indian 
Housing.
[FR Doc. 2017-26696 Filed 12-11-17; 8:45 am]
 BILLING CODE 4210-67-P