Family Self-Sufficiency Performance Measurement System (“Composite Score”)

AGENCY:

Office of Public and Indian Housing, HUD.

ACTION:

Notice of Proposed New Performance Measurement System (“Composite Score”) for the Family Self-Sufficiency Program.

SUMMARY:

This Notice describes and requests comment on a performance measurement system that HUD plans to implement for Public Housing Agencies (PHAs) that receive HUD Family Self-Sufficiency (FSS) program coordinator grants. The Notice also requests comment on whether and, if so, how to develop a performance measurement system for FSS programs that do not receive HUD FSS coordinator funding. The desired effect of this notice is to notify and solicit comments from public housing agencies regarding new proposed criteria for evaluating FSS programs.

DATES:

Comment Due Date: January 26, 2018.

ADDRESSES:

HUD invites interested persons to submit comments regarding the proposed FSS Performance Measurement System to the Regulations Division, Office of General Counsel, Department of Housing and Urban Development, 451 Seventh Street SW, Room 10276, Washington, DC 20410-0001. Communications must refer to the above docket number and title and should contain the information specified in the “Request for Comments” section. There are two methods for submitting public comments.

1. Submission of Comments by Mail. Comments may be submitted by mail to the Regulations Division, Office of General Counsel, Department of Housing and Urban Development, 451 7th Street SW, Room 10276, Washington, DC 20410-0500. Due to security measures at all federal agencies, however, submission of comments by mail often results in delayed delivery. To ensure timely receipt of comments, HUD recommends that comments submitted by mail be submitted at least two weeks in advance of the public comment deadline.

2. Electronic Submission of Comments. Interested persons may submit comments electronically through the Federal eRulemaking Portal at http://www.regulations.gov. HUD strongly encourages commenters to submit comments electronically. Electronic submission of comments allows the commenter maximum time to prepare and submit a comment, ensures timely receipt by HUD, and enables HUD to make them immediately available to the public. Comments submitted electronically through the http://www.regulations.gov website can be viewed by other commenters and interested members of the public. Commenters should follow instructions provided on that site to submit comments electronically.

Note:

To receive consideration as public comments, comments must be submitted through one of the two methods specified above. Again, all submissions must refer to the docket number and title of the notice.

No Facsimile Comments. Facsimile (FAX) comments are not acceptable.

Public Inspection of Public Comments. All properly submitted comments and communications regarding this notice submitted to HUD will be available for public inspection and copying between 8 a.m. and 5 p.m. weekdays at the above address. Due to security measures at the HUD Headquarters building, an advance appointment to review the public comments must be scheduled by calling the Regulations Division at 202-708-3055 (this is not a toll-free number). Individuals with speech or hearing impairments may access this number through TTY by calling the Federal Relay Service at 800-877-8339. Copies of all comments submitted are available for inspection and downloading at http://www.regulations.gov.

FOR FURTHER INFORMATION CONTACT:

Questions on this notice may be addressed to FSS@hud.gov or by contacting Anice Chenault at 502-618-8163 (email strongly preferred)

Electronic Data Availability. This Federal Register notice and a spreadsheet containing scores using the proposed methodology for FSS programs funded in any of the last three years will be available electronically from the HUD FSS Web page https://www.hud.gov/program_offices/public_indian_housing/programs/hcv/fss. Federal Register notices also are available electronically at https://www.federalregister.gov/,, the U.S. Government Printing Office website.

SUPPLEMENTARY INFORMATION:

This Notice sets forth a new performance measurement system for evaluating the efficacy of FSS programs, requests comment on that performance measurement system, and asks additional questions regarding these proposed changes.

I. Why has HUD developed the FSS performance measurement system?

In pursuit of advancing HUD's ability to evaluate the effectiveness of the FSS program, per statutory mandate (Section 23(i)(2) of the Housing Act of 1937), HUD has developed a new FSS performance measurement system to provide HUD, Congress, and public housing agencies (PHAs) with information on the performance of individual FSS programs. The information will help PHAs determine the extent to which PHAs are administering effective and impactful FSS programs that help participants to successfully graduate from the program and make progress toward economic security. The information will also help HUD understand the extent to which individual FSS program performance, and the performance of all FSS programs receiving HUD FSS coordinator funding as a group, improves or declines over time.

HUD plans to use the performance measures to identify high performing and troubled FSS programs. In the future, HUD will likely consider the FSS performance score of an FSS program in determining FSS funding awards. HUD may also use the rating system to identify PHAs that could benefit from technical assistance to improve their programs. At this time, HUD does not envision using this performance measurement system for tribes/TDHEs, who do not report into Public and Indian Housing Information Center (PIC), or for PHAs with a Moving to Work (MTW) designation, as they report differently into PIC, using Form HUD-50058-MTW. However, HUD is presently exploring a change to the reporting processes for MTW agencies in order to include them in the FSS performance scoring process.

II. What measures will HUD use to evaluate the performance of FSS programs receiving FSS funding?

HUD developed the approach described in this Notice based in part on feedback received on an earlier performance measurement approach proposed in the FY 2014 FSS Notice of Funding Availability (NOFA). In the FY 2014 NOFA, HUD proposed evaluating FSS programs based on the share of FSS participants that experience an increase in earned income (also known as “earnings growth”) over a specified time period. Among other feedback, commentators expressed concern that this approach did not adequately account for differences in local economic conditions and differences in the approach of local FSS programs. While some FSS programs encourage participants to increase their earnings immediately, others encourage FSS participants to build skills and credentials first and then seek higher paying jobs. The new FSS performance measurement system addresses these issues, as well as many others, allowing for a more nuanced evaluation of the performance of local FSS programs.

Under the planned performance measurement system, at least once per year, HUD will analyze data collected through the PIC to calculate FSS performance scores for each FSS program for which sufficient data are available to calculate the score. A PHA's FSS performance score will be calculated based on three measures, weighted as follows:

A. Earnings Performance Measure (50 percent)

B. Graduation Rate (30 percent)

C. Participation Rate (20 percent)

HUD has selected these measures because they are important indicators of program performance and are verifiable using the data HUD collects through the PIC data system. No outside or additional reporting will be required, ensuring the system does not increase the reporting burden of PHAs. No new Paperwork Reduction Act (PRA) Information Collection will be required for the scoring, as proposed.

As described below, the Earnings Performance Measure represents the difference between the earnings growth of FSS participants and the earnings growth of other similar households within the PHA within a specified time frame. This approach helps to control for variations in local economic conditions. Earnings growth is one of the primary outcomes desired from FSS; the FSS performance score therefore assigns the Earnings Performance Measure a high weight. HUD has assigned the next highest weight to the Graduation Rate indicator—which represents the rate of FSS participants who successfully “graduate” from the program—to encourage PHAs to work closely with individual FSS participants to increase graduation rates. (To graduate from FSS, a participant must be employed, be independent of welfare assistance for at least one year, and achieve the other goals set forth in the participant's contract of participation.) Finally, the FSS performance score looks at Participation Rate, which reflects the extent to which a PHA exceeds the minimum number of households that HUD requires the PHA to serve as a condition of receiving an FSS grant. PHAs with higher Participation Rates are serving more households than required, which is a desired output, provided the PHAs are serving those households effectively. Because the Earnings Performance Measure is weighted more heavily than the Participation Rate, however, PHAs should be careful not to execute more Contracts of Participation than they can serve effectively, because doing so would likely reduce their scores on the Earnings Performance Measure.

Together, the Earnings Performance Measure, Graduation Rate, and Participation Rate provide a balanced measurement of the performance of an individual FSS program. The three measures are calculated as follows:

A. Earnings Performance Measure Calculation

The Earnings Performance Measure gauges the extent to which the earnings of FSS participants increase over time after joining the FSS program. In developing the methodology for this measure, HUD has been sensitive to the fact that some FSS programs encourage FSS participants to immediately increase their earnings while others encourage FSS participants to first build human capital through education and training in order to qualify for higher paying jobs. The methodology is also sensitive to the fact that the earnings of low-income workers are often volatile, and that the economic conditions in which different FSS programs are operating vary from community to community.

To accommodate these different factors and control for variations among FSS programs, HUD calculates the Earnings Performance Measure for each FSS program using the process outlined below. HUD applies this process to the population of FSS participants who enrolled in the FSS program 3.5 to 7.5 years prior to the end of the most recent quarter of data available through PIC to calculate the latest FSS performance scores.

Controlling for Variations in the Composition of Local FSS Programs: While households with elderly heads or heads who are a person with disabilities may participate in FSS, such households are not included in the calculation of a PHA's earnings performance measure. This ensures that PHAs that serve larger shares of such households are not disadvantaged in the performance measurement process as compared to PHAs that serve smaller shares of such households.

Controlling for FSS Program Model and Earnings Fluctuations: To calculate an Earnings Performance Measure for a PHA, HUD first measures the growth in annual household earnings of each household enrolled in FSS at the PHA in two ways and selects the higher of the two measures for each household:

1. Earnings Growth Since Enrollment: The difference between (i) annual earnings upon enrollment in FSS and (ii) the most recent earnings estimate available in PIC for that household from an annual reexamination.

2. Average Annual Earnings While in FSS: The difference between (i) earnings upon enrollment in FSS and (ii) the household's average annual earnings during the time period between enrollment in FSS and the most recent annual reexamination of income available in PIC.

Controlling for FSS Program Model and Earnings Fluctuations: HUD selects the higher of the two measures for each household in order to accommodate different approaches to implementing FSS while also correcting for variations in year-to-year earnings, which can be volatile for low-income households. Some PHAs encourage FSS participants to focus immediately on increasing their earnings, while others encourage FSS participants to focus on obtaining education and building skills first and then seek a higher paying job once they have stronger credentials. Other agencies use both approaches, tailoring the approach to each individual. Measure 1, Earnings Growth Since Enrollment, accommodates programs that encourage participants to focus first on education and training, while both measures work acceptably for programs that encourage individuals to increase their earnings immediately. Measure 2, Average Annual Earnings While in FSS, focuses on the difference between starting and average annual earnings, which ensures that an FSS participant who has made good progress in increasing earnings while in FSS, but who nevertheless has experienced a temporary setback of job loss as of the most recent annual reexamination, nevertheless has his or her progress recognized. For each household, the Earnings Performance Measure focuses on the higher of the two measures, maximizing HUD's ability to recognize households' progress toward increased earnings while participating in FSS.

Controlling for Local Economic Conditions: Because economic conditions vary from one community to the next, HUD has built in a mechanism to control for these differences. HUD adjusts for local economic conditions by comparing the average earnings growth of FSS participants at a PHA to the average earnings growth for nonparticipants with similar characteristics at the same PHA. The difference in performance between the two groups represents the Earnings Performance Measure for that PHA. Since the earnings of non-FSS participants would be expected to grow faster at PHAs located in stronger job markets than in PHAs located in weaker job markets, this comparison helps to account for differences in local economic conditions, which facilitates a meaningful comparison of earnings growth across FSS programs. Specifically, to calculate an Earnings Performance Measure for each PHA, HUD:

• Selects three comparison households for each FSS household based on the extent to which the comparison households are similar to the FSS household on the following characteristics: Earnings as of the time of the FSS household's entry into FSS, age of head of household, length of time in the voucher or public housing program, number of adults in the household and number of children under age 5.
• Calculates the earnings growth for all of the comparison households using the same approach used to calculate the earnings growth for FSS households, with the FSS household's enrollment date being applied to its comparison households for purposes of calculating the comparison households' initial earnings.
• Calculates the difference between the average earnings growth for all FSS participants and the average earnings growth for all comparison households at each PHA. The difference between the two represents the PHA's earnings performance measure.

HUD applies this measure to all FSS participants with a head of household who is neither elderly nor a person with disabilities who joined FSS between 3.5 and 7.5 years prior to the end of the quarter of the PIC extract used to calculate the score. For example, if the most recent PIC data extract ended in March 31, 2017, HUD's calculation of earnings performance measures would focus on FSS participants who joined the FSS program between October 1, 2009 and September 30, 2013. This methodology aggregates information for four years of FSS entrants in order to generate a large enough sample to analyze. The methodology does not examine data for participants that have entered the FSS program more recently than 3.5 years ago to allow sufficient time to have passed for FSS participants to have benefitted from the program. At the same time, the methodology does not focus only on an older sample of FSS participants to ensure that the results reflect recent FSS program performance to the maximum extent practicable.

Technical note: In measuring earnings growth, the methodology focuses solely on earnings determined through annual reexaminations, disregarding the results of any interim reexaminations. The reason for doing this is that not all PHAs require interim reexaminations of income when earnings rise in between annual reexaminations. To ensure an apples-to-apples comparison of earnings growth across PHAs, HUD focuses only on annual reexaminations. An annual progress report is required for every FSS participant regardless of the spacing of rental re-examinations, so PHAs involved in rent reform demonstrations would be included in this scoring.

B. Graduation Rate Calculation

This measure examines the share of FSS participants at each PHA who have “graduated” from the FSS program. It is calculated based on the graduation rate of FSS participants who entered each PHA's FSS program 5 to 8 years before the end of the most recent quarter of available PIC data. The methodology focuses on these households to allow sufficient time for most of the FSS participants who will graduate to have done so. HUD considered focusing on an older cohort to capture 100 percent of the FSS participants who will graduate, but HUD determined that it was more advantageous for the period analyzed to include more recent performance by the PHA.

Controlling for Turnover Rates: Turnover rates at PHAs can vary significantly for reasons unrelated to FSS. To avoid penalizing programs with higher turnover, HUD excludes non-graduating FSS participants who exited the Housing Choice Voucher (HCV) or Public Housing programs before the end of the analysis period from both the numerator and the denominator in calculating the Graduation Rate.

C. Participation Rate Calculation

The Participation Rate is the ratio of the number of FSS participants being served to the minimum number expected to be served under the standards used for awarding funding under the FSS NOFA. Agencies that exactly meet the standard will have a ratio of 1.0. Agencies that serve more than the required number will have a ratio above 1.0. Agencies that serve fewer than the required number will have a ratio below 1.0.

To calculate the Participation Rate, HUD first calculates the minimum number of FSS participants that HUD expects each PHA to serve for each of the most recent three (3) fiscal years for which both funding award and number served data are available. HUD calculates this number based on the guidelines in the NOFA and the number of coordinators funded in each agency during each year. HUD then sums the number of FSS participants actually served in each of the three years based on PIC data. Finally, HUD divides the total number of FSS participants served in each PHA by the total minimum number expected for the PHA's HUD-funded coordinator positions to determine the participation rate. If funding is only awarded to the PHA in one or two of the three years, the measure only uses data for the years for which funding was awarded. Note that this metric, while similar, is different from the “number of participants served,” which has been used in NOFA competitions and assesses only the most recent period of performance.

Controlling for Annual Variation and PIC Reporting: HUD also separately calculates the Participation Rate for the most recent year and then grades a PHA's Participation Rate based on the higher of: (a) The PHA's three-year average and (b) the most recent year. Looking at the higher of the these two values allows HUD to use the most recent available data for PHAs that have made progress in increasing the number served while avoiding penalizing PHAs for the results of an atypical year. It also ensures that PHAs that have improved the quality of their PIC reporting on FSS participation can be judged based on the FSS participant counts derived from recent PIC reports, rather than from reports submitted in earlier years. Given the new guidance that HUD issued on PIC reporting for FSS on May 16, 2016 (PIH Notice 2016-08), HUD expects the quality of FSS reporting to PIC to be improved going forward and reminds PHAs of the importance of ensuring accurate and timely submissions of FSS Addendums to PIC.

As calculated using the procedures described above, the participation rate is higher if the PHA has served more participants relative to its funding level. The ratio required in the NOFA is 25 for one full-time coordinator and 50 for each additional full-time coordinator. For example, a PHA with 1 funded full-time coordinator is expected to serve at least 25 participants during the year, while a PHA with 3 funded full-time coordinators is expected to serve at least 125 participants. If the PHA with 1 coordinator serves 40 FSS participants (much more than the minimum required) and the PHA with 3 coordinators serves 130 participants (only slightly more than the minimum expected), the PHA with the smaller number of coordinators and participants will have a higher participation rate (40/25 = 1.60 versus 130/125 = 1.04).

PHAs that receive funding jointly with other PHAs are evaluated together in calculating the participation rate. HUD sums the number of FSS participants served by each of the jointly-funded agencies and the minimum number of participants the agencies are jointly expected to serve and provides the same participation score for each of the PHAs.

III. How will HUD convert the measures into an FSS Performance Score?

After making the calculations described above, HUD will develop an FSS Performance Score for each PHA using a two-step process.

A. Step One: Assigning Scores to Each of the Three Measures

In Step One, HUD will assign a score of 0 to 10 to each PHA's FSS program for each of the three measures. Scores will be assigned using the procedures described below. The ranges for awarding points between two values include those values as well as all intermediary values.

For each of the three measures, HUD has selected criteria for evaluating PHA performance. For each measure, the highest performers are assigned a score of 10, the next-highest performers are assigned a score of 7.5, and low performers are assigned a score of 0. HUD will award a score of 5 to PHAs whose performance does not satisfy the criteria for highest, next-highest, or low performance for that measure.

1. Earnings Performance Measure (50 Percent of Final Score)

• 10 points: Earnings performance measure of $6,400 or higher.
• 7.5 points: Earnings performance measure between $4,750 and $6,399.
• 0 points: Earnings performance measure below $1,500 and a p-value of .10 on a statistical test measuring the likelihood that a PHA's earnings performance measure is significantly lower than the median measure of $3,418 (see below for an explanation of this statistical test).
• 5 points: All PHAs that do not qualify for a 10, 7.5, or a 0.

As described above, a PHA's earnings performance measure represents the difference between: (a) The average earnings growth for FSS participants and (b) the average earnings growth for comparison households at the same PHA. A PHA's earnings performance measure is not simply a measure of the extent to which FSS participants increased their earnings. Instead, a PHA's earnings performance measure reflects the relative growth of FSS participants relative to a matched set of non-participants at that PHA. HUD assigns a higher score to FSS programs that achieve a higher earnings performance score.

In addition to focusing on the size of the earnings performance measure, the scoring for this measure applies a one-tailed test of statistical significance, designed to protect FSS programs from being scored “low performer” due to random variation and low sample size. For example, without this protection, an individual FSS program may include several anomalous participants or control households that skew research results. The statistical test measures the likelihood that a PHA's earnings performance measure is significantly lower than the median measure. The lower the p-value, the less likely it is that a PHA received a below-median earnings performance measure due to random variation. To receive 0 points, a PHA must not only have an earnings performance measure below $1,500 but also a p-value on this test of less than .10, which means there is at least a 90 percent probability that the earnings performance measure is truly below the median value of $3,418.

While a similar statistical test could theoretically be applied to help identify high performing programs, such a test would make it harder for small FSS programs to qualify. To avoid disadvantaging smaller FSS programs, p-values are not considered in determining whether to award 10 or 7.5 points.

2. Graduation Rate (30 Percent of Final Score)

• 10 points: Graduation rate of 38 percent or higher.
• 7.5 points: Graduation rate between 27 percent and 37.99 percent.
• 0 points: Graduation rate of 8 percent or lower.
• 5 points: All PHAs that do not qualify for a 10, 7.5, or a 0.

Under this approach, a higher graduation rate results in a higher score.

3. Participation Rate (20 Percent of Final Score)

• 10 points: Participation rate of 2.1 or higher.
• 7.5 points: Participation rate between 1.7 and 2.09.
• 0 points: Participation rate of 0.95 or lower.
• 5 points: All PHAs that do not qualify for a 10, 7.5, or a 0.

Under this approach, a higher participation rate results in a higher score.

Step Two: Developing the Final FSS Performance Score and Grade

After computing individual scores for each of the three measures, HUD aggregates each PHA's scores using the weights noted above to develop a final FSS Performance Score from 0 to 10. Based on this score, HUD assigns the following ranking to the PHA's performance:

• Excellent: FSS Performance score of 7.25 or higher.
• Standard: FSS Performance score between 4.0 and 7.24.
• Low: FSS Performance score between 3.00 and 3.99.
• Troubled: FSS Performance score of less than 3.00.

IV. How were these thresholds selected?

The thresholds for converting the three performance measures into scores in step one are fixed and will now apply to all future years until HUD revises the methodology. These thresholds were selected by applying the FSS Performance Score methodology to PIC data from the quarter ending December 31, 2016. The thresholds were selected as follows:

1. Earnings Performance Measure (50 Percent of Final Score)

• The threshold for awarding a score of 10 points (an earnings performance measure of $6,400) represents approximately the 80th percentile of the distribution of results of the earnings performance measure for PHAs whose measures have a p value >.10 on a statistical test measuring the likelihood that the earnings performance measure is different from $0. HUD calculated the distribution using agencies that receive a p value below .10 on this test to reduce the likelihood that the results would be affected by random variation.
• The threshold for awarding a score of 7.5 points ($4,750) represents approximately the 60th percentile of the distribution of results of the earnings performance measure for PHAs whose measures have a p value <.10 on the statistical test described above.
• The threshold for awarding a score of 0 points ($1,500) represents approximately the 20th percentile of the distribution of results of the earnings performance measure for all PHAs.

2. Graduation Rate (30 Percent of Final Score)

• The threshold for awarding a score of 10 points represents approximately the 80th percentile of the distribution of graduation rates.
• The threshold for awarding a score of 7.5 points represents approximately the 60th percentile of the distribution of graduation rates.
• The threshold for awarding a score of 0 points represents approximately the 20th percentile of the distribution of graduation rates.

3. Participation Rate (20 Percent of Final Score)

• The threshold for awarding a score of 10 points represents approximately the 80th percentile of the distribution of participation rates.
• The threshold for awarding a score of 7.5 points represents approximately the 60th percentile of the distribution of participation rates.
• The threshold for awarding a score of 0 points is 0.95, which falls below the minimum standard established by HUD. A PHA serving the minimum number of FSS participants required to obtain FSS funding would normally have a participation rate of 1.0. However, this methodology uses a score of 0.95 to give PHAs the benefit of the doubt and account for any temporary vacancies in the FSS program.

4. Composite FSS Performance Scores and Grades

• The threshold for awarding a ranking of Excellent represents approximately the 80th percentile of the distribution of FSS Performance Scores.
• The range for awarding a ranking of Low represents approximately the 10th through the 20th percentiles in the distribution of FSS Performance Scores.
• Programs falling below approximately the 10th percentile in the distribution of FSS Performance Scores are classified as Troubled.
• All other FSS programs are classified as “Standard” performers. The range for awarding a ranking of Standard represents approximately the 20th through the 80th percentiles of the distribution of FSS Performance Scores.

As noted above, all thresholds are now fixed and will not be recalculated each year. This will facilitate tracking individual PHA progress as well as that of all FSS programs over time. Further, this framework does not limit how many programs can receive any particular ranking. The thresholds are absolute, not relative.

V. What else do PHAs need to know about the FSS performance score methodology?

The following is additional information about how HUD calculates FSS performance scores:

1. For households entering FSS more than one time during the analysis period, the methodology focuses only on the FSS Contract of Participation that began 5 to 8 years before the end of the most recent quarter of available PIC data to calculate the FSS performance score. This facilitates appropriate evaluation of each program's graduation rate, which focuses on the same group of households. If a participant entered more than once during that period, the methodology focuses on the older entry.

2. FSS performance scores are calculated for any PHA that has sufficient data in PIC to calculate at least one of the three measures used to calculate the score. If there are insufficient data to calculate one or two of the measures, that PHA will receive a middle (standard) score of “5” for the missing measure(s) before calculating the FSS performance score.

3. A PHA for which none of the three scores are available will not receive a score.

4. Because the earnings performance measure and the graduation rate are calculated using data that spans a range of years, it will take time for a PHA to improve its FSS Performance Score through improvements in earnings and graduation outcomes. However, improvements in these areas will eventually become apparent in a PHA's FSS Performance Score. It is important for PHAs with low scores to begin implementing improvements as quickly as possible. PHAs with participation rates below 0.95 can quickly improve their FSS Performance Scores by increasing participation rates to meet HUD's minimum requirements.

VI. How will HUD assess the performance of FSS programs that do not receive funding?

HUD is interested in evaluating the performance of all FSS programs administered by PHAs, including programs that do not receive funding from HUD. However, there are several concerns with applying the methodology described above to the evaluation of the performance of non-funded agencies. First, the participation rate cannot be calculated using the methodology described in this notice because there are no set expectations for program size. Second, such programs tend to be smaller than NOFA-funded programs, which means their results are more subject to random variation due to the participation of individuals with idiosyncratic features. Third, these program participants tend to receive less personal attention from FSS coordinators due to the lack of dedicated funding from HUD for FSS.

HUD will continue studying options for measuring the performance of such agencies to determine if an approach can be developed for evaluating the quality of their FSS programs. To inform HUD's analysis of this issue, HUD requests comments on the following questions:

1. Should HUD evaluate FSS programs that do not receive funding from HUD?

2. Should the performance of an unfunded FSS program be considered by HUD in determining whether to award funding? If not, what factors should be used in determining whether to award funding to a currently unfunded agency?

3. Should the FSS performance score of an unfunded PHA be compared solely with that of other unfunded PHAs or also against the performance of funded agencies?

4. How should the procedures for evaluating the performance of funded FSS programs be adapted for purposes of measuring the performance of FSS programs that do not receive funding?

5. Should HUD calculate a participation rate for unfunded FSS programs in evaluating their performance and if so, how should it be calculated?

6. In addition to, or instead of a participation rate, should HUD limit the evaluation of non-funded agencies to FSS programs over a certain size, such as 15 or 25 participants? Focusing only on FSS programs of a certain minimum size should help to improve the reliability of the evaluation results while also focusing the evaluation (and any corresponding preference for funding) on PHAs that demonstrate a threshold level of commitment to the FSS program.

VII. Other Questions

In addition to the questions noted above, HUD requests feedback on the following questions:

1. Has HUD assigned the appropriate weight to each of the three measures? The proposed system uses the following weights: Earnings performance measure (50 percent); Graduation rate (30 percent); and Participation rate (20 percent).

2. In evaluating earnings growth, HUD focuses on the average of the earnings growth of individual households at a PHA, rather than median growth. HUD takes this approach to recognize the potential life-changing impacts of helping individuals move from unemployment to high-paying jobs. Such impacts are captured in looking at average earnings growth, but might be missed in looking only at the median growth. It is appropriate in this context to use averages, or should HUD switch to medians instead?

3. Has HUD adequately accounted for variations in local economic conditions? If not, what further adjustments should be made? The earnings performance measure accounts for local economic conditions by comparing the earnings growth for FSS participants at a PHA to the earnings growth for non-FSS participants at the same PHA with similar characteristics. The assumption underlying this approach is that earnings growth for non-FSS participants will be higher in areas with stronger job markets than in areas with weaker job markets. To attain the same earnings performance measure, a PHA in an area with a strong job market would thus need to demonstrate a higher level of earnings growth among FSS participants than would a PHA in an area with a weaker job market. After calculating the difference between earnings growth for FSS and non-FSS participants at a PHA, the proposed system makes no further adjustments. Should HUD further adjust its system to account for variations in local economic conditions, and if so, how should HUD make this adjustment? For example, HUD could divide the earnings performance measure by the average starting earnings for a PHA's FSS participants and then compare the resulting percentages across PHAs. Further, HUD could adjust the earnings performance measures by an index that accounts for local economic conditions.

4. HUD currently allows a PHA to count FSS participants living in multifamily FSS programs toward the minimum number of participants required to be served in order to qualify for FSS funding. The PIC data system, however, does not capture information on multifamily FSS participants. HUD requests suggestions on how best to capture information on multifamily FSS participants being served by a PHA's FSS coordinator to determine a PHA's participation rate.

5. HUD currently permits, and funds, FSS programs in Tribes and Tribally Designated Housing Entities (TDHEs). However, Tribes and TDHEs do not report into the PIC data system. HUD requests suggestions on how to best capture information on tribal FSS participants to determine a score.

6. HUD currently permits, and funds, FSS programs at MTW agencies. However, MTW agencies are only required to report select FSS data fields into the PIC system. HUD requests suggestions on how to best capture information on MTW FSS participants to determine a score.

7. How should HUD evaluate FSS programs offered by HUD-assisted multifamily properties with Section 8 contracts? These programs are very new and currently submit quarterly spreadsheets rather than an FSS addendum integrated into a HUD data reporting system.

VIII. Environmental Impact

This notice does not direct, provide for assistance or loan and mortgage insurance for, or otherwise govern or regulate, real property acquisition, disposition, leasing, rehabilitation, alteration, demolition, or new construction, or establish, revise or provide for standards for construction or construction materials, manufactured housing, or occupancy. Accordingly, under 24 CFR 50.19(c)(1), this notice is categorically excluded from environmental review under the National Environmental Policy Act of 1969 (42 U.S.C. 4321).