Fixing America's Surface Transportation Act Correlation Study

AGENCY:

Federal Motor Carrier Safety Administration (FMCSA), DOT.

ACTION:

Notice; request for comments.

SUMMARY:

On June 27, 2017, the National Academy of Sciences (NAS) published its report titled, “Improving Motor Carrier Safety Measurement.” This report was commissioned by FMCSA consistent with the requirements of Section 5221 of the Fixing America's Surface Transportation (FAST) Act. The FAST Act also requires that the Agency develop an action plan to address any identified deficiencies and submit it to Congress and the U.S. Department of Transportation's (DOT) Office of Inspector General (OIG). The purpose of this notice is to announce a public meeting to discuss the NAS recommendations and to solicit input to be considered by the Agency in the development and implementation of the action plan.

DATES:

The public meeting will take place on Friday, September 8, 2017, from 1:00 p.m. to 4:00 p.m., Eastern Time. A copy of the agenda for the meeting will be available in advance of the meeting at https://www.regonline.com/FMCSA_Correlation_Study_Action_PlanPublicMeeting. If all interested participants have had an opportunity to comment, the meeting may conclude early.

Public Comments: Comments must be received by September 27, 2017.

ADDRESSES:

The meeting will be held at the FMCSA National Training Center, 1310 N. Courthouse Road, Suite 600, Arlington, VA 22201-2508. Those interested in attending this public meeting must register at: https://www.regonline.com/FMCSA_Correlation_Study_Action_PlanPublicMeeting. Participants have the option of registering to attend in person, or via webinar.

You may submit comments identified by Docket Number FMCSA-2017-0226 using any of the following methods:

• Federal eRulemaking Portal: http://www.regulations.gov. Follow the online instructions for submitting comments.
• Mail: Docket Management Facility, U.S. Department of Transportation, Room W12-140, 1200 New Jersey Avenue SE., Washington, DC 20590-0001.
• Hand Delivery or Courier: West Building, Ground Floor, Room W12-140, 1200 New Jersey Avenue SE., Washington, DC, between 9 a.m. and 5 p.m. E.T., Monday through Friday, except Federal holidays.
• Fax: 1-202-493-2251.

FOR FURTHER INFORMATION CONTACT:

For information about the public meeting or for information on facilities or services for individuals with disabilities or to request special assistance at the meeting, contact Ms. Sharon Worthy, Director of External Affairs at (202) 366-2309 or by email at Sharon.Worthy@dot.gov, by September 1, 2017.

For information about the Correlation Study, please contact Ms. Theresa Rowlett, Senior Policy Advisor, Office of Enforcement, FMCSA, 1200 New Jersey Avenue SE., Washington, DC 20590, Telephone (202) 360-2924 or by email at Theresa.Rowlett@dot.gov.

If you have questions regarding viewing or submitting material to the docket, contact Docket Services, telephone (202) 366-9826.

SUPPLEMENTARY INFORMATION:

Submitting Comments

If you submit a comment, please include the docket number for this notice (FMCSA-2017-0226), indicate the specific section of this document to which each comment applies, and provide a reason for each suggestion or recommendation. You may submit your comments and material online or by fax, mail, or hand delivery, but please use only one of these means. FMCSA recommends that you include your name and mailing address, an email address, or a phone number in the body of your document so that FMCSA can contact you if there are questions regarding your submission.

To submit your comment online, go to http://www.regulations.gov,, put the docket number, FMCSA-2017-0226, in the keyword box, and click “Search.” When the new screen appears, click on the “Comment Now!” button and type your comment into the text box on the following screen. Choose whether you are submitting your comment as an individual or on behalf of a third party and then submit.

If you submit your comments by mail or hand delivery, submit them in an unbound format, no larger than 81/2 by 11 inches, suitable for copying and electronic filing. If you submit comments by mail and would like to know that they reached the facility, please enclose a stamped, self-addressed postcard or envelope.

Viewing Comments and Documents

To view comments, as well as any documents mentioned in this preamble as being available in the docket, to http://www.regulations.gov. Insert the docket number, FMCSA-2017-0226, in the keyword box, and click “Search.” Next, click the “Open Docket Folder” button and choose the document to review. If you do not have access to the Internet, you may view the docket by visiting the Docket Management Facility in Room W12-140 on the ground floor of the West Building, 1200 New Jersey Avenue SE., Washington, DC 20590, between 9 a.m. and 5 p.m., E.T., Monday through Friday, except Federal holidays.

Privacy Act

The Department of Transportation (DOT) solicits comments from the public to better inform its decision-making processes. DOT posts these comments, without edit, including any personal information the commenter provides, to www.regulations.gov,, as described in the system of records notice (DOT/ALL-14 FDMS), which can be reviewed at www.dot.gov/privacy.

Background

Section 5221 of the FAST Act, titled “Correlation Study,” required FMCSA to commission the National Research Council of the National Academies to conduct a study of FMCSA's Compliance, Safety, Accountability (CSA) program and Safety Measurement System (SMS). SMS is FMCSA's algorithm for identifying patterns of non-compliance and prioritizing motor carriers for interventions. FMCSA is prohibited from publishing SMS percentiles and alerts on the SMS Web site for motor carriers transporting property until the NAS Correlation Study is complete and all reporting and certification requirements under the FAST Act are satisfied.

The FAST Act also required FMCSA to submit the results of this study to both Congress and the DOT OIG. In addition, within 120 days of the submission of the report to Congress and the OIG, FMCSA must submit an action plan to the Senate Committee on Commerce, Science, and Transportation; and the House of Representatives Transportation and Infrastructure Committee. The OIG is required to review the action plan and submit a report to Congress on the responsiveness of the FMCSA's plan to the NAS report's recommendations.

Under Section 5221 of the FAST Act, the purpose of this study is to analyze:

a. The accuracy with which the Behavior Analysis Safety Improvement Categories (BASICs) used by SMS:

i. Identify high risk carriers.

ii. Predict or are correlated with future crash risk, crash severity, or other safety indicators for motor carriers, including the highest risk carriers.

b. The methodology used to calculate BASIC percentiles and identify carriers for enforcement, including the weights assigned to particular violations and the tie between crash risk and specific regulatory violations, with respect to accurately identifying and predicting future crash risk for motor carriers.

c. The relative value of inspection information and roadside enforcement data.

d. Any data collection gaps or data sufficiency problems that may exist and the impact of those gaps and problems on the efficacy of the CSA program.

e. The accuracy of safety data, including the use of crash data from crashes in which a motor carrier was free from fault.

f. Whether BASIC percentiles for motor carriers of passengers should be calculated separately from for motor carriers of freight.

g. The differences in the rates at which safety violations are reported to FMCSA for inclusion in the SMS by various enforcement authorities, including States, territories, and Federal inspectors.

h. How members of the public use the SMS and what effect making the SMS information public has had on reducing crashes and eliminating unsafe motor carriers from the industry.

The FAST Act required the NAS also to consider:

a. Whether the SMS provides comparable precision and confidence, through SMS alerts and percentiles, for the relative crash risk of individual large and small motor carriers.

b. Whether alternatives to the SMS would identify high risk carriers more accurately.

c. The recommendations and findings of the Comptroller General of the United States and the Inspector General of the Department of Transportation, and independent review team reports, issued before the date of the FAST Act.

NAS Report Recommendations and FMCSA Action Plan Overview

On June 27, 2017, NAS published the report titled, “Improving Motor Carrier Safety Measurement.” The report is available at https://www.nap.edu/catalog/24818/improving-motor-carrier-safety-measurement. A copy of the report has been placed in the docket referenced at the beginning of this notice. In preparing the report, NAS collected and analyzed all the quantitative data available to FMCSA in its databases, which contain information on the safety of commercial motor carriers and drivers subject to the Federal Motor Carrier Safety Regulations and the Hazardous Materials Regulations. In addition, NAS held three public meetings to engage stakeholders from the truck and bus industry, safety advocates, researchers, and other government organizations. The meeting agendas are included in an appendix to the report.

The NAS report concluded that SMS, in its current form, is structured in a reasonable way and its method of identifying motor carriers for alert status is defensible. In addition, NAS agreed that FMCSA's overall approach, based on crash prevention rather than prediction, is sound. NAS provided FMCSA with six recommendations to improve the system.

FMCSA accepts the NAS report's recommendations and outlines below several high-level proposals to address each recommendation. The proposals summarized below are intended to allow the public to provide input into the development of the action plan but do not themselves constitute the entirety of the action plan. FMCSA is still considering and evaluating actions to address the recommendations. FMCSA is also working with the NAS to establish a Standing Committee to oversee and provide advice relating to the Agency's work addressing these recommendations. In addition to reviewing and providing advice on the Agency's technical work, such as the Item Response Theory (IRT) modeling, NAS will advise on all recommendations, and establish a process for gathering stakeholder input in the implementation of the action plan as well.

Recommendation 1—FMCSA should develop the suggested IRT model over the next 2 years. If it is then demonstrated to perform well in identifying motor carriers for alerts, FMCSA should use it to replace SMS in a manner akin to the way SMS replaced SafeStat.

FMCSA Comment

To address this recommendation, FMCSA is securing additional expertise and resources to develop and test the proposed IRT statistical model. The testing of an IRT model is consistent with FMCSA's continuous improvement process of modifying and testing changes to SMS by focusing on data quality, data collection, and transparency. FMCSA will evaluate whether the new model performs well using existing effectiveness testing methods and/or methodologies recommended by the NAS, and based on that evaluation will determine the next steps in using that model.

FMCSA is seeking comments on the implementation of an IRT model and its application to the SMS as well as the process for development and testing of the model.

Recommendation 2—FMCSA should continue to collaborate with states and other agencies to improve the quality of the Motor Carrier Management Information System (MCMIS) data in support of SMS, focusing on carrier exposure and crash data. The current exposure data are missing with high frequency, and data that are collected are likely of unsatisfactory quality. To improve the exposure data collected involves not only collecting higher-quality Vehicle Miles Traveled (VMT) data, but also collecting this information by state and by month. This will enable SMS to (partially) accommodate existing heterogeneity in the environments where carriers travel. Crash data are also missing too often. Also, there is information available from police reports currently not represented on MCMIS that could be helpful in understanding the contributing factors in a crash. Such information could help to validate the assumptions linking violations to crash frequency. To address these issues, FMCSA should support the states in collecting more complete crash data, and in universal adoption of the Model Minimum Uniform Crash Criteria (MMUCC), as well as developing and supplying the code needed to automatically extract the data needed for the MCMIS crash file.

FMCSA Comments on VMT

Regarding exposure data, the Agency agrees that more VMT data from motor carriers would reduce the need for FMCSA to use substitute values and would improve the quality of the data in SMS. FMCSA is concerned that access to monthly and by-State VMT is not currently feasible. Currently, FMCSA rules require carriers to provide updated VMT data only every two years. FMCSA previously considered using other sources of VMT data such as the International Registration Plan data. However, FMCSA does not currently have access to that data. And even if the Agency had access to IRP data, that would not provide a complete data set, as IRP carriers are not required to report information on vehicles with a gross vehicle weight rating of less than 26,000 pounds.

FMCSA seeks information, through this notice, on potential sources of improved VMT data. Additionally, FMCSA requests input from industry and other stakeholders about other available sources for this data and the costs and benefits of voluntary submission of the data.

FMCSA expects to consider the effect of monthly, voluntarily-submitted, State-by-State VMT data from motor carriers, and the impact on the system if this information were provided by only a portion of the regulated community.

FMCSA Comments on Crash Data

In accordance with Section 5306 of the FAST Act, FMCSA established a Federal Advisory Committee to review Post-Accident Reports for tow-away crashes involving FMCSA-regulated commercial motor vehicles. That committee was charged with recommending changes to improve the quality and consistency of Police Accident Reports (PARs) data. More than half of the committee members represented States or State law enforcement officials, with the remainder representing industry, labor, safety advocates, and other interested parties. The FAST Act directed the working group to review existing State PARs to recommend best practices for the collection of PARs data by State and local law enforcement agencies. The Post Accident Review Advisory Committee recommended that all States use the National Highway Traffic Safety Administration's MMUCC and that FMCSA modify its data systems to receive all MMUCC data from the States. In January 2017, FMCSA's and NHTSA's senior leadership agreed to establish a workgroup to carry out the Committee's recommendations.

FMCSA recognizes that implementation of this recommendation will take additional resources for the States for training and information technology system changes. We are partnering with NHTSA to encourage States to participate in a USDOT national crash repository (that uses the MMUCC guidance as a framework). In addition, FMCSA will be examining the quality of the EDT data and comparing it to what we receive in MCMIS.

As a result, FMCSA specifically asks for information on issues that should be considered FMCSA determines how to best integrate MMUCC data.

Recommendation 3—FMCSA should investigate ways of collecting data that will likely benefit the recommended methodology for safety assessment. This includes data on carrier characteristics—including information on driver turnover rate, type of cargo, method and level of compensation, and better information on exposure.

FMCSA Comments

The Agency agrees that additional information about carrier operations might improve the Agency's analysis and identification of non-compliant motor carriers. To confirm this, FMCSA will use the IRT model and simulate the impacts and value of driver turnover rates, type of cargo, method and level of compensation, and exposure in identifying unsafe motor carriers before proceeding with an information collection. In addition, FMCSA would conduct a cost benefit analysis to determine how much it would cost the industry to provide the Agency with this additional information. Through this notice, FMCSA is specifically seeking input on these and other data fields to be considered and information on the costs associated with submitting this information.

Recommendation 4—FMCSA should structure a user-friendly version of the MCMIS data file used as input to SMS without any personally identifiable information to facilitate its use by external parties, such as researchers, and by carriers. In addition, FMCSA should make user-friendly computer code used to compute SMS elements available to individuals in accordance with reproducibility and transparency guidelines.

FMCSA Comments

FMCSA agrees that there could be benefits from making MCMIS data available to researchers and carriers. Through this Federal Register notice, FMCSA seeks input on how the MCMIS data would be used by researchers and others to determine the best method(s) for providing the data file.

The Agency is considering developing the programming, screen shots, and preview capacity so that changes from one month to another are explained to motor carriers to help carriers understand the implications of violations and crashes on their SMS data. Input on the information that would be helpful in reviewing SMS data is requested through this notice.

Recommendation 5—FMCSA should undertake a study to better understand the statistical operating characteristics of the percentile ranks to support decisions regarding the usability of public scores.

FMCSA Comments

Like NAS, FMCSA has been unable to quantify the impacts to motor carriers of publicly displaying the SMS percentiles. The Agency has only anecdotal information about the business impacts of the public percentiles on the SMS Web site. Historically, insurance companies and shippers have been reluctant to share data on how safety data is used to determine rates. And, while the Agency has been contacted by companies advising that they lost business because of SMS, these claims have not been validated or assimilated into a usable analysis.

Through this notice, FMCSA seeks data from motor carriers, insurance companies, and shippers regarding the impacts of the public display of SMS percentiles and alerts on businesses. This information will be used to identify next steps for this recommendation.

Recommendation 6—Given that there are good reasons for both an absolute and a relative metric on safety performance, FMCSA should decide on the carriers that receive SMS alerts using both the SMS percentile ranks and the SMS measures, and the percentile ranks should be computed both conditionally within safety event groups and over all motor carriers.

FMCSA Comments

The Agency has heard from motor carriers with increased measures or percentiles due to an increase in vehicles or clean inspections. Analysis of the number of carriers that received higher measures and percentiles without a violation or crash indicates this happens to a very small number of carriers. However, FMCSA agrees that the methodology should be revised so that a safety event that is not a violation or a crash is not the sole reason for an increased measure or percentile.

In addition, FMCSA anticipates investigating the use of a hybrid measure that combines relative and absolute metrics as part of its development of the IRT model. FMCSA seeks comment from stakeholders on this issue, how it could be implemented, and when such changes would be appropriate.