Landmark CSA Study Nears
This story appears in the June 5 print edition of Transport Topics.
A comprehensive National Academy of Sciences study of the federal Compliance, Safety, Accountability program likely will recommend sweeping changes to the program that is slated to play a significant role in how federal regulators assign motor carrier safety ratings, according to executives who have been following public proceedings about the research.
A panel of 12 elite researchers, working on behalf of the academy, has been assessing the quality and accuracy of the Federal Motor Carrier Safety Administration’s CSA safety measurement system data as well as algorithms intended to identify carriers at the highest risk for crashes. They also have analyzed how the agency scores violations used in the program’s Behavioral Analysis and Safety Improvement Categories (BASICs).
The study results could be made public later this month.
The panel has listened to dozens of presentations over the past year from regulators, academics, data analysts, commercial motor carriers representing every segment of the industry and other experts.
Data problems and data sufficiency gaps have been at the center of controversy among truckers and motorcoach operators.
An FMCSA spokesman did not reply to a request for comment by press time.
But a spokeswoman for the academy last week told Transport Topics that the study has been drafted and is undergoing a peer review. Once the review is completed, FMCSA, which is paying for the study, will have an opportunity to see it and be briefed by the academy before it is made public.
The study was a mandate of the Fixing America’s Surface Transportation (FAST) Act signed into law Dec. 4, 2015, and calls for the academy to submit its final study to congressional committees and the U.S. Department of Transportation Inspector General 18 months after legislation is enacted. After that, it gives FMCSA 120 days to come up with a corrective action plan.
“I’m confident that the study will vindicate what ATA has long believed, which is the CSA system is in need of significant repair,” said Sean Garney, director of safety policy for American Trucking Associations.
In March, FMCSA withdrew its proposed safety fitness determination rule that was based largely on CSA data, saying it would await the results of the academy study before proceeding with a supplemental rule.
Dan Murray, the American Transportation Research Institute’s vice president for research, recalled research panel members asking “extremely pointed questions” during his presentation to the group last summer.
“They very clearly were trying to dig into what’s behind the BASICs and whether that’s credible,” Murray told TT. “That is, do the BASICs really portend a safety outcome? Why would we collect and measure certain safety data points or incorporate them into a BASIC if there’s no nexus to safety?”
Murray added, “It’s inconceivable to think that the report wouldn’t focus on what does CSA really do for safety outcomes. If it’s not generating safety outcomes, then it needs an overhaul.”
Steve Bryan, CEO of Vigillo, a safety data analysis firm for fleets, attended all three of the panel’s public meetings. He said he’s optimistic that the study recommendations could greatly improve the CSA program.
“There were 12 people on this panel. All of them are Ph.D.s, statisticians and very smart people. They come from every kind of discipline and many of the best universities in the country,” Bryan told TT.
The panel was co-chaired by Joel Greenhouse, a professor in the Department of Statistics at Carnegie Mellon University; and health care policy professor Sharon-Lise Normand of the Harvard Medical School.
“From what I detected throughout the year as these meetings unfolded was a kind of a dawning of awareness that ‘man, this thing has got some problems,’?” Bryan said. “There was a lot of defensive posturing from various staff people at the FMCSA throughout this.”
But at the midpoint of the meeting schedule, Bryan said he saw the committee tell the agency to “stop defending the indefensible and let’s really get into what it would take to fix this thing.”
Bryan has criticized the CSA program for a lack of enough data, primarily collected at roadside inspections, as well as the agency’s seeming reluctance to remove crashes from carriers’ safety profiles when they are not at fault.
The program, implemented in late 2010, also has been the subject of critical research by the Government Accountability Office.
In an October follow-up report to an earlier study, GAO said FMCSA had taken action to improve the effectiveness and efficiency of CSA program interventions for high-risk carriers but still lacked measures to monitor progress.
“However, without improving the quality of information that FMCSA uses to evaluate its performance, the agency will continue to lack the information it needs to determine the extent to which it is achieving these fundamental programmatic improvements,” GAO said.
