FMCSA Making Progress as It Evaluates Possible CSA Changes, but Concerns Linger
Federal Motor Carrier Safety Administration officials held their first public meeting Aug. 30 to drill down on portions of their action plan to evaluate possible sweeping future changes to the Compliance, Safety, Accountability program’s Safety Measurement System used to identify motor carriers at high risk for future crashes.
While motor carriers and trade organizations exhibited a spirit of cooperation at the meeting, they expressed a number of concerns about a plan being considered by the agency to adopt what is called an “Item Response Theory” data method recommended by an elite academic panel appointed by the National Academy of Sciences to retool the agency’s CSA program data methodology.
One year into its evaluation of a two-year planned study of the IRT method, the agency said it was making progress but that it was too soon to say if the new methodology could improve on the current CSA program.
Joe DeLorenzo, director of the agency’s Office of Enforcement and Compliance, said the agency expects to have the IRT modeling evaluation completed by the fall of 2019. After that, he said the agency will have to decide whether it finds the method adaptable to regulating the safety of heavy trucks and buses.
I really can’t overstate how important this process is, and what comes out of the process is important to FMCSA in terms of how it will actually impact on safety when it is done.
— FMCSA Administrator Ray Martinez
The agency plans to run small-scale testing of data using the IRT method early next year, DeLorenzo told Transport Topics. “Come the beginning of next year when we run some models, I think we’ll have a better feel for exactly what it’s going to mean in terms of any change to SMS and when those would occur,” he said.
However, there were serious concerns raised by stakeholders at the meeting that centered on how to improve the quality of data collected, how to collect additional data in areas such as vehicle miles traveled and driver turnover and pay, and how to make the agency’s data more transparent and user-friendly.
The meeting was intended to solicit questions and hear concerns from commercial motor carriers and trade organizations in a more detailed look at three of six change areas that were recommended by the in-depth congressionally-mandated June 2017 study conducted by a national academy panel.
The academy panel was charged with analyzing the ability of the SMS to discriminate between low- and high-risk carriers, assess the public usage of SMS, review the data and methodology used to calculate the measures and provide advice on additional data collection and safety assessment methodologies.
Although carriers say the CSA program has raised the industry’s safety consciousness, it has been an ongoing target of criticism from many motor carriers, the Government Accountability Office and Department of Transportation Inspector General since it was first implemented in December 2010.
The complex IRT method takes a “statistically principled approach” — that is, a detailed data-oriented method that digs deeper and measures the performance of individual trucks and buses, not just at the motor carrier level.
The IRT approach has a long history in the United States, including use in commercial air and health data research, but has never been used in the commercial motor vehicle sector.
The meeting was kicked off by FMCSA Administrator Ray Martinez, who said the IRT project is critical to the agency’s safety mission.
“I really can’t overstate how important this process is, and what comes out of the process is important to FMCSA in terms of how it will actually impact on safety when it is done,” Martinez said. “But also how it impacts the credibility of our agency among the regulated community.”
But based on the comments at the meeting, the agency could face challenges in obtaining more accurate data such as vehicle miles traveled, driver pay and method of pay, type of cargo being hauled and driver turnover from motor carriers.
Horvath
For starters, there are many motor carriers who are not willing to hand over such data to the agency for competitive reasons, according to Dan Horvath, vice president of safety policy for American Trucking Associations.
“We don’t think from the driver compensation perspective that it is reflective of a carrier’s safety, or reliable to judge a carrier on that,” Horvath said. “As far as other questions such as cargo, driver turnover rate, commodities, or other information, there’s definitely a hesitation among carriers out there from providing this even at a vague level.”
Irwin Shires, government qualification coordinator for Panther Premium Logistics, noted that using driver turnover rates to measure a carrier’s safety is not necessarily reliable for a safety rating.
“A carrier adopting a proactive culture of safety is what they [regulators] want us to do,” Shires said. “If I have methods in place now, which I do, to identify potential bad eggs in terms of speeding and unsafe characters and I get them out of my fleet before they cause an accident, I still have to replace the driver. I would submit that that’s a kind of positive turnover.”
David Yessen, an official with the agency’s enforcement division, said obtaining some information from carriers is potentially problematic.
“If there’s information that we’re going to require be submitted, then that’s going to require some type of rulemaking,” Yessen said. “If it’s voluntary then the issue becomes what if we don’t get it from everybody.”
There also was concern expressed over uneven enforcement in the states.
“There is certainly enforcement disparity out there with states and jurisdictions, and that’s something that definitely needs to be taken in account,” Horvath said.
“I wanted to emphasize that IRT will not fix flawed data,” said Mike Daniels, an expert from the academy of sciences acting as an advisor to the agency on the IRT model. “The data quality and transition to the IRT model are two important, but distinct issues.”
