Maximize Field Operational Test (FOT) success by creating a sound experimental design and data acquisition plan.

Experience from a Field Operational Test (FOT) in Michigan to evaluate a Roll Advisor and Control (RA&C) system for commercial vehicles.

Date Posted
03/09/2010
TwitterLinkedInFacebook
Identifier
2010-L00523

Evaluation of the Freightliner Intelligent Vehicle Initiative Field Operational Test: Final Report

Summary Information

In 1999, the U.S. DOT entered into cooperative agreements with four partnerships to conduct Generation 0 Field Operational Tests (FOTs) of advanced intelligent vehicle safety systems (IVSS). Freightliner—in partnership with Praxair, the fleet operator; the University of Michigan Transportation Research Institute (UMTRI), the technology integrator; and Meritor-Wabco, the component supplier—tested two related but distinct functions of a new Roll Advisor and Control (RA&C) system. The RA&C system is designed to assist commercial vehicle drivers, especially drivers of tanker trucks, in avoiding rollover crashes. The benefits of RA&C for helping to avoid single vehicle roadway departure (SVRD) crashes were also evaluated. The first component is the Roll Stability Advisor (RSA), which is intended as an educational tool for drivers. The RSA will not prevent any particular crash through direct intervention. Instead, it merely advises a driver, after a maneuver is finished, that the lateral forces on the vehicle were higher than might have been desirable. The RSA’s advisory notices are provided to the driver as briefly worded messages appearing on the instrument panel display. The second component is the Roll Stability Control (RSC). This system takes partial, momentary control of the vehicle if it deems that a serious rollover threat is developing. The system’s authority in the FOT was limited to reducing the throttle or applying engine braking. Only a minimal amount of deceleration is applied in this manner, but the hope is that a bad situation can be prevented from becoming worse.

Lessons Learned

This lesson is drawn from the experiences of the Field Operation Test (FOT) of a Roll Advisor and Control (RA&C) system. The experiences from this FOT demonstrate that when doing this type of FOT based study, it is important to carefully consider the experimental design, the data collection plan, and the FOT sample size to improve the chances of having a successful FOT. The following discussion illustrates these lessons learned in more detail.

  • Develop a sound experimental design.

    There are a number of important things to consider when developing an experimental design. Selecting a long enough period for data collection is important to help eliminate seasonal effects from the main phenomena being studied. In the current study, there were two ways that the experimental design limited the strength of the conclusions. The FOT was conducted essentially within a one-year period, with one phase conducted as the weather was warming and the other phase conducted as the days shortened. Separating the seasonal effects from the RA&C’s effects could not be done with absolute certainty. Secondly, the demographics of the drivers were uniform. All were experienced, and all were within a twenty-year age span. A better test of the educational benefits of the RSA would have been to study two similar groups of novice drivers, one with the system and one without. Battelle, the evaluator of the FOT, understands that any experiment must be planned within constraints, and the partnership worked around these constraints as well as possible.
  • Automate data collection as much as possible.

    In this study, data collection was largely automatic. UMTRI reviewed the data to make sure all was working well. On ordinary days it took no help from Praxair personnel. In the cases where valid data were missing or the data collected were invalid, the partnership could readily trace the path through the server at the terminal to diagnose and fix the problem.
  • When planning the FOT size, carefully consider the trade-off between ease of data collection and statistical power.

    During this FOT, the use of a single terminal kept logistics to a manageable level. There were two occasions when a Battelle staff member interviewed the drivers. In both instances, nearly all of them were seen in a single trip of only a couple days. A benefit of the limited delivery area was that there were many road segments through which each driver passed many times. There were, however, disadvantages to the selection of a single terminal. Only six tractors, driven for six months each in the control and treatment phases, produced a fairly small number of serious events to analyze. By the nature of statistics, confidently estimating small changes, as were expected and observed in this FOT, requires large numbers of near-rollover incidents, which were simply not available. In this sense, the compact size of the FOT fleet was largely responsible for the lack of statistical significance in the final safety benefits estimate. Also, while the convenience of the terminal’s proximity to both UMTRI and Battelle had its benefits, the RA&C was tested exclusively in the Great Lakes region of the country, which is fairly free of hills and curves. As with many experiments, there is an inherent trade-off: a larger experiment provides richer data, but a smaller one is easier to conduct.

The lessons learned discussed above suggest that the creation of a sound experimental design, automating data collection as much as possible, and carefully considering the optimal size of the FOT can have a significant impact on the success of the FOT. Following these guidelines will assist FOT planners and stakeholders in helping to maximize the success and effectiveness of their FOT efforts.

Goal Areas

Keywords Taxonomy: