Expect to spend time and effort in testing and refining the voice recognition features of a 511 system.

An Arizona Department of Transportation experience in 511 implementation.

Date Posted
05/17/2006
TwitterLinkedInFacebook
Identifier
2006-L00240

Model Deployment of a Regional, Multi-Modal 511 Traveler Information System: Final Report

Summary Information

On July 21, 2000, the Federal Communications Commission assigned 511 as the nationwide traveler information telephone number and granted responsibility for it to government transportation agencies. Since that time, approximately 27 statewide and regional 511 systems have been implemented throughout the United States. In 2002, the Federal Highway Administration (FHWA) and Federal Transit Administration (FTA) jointly issued a request for proposals for participation in a national 511 Model Deployment. The Model Deployment was intended to demonstrate the benefits of 511 systems and generate lessons learned of use to other 511 deployers and the U.S. Department of Transportation (USDOT). The 511 Model Deployment was awarded to an Arizona Department of Transportation (ADOT)-led coalition of transportation agencies in Arizona. This lesson is based on the FHWA evaluation of the Arizona 511 Model Deployment.

Lessons Learned

The Model Deployment called for "pushing the envelope" in a number of areas and specified a user interface utilizing proven voice recognition technology. The Model Deployment featured a large number of enhancements to the existing Arizona 511 system, including adding new information types and converting from a touch-tone (i.e., keypad)-only user interface to a voice recognition interface. ADOT operates their own 511 system and utilized their existing phone system consultant for the voice recognition implementation.



Although voice recognition is a proven telephone user interface, its implementation in a 511 system can be challenging and 511 deployers should expect to spend considerable effort testing and refining the voice-recognition system. The ADOT experience provides several suggestions for 511 deployers:

  • Capture user comments to identify and resolve problems with the voice-recognition system. In addition to its own testing, ADOT discovered many problems from user comments. Many users of the early ADOT voice recognition system reacted quite negatively and were able to express their frustration via the caller comment feature (a voice mailbox on the 511 menu). Most problems fell into one of two categories: (1) errors in menu system and message programming, i.e., requests for "roads" being sent to the transit menu, or information for Interstate 15 being given in response to a request for information on Interstate 10; (2) the system misinterpreting user inputs (i.e., utterances) or interpreting background noises as utterances.
  • Be sure to include testing and refinement in the project schedule. ADOT found that it took considerable time to discover and correct problems. The second type of malfunction noted above, that is the system misinterpreting user utterances, proved especially challenging. Many users continued to find the voice recognition system unsatisfactory: 10 months after implementation, 35% of surveyed users were dissatisfied. ADOT found the complexity of the voice-recognition implementation resulted in between 1 and 3 months schedule delay.
  • Be sure to budget for testing and refining the voice-recognition system. ADOT estimates that they spent approximately $560,000 on the voice recognition related enhancement, about 61% more than they had estimated.
  • Beware that the larger the transportation system covered by 511, the more problems will be encountered with voice-recognition. As the system covered by 511 increases, especially in the number of roadways covered, so too does the number of valid utterances the system must distinguish. In this respect, large 511 systems like the statewide Arizona 511 system strain basic voice recognition performance much more than systems that limit users to a few distinct utterances at each prompt, e.g., "arrivals" or "departures" in an airline customer information system, or systems which list the options and, for each, ask the user to say "yes" or "no".



Implementing and fine-tuning voice recognition performance is one of the more challenging, labor intensive activities associated with developing or enhancing a 511 system. Deployers should allocate significant time and resources to this activity, with the understanding that the amount of effort will increase with the number of possible user inputs the system must recognize. Systems covering larger areas, such as statewide systems, will require more effort to implement and fine-tune. Systems that accept many different specific names for transportation facilities and services (rather than listing the options and asking only for users to confirm with "yes" or "no") will also require more effort.