Perform System Monitoring of 511 Systems.

A national experience with the development and deployment of 511 Systems.

Date Posted
06/27/2007
TwitterLinkedInFacebook
Identifier
2007-L00389

America's Travel Information Number: Implementation and Operational Guidelines for 511 Services Version 3.0

Summary Information

In March of 1999, the U.S. Department of Transportation (USDOT) petitioned the Federal Communications Commission (FCC) to designate a nationwide three-digit telephone number for traveler information. In July 2001, the FCC designated 511 as the national traveler information number. As of March 2005, twenty-six 511 services across the country are operational and many have learned valuable lessons on deploying and operating systems.

In early 2001, the American Association of State Highway and Transportation Officials (AASHTO), the American Public Transportation Association (APTA), and the Intelligent Transportation Society of America (ITS America) with the support of the USDOT established a 511 Deployment Coalition. The goal of the Coalition is that 511 will be a “customer driven multi-modal traveler information service, available across the United States, accessed via telephones and other personal communications devices, realized through locally deployed interoperable systems, enabling a safer, more reliable and efficient transportation system.” In July 2005, the Coalition published the Implementation and Operational Guidelines for 511 Services, Version 3.0 to assist implementers in developing quality systems and increasing the level of operational knowledge among the 511 community. The lesson below is gathered from this guide, which has captured the experiences from many of the existing 511 services nationwide.

Lessons Learned

System monitoring is a critical function for 511. It can help agencies manage their projects and direct resources more effectively, and it can also be useful when seeking funding to continue or expand 511 services. In addition, system monitoring is an effective tool for ensuring customer satisfaction. Through monitoring the accuracy and reliability of the 511 system, deployers can obtain feedback on the quality of their 511 service, and can respond to problems or issues as they arise. System monitoring is important to ensuring that customers receive a high quality product.



System monitoring can be divided into three primary categories: usage, reliability and accuracy.

  • Monitor system usage: The 511 Deployment Coalition asks that 511 deployers report usage statistics of their system each month. The usage information is valuable to the Coalition, other deployers and in marketing and outreach activities. These statistics can help deployers gauge the consumer response to their services and enable comparisons among like systems. The list of statistics tracked by the Deployment Coalition is included in Appendix A of the Guidelines, and is shown below.
    • Calls Per Month – the total number of calls to the 511 system. This information is gathered as of the date of the launch of 511 services by the deployer. Some systems have an official public launch ceremony while others choose to slowly roll out the service as carriers reprogram switches.
    • Peak Call Day, Count and Reason – the day of the month that the system received the most total calls, the number of calls received that day and the reason for the influx of calls. Overtime, the peak call day usually has been caused by holiday travel, a major incident or weather phenomena.
    • Peak Call Hour, Count, Date and Reason – the hour of the month that the system received the most total calls, the number of calls received, the date and the reason for the influx of calls. Over time, the peak call hour usually has been caused by holiday travel, a major incident or weather phenomena, but it is not necessarily on the peak call day.
    • Highest Simultaneous # of Ports – the peak number of simultaneous ports (or calls) during a calendar month. This measure can assist a deployer in making decisions to expand or contract the maximum number of simultaneous calls that their system can handle.
    • Number of Regular Users – those service users who access information five or more times per calendar month.
    • Average Call Length (Seconds) – the total length of all calls in seconds divided by the total number of calls to 511. An increasing average call length may indicate that consumers are having problems accessing information. Deployers usually see their average call length decrease when switching to a voice response system and as users become familiar with the barge-in feature, menu structure and shortcuts.
    • Maximum Call Length – some deployers have a time limit established while others allow "unlimited" access which can become a cost issue.
    • Total Minutes per Month – the total number of minutes that calls to 511 were connected to the system. The rationale for total minutes per month is similar to average call length above.
    • Percent Wireless and Landline – the number of calls received via wireless and landline calls divided by the total number of calls. These percentages are useful in determining the effectiveness of 511 marketing campaigns as new billboards and road signs should see an increase in wireless usage and bill inserts may increase landline usage. A decrease in wireless usage may indicate that there is a problem with switch programming or call routing.
    • Percent Calls from Service Area – determined by the area codes or area code and exchange where the service is designed to be available.
    • Percent Calls from Outside Service Area – all other calls other than those above with some deployers tracking the states from which these callers "originate."
    • Percent Category – the Coalition currently asks deployers the percentage of calls to the following categories: Construction, Ferry, No Selection / Information, Other Language, Road Conditions, Segment Reports, Transfers (to other 511 services and agencies), Traffic, Transit, Weather, and Other Categories (Airports; Bicycling; Carpooling / Vanpooling; Commuter Incentives; Paratransit; Parks– information on transportation options or activities at national, state and local parks; Services; Spare the Air; Tourism – information specifically designed for tourists; and Travel Times– segment or point to point information). These categories are based on actual menu choices for systems around the country and enable comparisons between like systems. An increase in the No Selection / Information category may indicate problems that customers are experiencing with the system dropping calls or recognizing voice inputs.
    • Caller Comments – not all user comments are positive relating to service availability or content and there can be negative comments where callers do not “like” the service or voice recognition system. Some comments can be suggestions for improving the service such as by adding coverage or having information available in a different manner (by mile marker, for example).
  • Monitor system reliability: System reliability is the second key component of system monitoring. It is measured by comparing the system availability with a pre-determined standard. These Guidelines recommend that the 511 system be available 99.8% of the time (allowing for 17.5 hours of downtime annually). Other metrics that could be used are Mean Time between Failures and Mean Time to Repair. Deployers should focus on a few key metrics because system monitoring can demand significant resources if it is not narrowly focused to achieve project related goals. Detailed performance measures are also provided in Appendix B of the Guidelines.
  • Monitor system accuracy: The purpose of system accuracy monitoring is to ensure that the 511 information that is provided to users matches actual conditions on the roadway. This is especially important since 511 depends on information from external sources, the reliability of which may be uncertain. System accuracy monitoring is generally accomplished by taking a sample of incidents (or non-reports) and determining for a specific period, whether 511 was providing accurate information. If not, the lead agency can work to improve the data feeds it is receiving.

This lesson learned highlights the need for deployers to monitor their 511 system. System monitoring assists deployers in better managing their projects and in ensuring customer satisfaction. Through system monitoring, deployers can track the performance of their service against pre-determined standards, and can address any issues or problems that arise. Moreover, the usage data provides the Deployment Coalition with a means for understanding nationally the full scope and impact of 511.