Voice Traffic Gets Priority at AIG

Datamation content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

VoIP offers enterprises many advantages including less wiring and fewer protocols to support. It does, however, require tighter control of network service levels than required by strictly data networks.

“You have to keep voice at the top of your priority queues,” says Jose L. Alvarez, Network Management Engineer for the insurance company American International Group, Inc. (AIG) in Wilmington, Delaware. To keep voice on top, the company started using Scrutinizer NetFlow management software from Plixer International Inc. of Sanford, Maine.

AIG’s main data center is located at its Wilmington headquarters, with a backup disaster recovery site in Alpharetta, a suburb of Atlanta. Each of the five remote sites – four call centers and a print shop – home in on both of the data centers via E-3 (34Mbps) connections. The main site hosts an Avaya 8720 PBX to service the firm’s 2500 users, with a backup 8700 series box in Alpharetta. Cisco 6500 series switches and 3800 series routers direct the traffic where it is needed. But he needed a method of seeing exactly what was flowing through those routers.

“I had no visibility into my network,” says Alvarez. “I could tell when utilization was high on a particular connection, but I couldn’t tell what was causing it.”

Standard Network Management

The need for greater visibility led Alvarez to look into ways of using the NetFlow data produced by his Cisco equipment. NetFlow provides data on the protocols and applications flowing through the ports on a switch or router, as well as the source and destination addresses. Developed by Cisco in the mid-90s, NetFlow has been broadly adopted by companies as an additional means of improving service levels.

“NetFlow is a mature standard that is used by almost all large service providers and enterprises,” says Cliff Meltzer, Sr. VP of Cisco’s Network Management Technology Group.

While NetFlow had become a de facto standard due to Cisco’s large market share, as well as adoption by other vendors, recently it became an official standard when the IETF (Internet Engineering Task Force) adopted NetFlow v.9 as the basis for an open standard called IPFIX – Internet Protocol Flow Information eXport.

NetFlow examines the packets based on a set of attributes such as IP source address, source/destination ports, Layer 3 protocol, class of service, and router or switch interface. When a series of packets have the same attributes, they are assigned to the same flow.

For example, all packets in a single phone conversation would have the same source and destination addresses, use the same ports, have the same protocol and service class. In that case, the device would add up all those the packets and report that the flow had X number of packets. That information can then be used to see who were the top users, or what were the top applications, at a particular point in time. If users reported slow response times, a look at NetFlow might show that software updates were travelling the network at that point in time, and these could then be scheduled for off hours.

Giving Voice Priority

NetFlow is a push technology consisting of two elements – a data generator and a data collector. A data generator is any device that has NetFlow enabled. While it is a native component of Cisco’s operating system, administrators have to activate it using a series of simple commands.

At that point the device will start exporting the NetFlow data. A 20Mb interface will generate about 1 Gb of data daily and send it to a NetFlow collector for storage, analysis and reporting. The collector can be an appliance, such as NetQoS’s ReporterAnalyzer, which is used primarily by Global 2000 organizations including Cisco, Chevron, Hewlett-Packard and Verizon. The other option is to install the collector software on a server, which is the approach AIG took.

Alvarez looked at several NetFlow analyzers before settling on Scrutinizer. He says that NetIQ Corporation’s software offered a broader range of features than Scrutinizer, but at a much higher price.

“When you get down to basics, both Scrutinizer and NetIQ will give you what you want: a way to identify your traffic,” he says. “So, was it worth an extra $250,000 to get NetIQ’s bells and whistles? The answer was ‘no,’ I couldn’t justify that price.”

Scrutinizer comes in several versions costing from $1995 to $8995, and is also available as an appliance for $29,995. AIG loaded its copy of the software on a dedicated Fujitsu blade server that hosts the application and the database containing the NetFlow data.

One issue that cropped up immediately on installation was that the control data wasn’t running on the UDP ports Avaya said it was.

“We set up the UDP port ranges to monitor for VoIP payload and control characters, based on the data from Avaya, but I couldn’t locate any control traffic,” says Alvarez. “That was when I realized that the PBX was using the H.323 protocol instead of the even UDP ports for control.”

Alvarez says he uses Scrutinizer to monitor all his network traffic, not just the VoIP.

“Once you have the application identified, it will tell you all the traffic that fits within those parameters,” he says. “You are able to break out the 20 percent of my traffic that is voice and can see the most common protocols such as HTTP or Telnet. We can also identify our policy systems and claims systems in Scrutinizer, separate from other types of traffic.”

If a circuit is overloaded, he clicks on the bar showing that circuit to bring up a summary of the last five minutes’ traffic. If a particular protocol or application is running higher than expected, he can drill down into the specific conversations to see where the problem lies.

“The minute we get a ticket saying a particular site or application is down, Scrutinizer is the first place we go,” says Alvarez. “We have other monitoring tools, but with Scrutinizer we can see in an instant what kind of traffic is going through.”

In addition to isolating the cause of network slowdowns, he says he sometimes finds when looking at the data that some of the Avaya phones have been assigned to the wrong VLANs. He also uses Scrutinizer to provide Finance with information on how much bandwidth different sites or applications are using for chargeback purposes. And, with the latest version of the software, he was able to start monitoring the QoS on the phone lines.

“Now I can identify the traffic within my priority queues so I can see that my QoS is working,” he says. “I can tell that no user going to YouTube is sneaking into my voice queues and I know my voice traffic is going to get priority over everything else.”

This article was first published on EnterpriseITPlanet.com.