Adjusting Buffers

Excessively high bandwidth utilization greater than 70 percent results in reduced overall performance and can cause intermittent failures. For example, DECnet file transmissions might be failing because of packets being dropped somewhere in the network.

If the situation is bad enough, you must increase the bandwidth of the link. However, increasing the bandwidth might not be necessary or immediately practical. One way to resolve marginal serial line overutilization problems is to control how the router uses data buffers.

Caution In general, do not adjust system buffers unless you are working closely with a Cisco technical support representative. You can severely affect the performance of your hardware and your network if you incorrectly adjust the system buffers on your router.

Use one of the following three options to control how buffers are used:

•Adjust parameters associated with system buffers.

•Specify the number of packets held in input or output queues (hold queues).

•Prioritize how traffic is queued for transmission (priority output queuing).

The configuration commands associated with these options are described in the Cisco IOS configuration guides and command references.

The following section focuses on identifying situations in which these options are likely to apply and defining how you can use these options to help resolve connectivity and performance problems in serial/WAN interconnections.

Tuning System Buffers

There are two general buffer types on Cisco routers: hardware buffers and system buffers. Only the system buffers are directly configurable by system administrators. The hardware buffers are specifically used as the receive and transmit buffers associated with each interface and (in the absence of any special configuration) are dynamically managed by the system software itself.

The system buffers are associated with the main system memory and are allocated to different-size memory blocks. A useful command for determining the status of your system buffers is the show buffers exec command. Figure 15-8 shows the output from the show buffers command.

Figure 15-8 show buffers Command Output

In the show buffers output, the following is true:

•total identifies the total number of buffers in the pool, including used and unused buffers.

•permanent identifies the permanent number of allocated buffers in the pool. These buffers are always in the pool and cannot be trimmed away.

•in free list identifies the number of buffers currently in the pool that are available for use.

•min identifies the minimum number of buffers that the route processor (RP) should attempt to keep in the free list:

–The min parameter is used to anticipate demand for buffers from the pool at any given time.

–If the number of buffers in the free list falls below the min value, the RP attempts to create more buffers for that pool.

•max allowed identifies the maximum number of buffers allowed in the free list:

–The max allowed parameter prevents a pool from monopolizing buffers that it doesn’t need anymore, and frees this memory back to the system for further use.

–If the number of buffers in the free list is greater than the max allowed value, the RP should attempt to trim buffers from the pool.

•hits identifies the number of buffers that have been requested from the pool. The hits counter provides a mechanism for determining which pool must meet the highest demand for buffers.

•misses identifies the number of times that a buffer has been requested and that the RP detected that additional buffers were required. (In other words, the number of buffers in the free list has dropped below min.) The misses counter represents the number of times that the RP has been forced to create additional buffers.

•trims identifies the number of buffers that the RP has trimmed from the pool when the number of buffers in the free list exceeded the number of max allowed buffers.

•created identifies the number of buffers that has been created in the pool. The RP creates buffers when demand for buffers has increased until the number of buffers in the free list is less than min buffers or a miss occurs because of zero buffers in the free list.

•failures identifies the number of failures to grant a buffer to a requester even after attempting to create an additional buffer. The number of failures represents the number of packets that have been dropped due to buffer shortage.

•no memory identifies the number of failures caused by insufficient memory to create additional buffers.

The show buffers command output in Figure 15-8 indicates high numbers in the Trims and Created fields for large buffers. If you are receiving high numbers in these fields, you can increase your serial link performance by increasing the max free value configured for your system buffers. trims identifies the number of buffers that the RP has trimmed from the pool when the number of buffers in free list exceeded the number of max allowed buffers.

Use the buffers max free number global configuration command to increase the number of free system buffers. The value that you configure should be approximately 150 percent of the figure indicated in the total field of the show buffers command output. Repeat this process until the show buffers output no longer indicates trims and created buffers.

If the show buffers command output shows a large number of failures in the (no memory) field (see the last line of output in Figure 15-8), you must reduce the usage of the system buffers or increase the amount of shared or main memory (physical RAM) on the router. Call your technical support representative for assistance.

Implementing Hold Queue Limits

Hold queues are buffers used by each router interface to store outgoing or incoming packets. Use the hold-queue interface configuration command to increase the number of data packets queued before the router will drop packets. Increase these queues by small increments (for instance, 25 percent) until you no longer see drops in the show interfaces output. The default output hold queue limit is 100 packets.

Note The hold-queue command is used for process-switched packets and periodic updates generated by the router.

Use the hold-queue command to prevent packets from being dropped and to improve serial link performance under the following conditions:

•You have an application that cannot tolerate drops, and the protocol is capable of tolerating longer delays. DECnet is an example of a protocol that meets both criteria. Local-area transport (LAT) does not meet this criteria because it does not tolerate delays.

•The interface is very slow (bandwidth is low or anticipated utilization is likely to sporadically exceed available bandwidth).

Note When you increase the number specified for an output hold queue, you might need to increase the number of system buffers. The value used depends on the size of the packets associated with the traffic anticipated for the network.

Using Priority Queuing to Reduce Bottlenecks

Priority queuing is a list-based control mechanism that allows traffic to be prioritized on an interface-by-interface basis. Priority queuing involves two steps:

Step 1 Create a priority list by protocol type and level of priority.

Step 2 Assign the priority list to a specific interface.

Both of these steps use versions of the priority-list global configuration command. In addition, further traffic control can be applied by referencing access-list global configuration commands from priority-list specifications. For examples of defining priority lists and for details about command syntax associated with priority queuing, refer to the Cisco IOS configuration guides and command references.

Note Priority queuing automatically creates four hold queues of varying size. This overrides any hold queue specification included in your configuration.

Use priority queuing to prevent packets from being dropped and to improve serial link performance under the following conditions:

•When the interface is slow, a variety of traffic types are being transmitted, and you want to improve terminal traffic performance

•If you have a serial link that is intermittently experiencing very heavy loads (such as file transfers occurring at specific times), and priority queuing will help select which types of traffic should be discarded at high traffic periods

In general, start with the default number of queues when implementing priority queues. After enabling priority queuing, monitor output drops with the show interfaces serial exec command. If you notice that output drops are occurring in the traffic queue that you have specified to be high priority, increase the number of packets that can be queued (using the queue-limit keyword option of the priority-list global configuration command). The default queue-limit arguments are 20 packets for the high-priority queue, 40 for medium, 60 for normal, and 80 for low.

Note When bridging Digital Equipment Corporation (Digital) LAT traffic, the router must drop very few packets, or LAT sessions can terminate unexpectedly. A high-priority queue depth of about 100 (specified with the queue-limit keyword) is a typical working value when your router is dropping output packets and the serial lines are subjected to about 50 percent bandwidth utilization. If the router is dropping packets and is at 100 percent utilization, you need another line.

Another tool to relieve congestion when bridging Digital LAT is LAT compression. You can implement LAT compression with the interface configuration command bridge-group group lat-compression. 

 Troubleshooting Clocking Problems

Clocking conflicts in serial connections can lead either to chronic loss of connection service or to degraded performance. This section discusses the important aspects of clocking problems: clocking problem causes, how to detect clocking problems, how to isolate clocking problems, and clocking problem solutions.

Clocking Overview

The CSU/DSU derives the data clock from the data that passes through it. To recover the clock, the CSU/DSU hardware must receive at least one 1-bit value for every 8 bits of data that pass through it; this is known as ones density. Maintaining ones density allows the hardware to recover the data clock reliably.

Newer T1 implementations commonly use Extended Superframe Format (ESF) framing with binary eight-zero substitution (B8ZS) coding. B8ZS provides a scheme by which a special code is substituted whenever eight consecutive zeros are sent through the serial link. This code is then interpreted at the remote end of the connection. This technique guarantees ones density independent of the data stream.

Older T1 implementations use D4 (also known as Superframe Format [SF]) framing and Alternate Mark Inversion (AMI) coding. AMI does not utilize a coding scheme like B8ZS. This restricts the type of data that can be transmitted because ones density is not maintained independent of the data stream.

Another important element in serial communications is serial clock transmit external (SCTE) terminal timing. SCTE is the clock echoed back from the data terminal equipment (DTE) device (for example, a router) to the data communications equipment (DCE) device (for example, the CSU/DSU).

When the DCE device uses SCTE instead of its internal clock to sample data from the DTE, it can better sample the data without error even if there is a phase shift in the cable between the CSU/DSU and the router. Using SCTE is highly recommended for serial transmissions faster than 64 kbps. If your CSU/DSU does not support SCTE, see the section “Inverting the Transmit Clock,” later in this chapter.

Clocking Problem Causes

In general, clocking problems in serial WAN interconnections can be attributed to one of the following causes:

•Incorrect DSU configuration

•Incorrect CSU configuration

•Cables out of specification (longer than 50 feet [15.24 meters] or unshielded)

•Noisy or poor patch panel connections

•Several cables connected in a row

Detecting Clocking Problems

To detect clocking conflicts on a serial interface, look for input errors as follows:

Step 1 Use the show interfaces serial exec command on the routers at both ends of the link.

Step 2 Examine the command output for CRC, framing errors, and aborts.

Step 3 If either of these steps indicates errors exceeding an approximate range of 0.5 percent to 2.0 percent of traffic on the interface, clocking problems are likely to exist somewhere in the WAN.

Step 4 Isolate the source of the clocking conflicts, as outlined in the following section, “Isolating Clocking Problems.”

Step 5 Bypass or repair any faulty patch panels.

Isolating Clocking Problems

After you determine that clocking conflicts are the most likely cause of input errors, use the following procedure to isolate the source of those errors:

Step 1 Perform a series of ping tests and loopback tests (both local and remote), as described in the section “CSU and DSU Loopback Tests,” earlier in this chapter.

Step 2 Determine which end of the connection is the source of the problem, or whether the problem is in the line. In local loopback mode, run different patterns and sizes in the ping tests (for example, use 1500-byte datagrams). Using a single pattern and packet size may not force errors to materialize, particularly when a serial cable to the router or CSU/DSU is the problem.

Step 3 Use the show interfaces serial exec command, and determine whether input errors counts are increasing and where they are accumulating.

If input errors are accumulating on both ends of the connection, clocking of the CSU is the most likely problem.

If only one end is experiencing input errors, there is probably a DSU clocking or cabling problem.

Aborts on one end suggest that the other end is sending bad information or that there is a line problem.

Note Always refer to the show interfaces serial command output (see Figure 15-1). Log any changes in error counts, or note if the error count does not change.

Clocking Problem Solutions

Table 15-8 outlines suggested remedies for clocking problems, based on the source of the problem.

Inverting the Transmit Clock

If you are attempting serial connections at speeds greater than 64 kbps with a CSU/DSU that does not support SCTE, you might have to invert the transmit clock on the router. Inverting the transmit clock compensates for phase shifts between the data and clock signals.

The specific command used to invert the transmit clock varies between platforms. On a Cisco 7000 series router, enter the invert-transmit-clock interface configuration command. For Cisco 4000 series routers, use the dte-invert-txc interface configuration command.

To ensure that you are using the correct command syntax for your router, refer to the user guide for your router or access server and to the Cisco IOS configuration guides and command references. 

 Using Extended ping Tests

The ping command is a useful test available on Cisco internetworking devices as well as on many host systems. In TCP/IP, this diagnostic tool is also known as an Internet Control Message Protocol (ICMP) echo request.

Note The ping command is particularly useful when high levels of input errors are being registered in the show interfaces serial display. See Figure 15-1.

Cisco internetworking devices provide a mechanism to automate the sending of many ping packets in sequence. Figure 15-5 illustrates the menu used to specify extended ping options. This example specifies 20 successive pings. However, when testing the components on your serial line, you should specify a much larger number, such as 1000 pings. Also increase the datagram size to a larger number, such as 1500.

Figure 15-5 Extended ping Specification Menu

In general, perform serial line ping tests as follows:

Step 1 Put the CSU or DSU into local loopback mode.

Step 2 Configure the extended ping command to send different data patterns and packet sizes. Figure 15-6 and Figure 15-7 illustrate two useful ping tests, an all-zeros 1500-byte ping and an all-ones 1500-byte ping, respectively.

Step 3 Examine the show interfaces serial command output (see Figure 15-1) and determine whether input errors have increased. If input errors have not increased, the local hardware (DSU, cable, router interface card) is probably in good condition.

Assuming that this test sequence was prompted by the appearance of a large number of CRC and framing errors, a clocking problem is likely. Check the CSU or DSU for a timing problem. See the section “Troubleshooting Clocking Problems,” next.

Step 4 If you determine that the clocking configuration is correct and is operating properly, put the CSU or DSU into remote loopback mode.

Step 5 Repeat the ping test and look for changes in the input error statistics.

Step 6 If input errors increase, there is a problem either in the serial line or on the CSU/DSU. Contact the WAN service provider and swap the CSU or DSU. If problems persist, contact your technical support representative.

Figure 15-6 All-Zeros 1500-Byte ping Test

Figure 15-7 All-Ones 1500-Byte ping Test