The 7500 platform use the VIP (Versatile Interface Processor) for WAN interfaces. When dCEF is active the actual processing of the packets is done by these interfaces. Each one has its own CPU, memory, etc. The troubleshooting of this VIPs is difficult from the main IOS. In order to gain access to a deeper level you can access the hidden “console” of each VIP.
The hidden command for this is “if-con X” where X is the position of the VIP. There you will have a subset of commands only related to the VIP hardware itself. Under that CLI you may also run the also hidden command “sh process cpu” or “sh process memory” which will inform the processes and memory utilization related to the VIP itself.
Take a look at this sample session:
RT7500#if-con 2
Console or Debug [C]: C
Entering CONSOLE for VIP6-80 RM7000B 2
Type “^C^C^C” or “if-quit” to end this session
% Type “show ?” for a list of subcommands
VIP-Slot2>sh process memory sorted
Total: 257505888, Used: 91158344, Free: 166347544
PID TTY Allocated Freed Holding Getbufs Retbufs Process
9 0 294293084 4241136340 62089916 0 0 CEF LC IPC Backg
0 0 156416 1568 28046376 0 0 *Init*
3 0 327264 0 327296 0 0 CEF process
0 0 95840700 151828 268172 405780 0 *Dead*
24 0 2759865168 3045964672 72608 0 0 CEF LC Stats
38 0 14832 12576 37904 0 0 Exec
15 0 20896 0 13216 0 0 Net Background
26 0 0 0 12864 0 0 MDFS MFIB Proces
30 0 0 0 12864 0 0 MDFS LC Process
23 0 0 0 12864 0 0 MLP Input
31 0 0 0 12864 0 0 Encrypt Proc
16 0 0 0 12864 0 0 Logger
32 0 0 0 8864 0 0 Key Proc
36 0 712384 0 7904 168740 0 Per-minute Jobs
11 0 2368 0 7616 0 0 IPC Seat Manager
37 0 160 0 7040 0 0 CEF Scanner
17 0 25968 0 6944 0 0 TTY Background
14 0 0 0 6848 0 0 Critical Bkgnd
13 0 0 0 6848 0 0 SERIAL A’detect
12 0 0 0 6848 0 0 IPC Deferred Por
10 0 20992 0 6848 0 0 IPC Periodic Tim
…
VIP-Slot2>sh process cpu sorted
CPU utilization for five seconds: 6%/5%; one minute: 6%; five minutes: 6%
PID Runtime(ms) Invoked uSecs 5Sec 1Min 5Min TTY Process
38 152 133 1142 0.32% 0.05% 0.01% 0 Exec
2 24 942306 0 0.00% 0.00% 0.00% 0 Load Meter
3 58308 7051777 8 0.00% 0.00% 0.00% 0 CEF process
4 9021216 540686 16684 0.00% 0.13% 0.12% 0 Check heaps
5 0 649 0 0.00% 0.00% 0.00% 0 Pool Manager
6 0 1 0 0.00% 0.00% 0.00% 0 Timers
7 0 1 0 0.00% 0.00% 0.00% 0 Serial Backgroun
8 12 78527 0 0.00% 0.00% 0.00% 0 IPC Dynamic Cach
9 446046 24596127 18 0.00% 0.00% 0.00% 0 CEF LC IPC Backg
1 0 1 0 0.00% 0.00% 0.00% 0 Chunk Manager
11 4 13 307 0.00% 0.00% 0.00% 0 IPC Seat Manager
12 104 4711140 0 0.00% 0.00% 0.00% 0 IPC Deferred Por
13 0 1 0 0.00% 0.00% 0.00% 0 SERIAL A’detect
14 0 1 0 0.00% 0.00% 0.00% 0 Critical Bkgnd
15 916 604255 1 0.00% 0.00% 0.00% 0 Net Background
16 0 20 0 0.00% 0.00% 0.00% 0 Logger
17 284 4711140 0 0.00% 0.00% 0.00% 0 TTY Background
18 1344 4711140 0 0.00% 0.00% 0.00% 0 Per-Second Jobs
10 268 4711140 0 0.00% 0.00% 0.00% 0 IPC Periodic Tim
19 0 1 0 0.00% 0.00% 0.00% 0 CSP Timer
21 0 1 0 0.00% 0.00% 0.00% 0 Hawkeye Backgrou
…
VIP-Slot2>sh cef interface atm 0/0
ATM2/0/0 is up (if_number 3)
Corresponding hwidb fast_if_number 53
Corresponding hwidb firstsw->if_number 3
Internet Protocol processing disabled
Hardware idb is ATM0/0 (53)
Software idb is ATM0/0 (3)
Fast switching type 9, interface type 60
IP Distributed CEF switching enabled
IP VIP CEF VPN switching turbo vector
Input fast flags 0×24001, Output fast flags 0×1, Flags 0×80000
ifindex 2(1)
Slot 2 Slot unit 0 VC -1
Transmit limit accumulator 0xE8001A0A (0×0)
IP MTU 4096
Switching statistics:
Input: 2045821909 packets, 4763385945326 bytes
Output: 258298428 packets, 30703495813 bytes
0 input drops, 5112 output drops
VIP-Slot2>sh controllers
Interface ATM0/0 is up
Hardware is TI1570 ATM
slot 0, unit 0, subunit 0, fci_type 0×00000017, max_pak_size 4154
particle size 512, pool size 2048, cache size 2048, cache end 1023
Fallback pool cache size 89761, cache end 89761
rx free buf ring size 1024, init buffers 1024, current buffers 1023
rx max spin 64
tx seg rings size 256, tx max spin 64, tx scheduler table size 48
enabled 0, disabled 0, throttled 0
vc_per_vp 1024, max_vp 6, max_vc 6144, total_vc 30
rx cells 4082603089
hec error 0, aal5 discard 0, unknown prot 10214, unknown reg 0×100000C2
pkt overflow 0, crc error 10390, no buf 0, rx freeze 2
timeout 0, abort 134441, congestion_cell 0, rx dma error 0
tx cells 3733330619, tx count 0, tx limited FALSE
abort 0, no desc 0, align 0, discard 0, tx freeze 0
tx dma error 0
PCI detect parity error 0, system error 0
master abort error 0, receive target abort error 0
signaled target abort error 0, data parity error 0
Device values:
TI1570 device number 0, base addr 0×50800000, pci base off 0×800000
PLX9060ES device number 1, base addr 0×50900000, pci base off 0×904000
rx vpi table 0×50804000, rx dma table 0×50810000, rx fbrp table 0×50803800
tx sched table 0×50800000, tx dma table 0×50808020, tx sched interv 48
rcmplr 0×38008000, rcmplr ptr 0×3800D960, rfbr 0×30018000, rfbr
ptr 0×30018128
tcmplr 0×38010000, tcmplr ptr 0×380125CC
PLIM type ID 00A0, state 3, line rate 155520 kbps, rx fifo ovr 0
pacb.pa_bay 0×0, pacb.pa_device 0×0
pacb.pak_to_host 0×602A47F8, pacb.vip_memd_ifcntl 0×6093C1C0
pacb.pa_ifindex 0×0, idb->lc_ip_turbo_fs 0×60382B48
vcstats failed 0
dfs is enabled, idb->ip_routecache = 0×15
idb : 0×60C8DF00, lane client mac address : 00d0.bc80.f840
VIP-Slot2#if-quit
Disconnecting from slot 2 CONSOLE after 00:42:39
Anyway, this is a nice to know. If you have real issues and you want to troubleshoot the VIP you might find more useful the following links:
Hardware Troubleshooting for the Cisco Versatile Interface Processor (VIP)
http://www.cisco.com/en/US/products/hw/modules/ps3088/products_tech_note09186a008009476c.shtml#symptoms
Troubleshooting Versatile Interface Processor (VIP) Crashes
http://www.cisco.com/en/US/products/hw/modules/ps2643/products_tech_note09186a0080094b15.shtml#beforecase
