6.6. System Information

The file sysinfo.txt contains the system information of the Steelhead appliance. It contains the operating system information, hard disk utilization, ifconfig output, ECC counter details, system and in-path routing tables, the output of the arp command, the process list, the directory listing of /var/opt and /var/log, IP/TCP/UDP/ICMP networking statistics and NIC interface statistics.

6.6.1. Operating system and RiOS versions

The first section contains the hostname, serial number of the device, the RiOS version on the device, the original software version and the date and time and uptime of the device:

Figure 6.58. System Information for a 550M model

System information:

Hostname: CSH
Serial Number: J47NG00012345
Model Number: 550M
Version:  rbt_sh 6.5.2 #113_20 2011-10-04 15:39:37 i386 root@moscow:svn://svn/mgmt/branche \
    s/canary_113_fix_7_sedgman_branch
Manufactured version: 5.0.8
Date:     2011-12-16 17:42:23
Uptime:   70d 4h 4m 21s

Service Uptime [DD-HH:MM:SS]: 24-04:10:37

This device is running the 64-bit version of RiOS version 6.5.2. The i386 shows it is a 32-bit version, if it shows x86_64 then it is the 64-bit version.

The Uptime is the uptime of the Steelhead appliance, the Service Uptime is the uptime of the optimization service.

The next section contains the build details of the sport binary of this RiOS version. For RiOS EX, this can be used to determine the RiOS version used in this RiOS EX software:

Figure 6.59. System information for an EX1160L model

System information:

Hostname: CSH
Serial Number: F84YU00012345
Model Number: EX1160L
Disk Layout: vsp_granite
Version:  rbt_ex 1.0.5 #556_10 2013-05-07 17:11:29 x86_64 root@montreux:svn://svn/build/br \
    anches/malta-ex_556_fix_branch
Manufactured version: 1.0.1a
Date:     2013-12-18 16:28:47
Uptime:   12d 17h 54m 5s

Service Uptime [DD-HH:MM:SS]: 12-17:49:31

==================================================
Output of '/opt/rbt/bin/sport -v':

Version: rbt_sh 7.0.6 (malta/fix/556_10)
Options: configure --disable-debug --enable-optimize --enable-epoll --enable-nfs --with-na \
    t=DEVNBT --enable-linker-map --with-tmslib=/work/prebuilt-modules/built-mgmt/branches/ \
    malta_556_fix_branch/125493/centos-4.2/x86_64/rbt_sh-7.0.6/tmslib --with-vmwaretools=/ \
    mnt/builds/prebuilt-modules/built-vmware-tools/branches/malta_556_fix_branch/4752/cent \
    os-4.2/x86_64 --with-sysincl=/work/prebuilt-modules/built-kernel/branches/malta_556_fi \
    x_branch/16634/2.6.9-34.EL/centos-4.2/x86_64//smp/smp
Builder: root@montreux (Linux CentOS release 4.8 (Final))
Kernel : 2.6.9-34.EL-rbt-16251SMP
Time   : Tue May  7 16:24:05 PDT 2013
Input  : /work/flamebox/sport-build-19763/sport
Output : /work/flamebox/sport-build-19763/sport/build
MD5    : b576d0f26bea7fe622721e3adb2c4a88

After RiOS EX 3.0 the output of this command only shows the RiOS EX software version, not the corresponding RiOS software version.

The next table contains the full list of RiOS EX software and matching RiOS version. An up-to-date list can be found in KB article S20423.

Table 6.1. RiOS EX software and matching RiOS version

RiOS EX softwareCorresponding RiOS version
1.0.17.0.2
1.0.27.0.3
1.0.47.0.5
1.0.57.0.6
2.0.08.0.0
2.0.18.0.1
2.0.28.0.1a
2.1.08.0.2
2.1.18.0.2
2.1.28.0.2
2.5.08.0.2d
2.5.18.0.2d
2.5.28.0.4
3.0.08.5.0

6.6.2. Hard disk utilization

The hard disk utilization part shows the usage of the various partitions on the Steelhead appliance. The only ones interesting are the /var partition which stores the various log files and data files, the /config partition which stores the systems configuration and the /proxy partition which contains PFS and RSP data.

The data store partition is not visible here since its format is not known by the df utility.

Figure 6.60. Hard disk utilization overview

Output of 'df -Pka':

Filesystem         1024-blocks      Used Available Capacity Mounted on
rootfs                  516008    292116    197608      60% /
/proc                        0         0         0       -  /proc
none                   1815084       232   1814852       1% /dev
/dev/root               516008    292116    197608      60% /
none                   1815084       232   1814852       1% /dev
tmpfs                  1815084      7260   1807824       1% /lib/modules
/proc                        0         0         0       -  /proc
/proc/bus/usb                0         0         0       -  /proc/bus/usb
/sys                         0         0         0       -  /sys
none                         0         0         0       -  /dev/pts
/dev/sdb2               128716     11254    110816      10% /boot
/dev/sdb1               253611      4246    236271       2% /bootmgr
/dev/sdb7               418806     49904    347277      13% /config
/dev/sda3             16513448   2769100  12905408      18% /var
none                   1815084         0   1815084       0% /dev/shm
none                     16384         4     16380       1% /tmp
/dev/disk0p6          56765020  39510668  14370736      74% /proxy
encfs                   418806     49904    347277      13% /var/opt/rbt/decrypted

The /boot mount point is sometimes on the second partition ( /dev/sdb2 on a 550 model) and sometimes on the third partition ( /dev/sdb3 on a 550 model), depending on which boot partition the Steelhead appliance is booted from. This is located on the USB Flash disk.

The /config mount point is the partition with the configuration of the Steelhead appliance. It is located on the USB Flash disk.

The /var mount point is the partition which is writable by the operating system. It contains:

  • The log files are in /var/log.

  • RiOS images are uploaded to /var/opt/tms/images.

  • System dumps are created into /var/opt/tms/sysdumps.

  • Process dumps are created in /var/opt/tms/snapshots.

  • The Secure Vault is in /var/opt/rbt/decrypted, which contains the SSL certificates and private keys and the encrypted data store password. It is not a physical file system but an encrypted file system which lies on /var/opt/rbt/encrypted.

The /proxy mount point is the partition which contains the Proxy File System data (PFS) or the Riverbed Services Platform data (RSP).

6.6.3. Ifconfig output

The ifconfig output shows some high level statistics of various network interfaces. The most important fields are the administrative status UP, the IP address inet addr, the hardware MAC address HWaddr and some general counters like RX packets and TX packets.

Figure 6.61. Ifconfig output

Output of 'ifconfig -a':

inpath0_0 Link encap:Ethernet  HWaddr 00:0E:B6:12:34:57  
          inet addr:10.0.1.6  Bcast:10.0.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5703948 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5649152 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:1403364965 (1.3 GiB)  TX bytes:1171662453 (1.0 GiB)

lan0_0    Link encap:Ethernet  HWaddr 00:0E:B6:12:34:57  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:4643691 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2781142 errors:2 dropped:0 overruns:0 carrier:2
          collisions:0 txqueuelen:100 
          RX bytes:1040279574 (992.0 MiB)  TX bytes:601245915 (573.3 MiB)
          Memory:fdce0000-fdd00000 

primary   Link encap:Ethernet  HWaddr 00:0E:B6:12:34:56  
          inet addr:10.0.1.5  Bcast:10.0.1.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:38967 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1874049 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:3564489 (3.3 MiB)  TX bytes:472288871 (450.4 MiB)
          Memory:fd9e0000-fda00000 

wan0_0    Link encap:Ethernet  HWaddr 00:0E:B6:12:34:58  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1060250 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2867967 errors:2 dropped:0 overruns:0 carrier:1
          collisions:0 txqueuelen:100 
          RX bytes:466055536 (444.4 MiB)  TX bytes:627854299 (598.7 MiB)
          Memory:fdc80000-fdca0000 

The following information can be seen in this overview:

  • The virtual interface named inpath0_0 has the IP address 10.0.1.6, the physical lan0_0 and wan0_0 interfaces do not have an IP address.

  • The MAC address of the inpath0_0 interface is the MAC address of the lan0_0 interface.

  • The MAC address of the lan0_0 interface is one lower than the MAC address of the wan0_0 interface.

  • The administrative status of all interfaces is UP. In virtual in-path deployment the administrative status of the lan0_0 does not contain the string UP.

  • The counters on the RX packets and TX packets are cumulative and should have zero errors. These counters can be zeroed out with the command clear interfaces all. In the example there have been two carrier errors on the lan0_0 interface, which could be because of loss of the Ethernet link with the connected device.

  • The interfaces named rios_wan0 and rios_wan0 are used by the RSP system.

  • The interface named prihw is the physical interface on which the primary interface is mapped.

  • The interface named rbtpipe is used by the Steelhead Cloud Accelerator functionality.

6.6.4. Memory ECC counters

The memory used in the Steelhead appliances has Error Correcting Code capabilities, which means that if it finds a bit-error it will try to correct it: It will be marked as a Correctable Error if successful and as an Uncorrectable Error if unsuccessful.

Figure 6.62. ECC counter output

Output of 'show_ecc_status':

/sys/devices/system/edac/mc/mc0/csrow0/ce_count:0
/sys/devices/system/edac/mc/mc0/csrow0/ch0_ce_count:0
/sys/devices/system/edac/mc/mc0/csrow0/ue_count:0
/sys/devices/system/edac/mc/mc0/csrow3/ce_count:0
/sys/devices/system/edac/mc/mc0/csrow3/ch0_ce_count:0
/sys/devices/system/edac/mc/mc0/csrow3/ue_count:0
/sys/devices/system/edac/mc/mc0/csrow0/ch0_dimm_label:DIMM0
/sys/devices/system/edac/mc/mc0/csrow3/ch0_dimm_label:DIMM0
/sys/devices/system/edac/mc/mc0/csrow0/size_mb:1024
/sys/devices/system/edac/mc/mc0/csrow3/size_mb:1024

The output shows ce_count (the number of Correctable Errors), ue_count (the number of Uncorrectable Errors), the name of the memory stick (dimm_label) and the size of the memory stick (size_mb).

6.6.5. System and in-path routing tables

The Steelhead appliance has multiple routing tables: One for the primary and auxiliary interface and one for every in-path interface.

6.6.5.1. The system routing tables

The system routing table is used for management traffic send from the Steelhead appliance.

Figure 6.63. System routing table output

Output of 'route -n':

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
10.0.1.0        0.0.0.0         255.255.255.0   U     0      0        0 primary
0.0.0.0         10.0.1.9        0.0.0.0         UG    1      0        0 primary

Output of 'route -Cn':

Kernel IP routing cache
Source      Destination     Gateway         Flags Metric Ref    Use Iface
10.0.1.7    192.168.1.98    10.0.1.9        0      0        2 inpath0_0
10.0.1.7    192.168.1.52    10.0.1.9        0      0        2 inpath0_0
10.0.1.7    192.168.1.175   10.0.1.9        0      0        2 inpath0_0
10.0.1.7    192.168.1.39    10.0.1.9        0      0        2 inpath0_0
[..]

6.6.5.2. The in-path routing tables

The in-path routing tables are the routing tables specifically for the in-path interfaces. For simple networks where there is only one IP subnet behind the Steelhead appliance, most of the time it will only contain the default gateway. For Steelhead appliances which use the simplified routing table it only needs to contain the default gateway.

Figure 6.64. In-path routing table output

Output of 'ip route show table proxytable0_0':

10.0.1.0/24 dev inpath0_0  scope link 
default via 10.0.r.9 dev inpath0_0 onlink 

6.6.6. The output of the arp command

The output of the ARP table contains the IP addresses of the devices on the IP subnets connected to both the primary and auxiliary interface and the in-path interfaces:

Figure 6.65. ARP table overview

Output of 'arp -na':

? (10.0.1.1) at 00:1B:0C:B0:34:38 [ether] on primary
? (10.0.1.9) at 00:0D:B9:17:28:DE [ether] on inpath0_0
? (10.0.1.1) at 00:1B:0C:B0:34:38 [ether] on inpath0_0
? (10.0.1.7) at <incomplete> on inpath0_0

The only MAC address known on the primary interface is the MAC address of the client with which we connect to the CLI of the Steelhead appliance. On the in-path interface there is the MAC address of the default gateway and the MAC address of the clients which have optimized TCP connections. The Steelhead appliance has tried to find out the MAC address of the host with IP address 10.0.1.7 but didn't get answer on its ARP request.

6.6.7. Output of netstat

In RiOS version 8.0 and below, the netstat section shows the open sockets on the Steelhead appliance and the state of the socket.

Figure 6.66. Output of the netstat command

Output of 'netstat -a --numeric-hosts':

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address    Foreign Address    State
tcp        0      0 10.0.1.6:7810    0.0.0.0:*          LISTEN
tcp        0      0 127.0.0.1:8005   0.0.0.0:*          LISTEN
tcp        0      0 0.0.0.0:904      0.0.0.0:*          LISTEN
tcp        0      0 0.0.0.0:8009     0.0.0.0:*          LISTEN
tcp        0      0 0.0.0.0:8333     0.0.0.0:*          LISTEN
tcp        0      0 127.0.0.1:7821   0.0.0.0:*          LISTEN
tcp        0      0 10.0.1.6:http    0.0.0.0:*          LISTEN
tcp        0      0 127.0.0.1:8307   0.0.0.0:*          LISTEN
tcp        0      0 0.0.0.0:8308     0.0.0.0:*          LISTEN
tcp        0      0 127.0.0.1:8086   0.0.0.0:*          LISTEN
tcp        0      0 10.0.1.6:ssh     0.0.0.0:*          LISTEN
tcp        0      0 10.0.1.6:7800    0.0.0.0:*          LISTEN
tcp        0      0 10.0.1.6:7801    0.0.0.0:*          LISTEN
tcp        0      0 10.0.1.6:https   0.0.0.0:*          LISTEN
tcp        0      0 0.0.0.0:8222     0.0.0.0:*          LISTEN
tcp        0      0 10.0.1.6:7801    192.168.1.6:19472  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:55588  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:55591  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:55596  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:58625  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:55597  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:55598  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:55599  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:39427  ESTABLISHED
tcp        0      0 10.0.1.6:7801    192.168.1.6:25651  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:62215  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:59687  ESTABLISHED
tcp        0      0 10.0.1.6:7800    192.168.1.6:57636  ESTABLISHED
tcp        0      0 127.0.0.1:33387  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33388  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33403  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33402  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33401  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33406  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33405  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33404  127.0.0.1:8086     CLOSE_WAIT
tcp        0      0 127.0.0.1:33392  127.0.0.1:8086     TIME_WAIT
tcp        0      0 127.0.0.1:33411  127.0.0.1:8086     FIN_WAIT2

The states here are:

  • LISTEN, when the socket is opened by a process waiting for a new TCP session to be setup to it.

  • ESTABLISHED, when a TCP session has been setup towards the service.

  • CLOSE_WAIT, where the socket is still open on this computer despite having received a TCP FIN from the remote computer.

  • TIME_WAIT, when a TCP session has been torn down properly and is now lingering to catch any leftover packets.

  • FIN_WAIT2, when the socket has been closed but the remote computer has not send the FIN yet.

Normally the Recv-Q and Send-Q should be zero or close to zero. If the values in there are non-zero then it might be an indication of a slow client.

6.6.8. Output of ss

In RiOS version 8.5 and higher, the output of the tool ss, short for socket statistics, is included in the file sysinfo.txt in favour of the netstat command.

Figure 6.67. Output of the command "ss"

==================================================
Output of 'ss -meanoiA inet,unix':

Netid  State      Recv-Q Send-Q     Local Address:Port       Peer Address:Port 
[...]
tcp    LISTEN     0      128      ::ffff:10.0.1.6:7801                 :::*      ino:56032 \
    08 sk:ffff8800703c01c0
         mem:(r0,w0,f0,t0) 
tcp    LISTEN     0      128                   :::443                  :::*      ino:32080 \
     sk:ffff8800635faa80
         mem:(r0,w0,f0,t0) 
[...]
tcp    ESTAB      0      0        ::ffff:10.0.1.6:7800    ::ffff:10.92.31.243:38880  timer \
    :(keepalive,4min58sec,0) ino:40312268 sk:ffff880065d8ca40
         mem:(r0,w0,f0,t0) ts sack reno wscale:8,2 rto:573 rtt:191/71.75 ato:40 cwnd:4 sen \
    d 242.6Kbps rcv_space:31856

After the header, the first socket listens on a specific IP address for connections on TCP port 7801, while the second socket listens on all IP addresses on TCP port 443.

The third socket shows an established connection with the following features and characteristics:

  • ts: TCP Time stamps are used.

  • sack: Selective ACKing can be used.

  • ecn: Explicit congestion notification is used.

  • ecnseen: At least one ECT packets was seen.

  • fastopen: The SYN packet contained data.

  • reno: TCP Reno will be used for network congestion diagnostics.

  • wscale:8,2: TCP Window scaling is used with a factor 8 on the outgoing packets and factor 2 on the incoming packets.

  • ato:40: ???

  • mss:1434: The maximum segment size is 1434 bytes.

  • cwnd:4: The congestion window is 4.

  • ssthresh:?: The slow-start threshold.

  • send 242.6Kbps: Calculated send-speed, based on sender congestion window, sender maximum segment size and round trip time.

  • unacked:2920: Number of bytes not yet acknowledged.

  • retrans:1460: Number of bytes retransmitted for this socket.

  • lost:1460: Number of bytes lost for this socket.

  • sacked:4: Number of frames retransmitted via SACK.

  • fackets:12: Number of frames retransmitted because of a fast ACK.

  • reordering:123: Number of frames which did not arrive in the right order.

  • rcv_rtt:1.3: The receiver round trip time ???

  • rcv_space:31856: The socket receive space, for incoming packets.

  • rto:573: The TCP Retransmission timeout is 573ms.

  • rtt:191/71.75: The round trip time ???

  • cwnd:4: The congestion window is 4.

  • ssthresh:???: The slow-start threshold.

  • qack:???: ???

  • bidir:???: ???

6.6.9. The process list

There are two different process lists available in the system dump. The first one is in the file top_output.txt, which is a snapshot of the process list made by the "top" program. The second one is the output of the "ps" program in the file sysinfo.txt.

6.6.9.1. Output of the top program

Top is a Unix program which gives a continuous updated overview of the CPU load, memory usage, process statistics and the processes.

Figure 6.68. Output of the top program

top - 17:47:03 up  1:53,  0 users,  load average: 0.58, 0.54, 0.48
Tasks:  96 total,   4 running,  92 sleeping,   0 stopped,   0 zombie
Cpu(s):  7.6% us,  3.5% sy,  0.0% ni, 76.9% id, 11.1% wa,  0.1% hi,  0.9% si
Mem:   2054092k total,  2037524k used,    16568k free,     3172k buffers
Swap:  2097136k total,      248k used,  2096888k free,  1557400k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND            
 6693 admin     12  -3 1351m 1.3g 1.1g R   48 66.9   4:34.58 sport              
 6601 admin     15   0 70056  34m 2764 S    4  1.7   2:30.85 statsd             
 4466 admin     15   0  180m  36m 8940 S    1  1.8   0:53.26 mgmtd              

The first part is split in five lines:

  • The system time, uptime, number of users and the 1, 5 and 15 minute average load. The load average is the percentage of time when there are processes running or ready to run.

  • The number of processes in total, the ones currently running, the ones waiting for data to run or for a timer to expire, the number of stopped processes and the number of zombie processes. The last two should be zero.

  • The CPU overview shows the CPU utilization, averaged out from the number of cores in the system. There are seven states: User-land (non-kernel processes), system (kernel processes), niced user-land, idle, waiting for I/O, hardware interrupt handling and software interrupt handling.

  • The real memory numbers: The total amount of memory, the amount used, the amount free and the amount used in various network and disk buffers.

  • The swap memory numbers: The total amount of swap memory, the amount used, the amount free and the amount which is currently not used but can used for swap without having to reinitialize it.

6.6.9.2. Output of ps

This section shows the process list output as seen on Unix machines. It contains the process ID, the parent process ID, the amount of CPU it used, the memory utilization, the start time of the process and the process name.

Figure 6.69. The process lists.

Output of 'ps -Aww -o user,pid,ppid,pcpu,pri,nice,vsize,rss,majflt,tty,stat,wchan,start,bs \
    dtime,command':

USER       PID  PPID %CPU PRI  NI   VSZ  RSS MAJFLT TT       STAT WCHAN   STARTED   TIME C \
    OMMAND
admin        1     0  0.0  23   0  2016  608     14 ?        S    -        May 24   0:11 i \
    nit [3]                                                   
[...]
admin     5081     1  0.0  23   0  8724 5992      1 ?        Ss   -        May 24   0:04 / \
    opt/tms/bin/pm
admin     5082  5081  0.7  23   0 70124 63184     3 ?        Ss   -        May 24  64:57 / \
    opt/tms/bin/mgmtd
admin     7148  5081  0.0  23   0  6784 4320     21 ?        Ss   -        May 24   0:02 / \
    opt/tms/bin/snmpd -f -s -c /etc/snmpd.conf
admin     7149  5081  0.4  24   0 16728 9700      9 ?        Ss   -        May 24  38:39 / \
    opt/tms/bin/statsd
admin     7164  5081  0.0  23   0  6188 2780      5 ?        Ss   -        May 24   0:08 / \
    opt/tms/bin/cmcfc
admin     7177  5081  0.1  23   0  4972 2544      1 ?        Ss   -        May 24  11:19 / \
    opt/tms/bin/crld
admin     7186  5081  0.0  23   0  1540  600      1 ?        Ss   -        May 24   0:00 / \
    usr/sbin/crond -n
admin     7188  5081  0.0  23   0  5984 2936      9 ?        Ss   -        May 24   0:01 / \
    usr/sbin/httpd -D NO_DETACH -f /etc/opt/tms/output/httpd.conf
ntp       7193  5081  0.0  23   0  3760 1588      5 ?        Ss   -        May 24   0:01 / \
    usr/sbin/ntpd -n -u ntp -g

In this output you will see the earlier described processes running on the Steelhead appliance: pm, mgmtd, snmpd, statsd and so on.

6.6.10. The list of currently logged in users

This will show the users logged in and on which terminals they are logged in via:

Figure 6.70. Output of the "who" command

Output of 'who -a':

                        Sep 12 15:34               436 id=si    term=0 exit=0
           system boot  Sep 12 15:34
           run-level 3  Sep 12 15:34                   last=S
                        Sep 12 15:34              2797 id=l3    term=0 exit=0
LOGIN      tty1         Sep 12 15:34              8156 id=1
LOGIN      tty2         Sep 12 15:34              8157 id=2
LOGIN      ttyS0        Sep 14 15:49             24796 id=co
admin    + pts/0        Sep 14 15:46 00:02       16987 (57.200.1.39)
           pts/1        Sep 12 16:56             21448 id=ts/1  term=0 exit=0
LOGIN      ttyS0        Sep 14 13:06             22714 id=S0

6.6.11. The directory listing of /var/opt and /var/log

From a user-land perspective, there is only one partition writable on the Steelhead appliance and that is the /var partition. It is used for storing the log files, uploading new RiOS images, statistics files, keeping state for processes and so on.

Figure 6.71. Contents of /var/log

Output of 'find /var/log -type f -ls':

985101  496 -rw-r--r--   1 admin    root       503664 May 23 23:50 /var/log/sa/sa23
985014  496 -rw-r--r--   1 admin    root       503664 May 24 23:50 /var/log/sa/sa24
985027  496 -rw-r--r--   1 admin    root       503664 May 22 23:50 /var/log/sa/sa22
984975  496 -rw-r--r--   1 admin    root       503664 May 25 23:50 /var/log/sa/sa25
984998 373836 -rw-r--r--   1 admin    root     382423665 May 30 17:46 /var/log/messages
985073    8 -rw-r--r--   1 admin    root         7020 May 30 17:46 /var/log/user_messages
985077    4 -rw-------   1 admin    root         1508 May  3 15:20 /var/log/web_access_log \
    .4.gz
985031 1608 -rw-rw-rw-   1 admin    root      1638508 May 29 00:00 /var/log/sdr.stats.2.gz
985051  268 -rw-r--r--   1 admin    root       270153 May 28 00:00 /var/log/messages.3.gz
985094    4 -rw-------   1 admin    root          101 May 29 00:00 /var/log/web_error_log. \
    2.gz
984999   16 -rw-------   1 admin    root        12392 May 30 17:00 /var/log/oom_profile.lo \
    g.3.gz
985089    4 -rw-------   1 admin    root          100 May 28 00:00 /var/log/web_error_log. \
    3.gz
985064    4 -rw-r--r--   1 admin    root          389 May 29 00:00 /var/log/user_messages. \
    2.gz
985084    4 -rw-------   1 admin    root         2629 May  5 00:00 /var/log/web_access_log \
    .2.gz
[...]

The files sa* contain the System Accounting Records, various log files named messages.*, log files for the web server named web*.

Figure 6.72. Contents of /var/opt

Output of 'find /var/opt ( -name .running -prune ) -o -type f -ls':

608197 104020 -rw-rw-rw-   1 admin    root     106404863 Jan 27  2010 /var/opt/tms/images/ \
    cmc.upgrade.tbz
607889 4336 -rw-------   1 admin    root      4426824 Apr 29  2009 /var/opt/tms/snapshots/ \
    CSH-webasd-20090409-090119.tar.gz
460290 5644 -rw-rw-rw-   1 admin    root      5764037 Apr  9  2009 /var/opt/tms/sysdumps/s \
    ysdump-CSH-20090409-085757.tgz
459922  528 -rw-rw-rw-   1 admin    root       532985 Apr  9  2009 /var/opt/tms/sysdumps/s \
    ysdump-CSH-20090409-090119.tgz
459990 27020 -rw-rw-rw-   1 admin    root     27634079 May 30 16:22 /var/opt/tms/sysdumps/ \
    sysdump-CSH-20110530-162201.tgz

Here you can see a RiOS image uploaded by the CMC, a process dump of the process webasd and three system dumps.

6.6.12. IP/TCP/UDP/ICMP networking statistics

This is the statistics output of the netstat command.

Figure 6.73. Output of the "netstat -s" command

Output of '/bin/netstat -s':

Ip:
    3313608 total packets received
    0 forwarded
    0 incoming packets discarded
    3312203 incoming packets delivered
    5123477 requests sent out
Icmp:
    4786 ICMP messages received
    2 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 56
        echo requests: 4730
    4786 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 56
        echo replies: 4730
Tcp:
    18875 active connections openings
    18098 passive connection openings
    4 failed connection attempts
    2118 connection resets received
    1506 connections established
    3301666 segments received
    3274784 segments send out
    2287 segments retransmited
    0 bad segments received.
    1536 resets sent

6.6.13. NIC interface settings and statistics.

This section shows more details on the Ethernet cards than the ifconfig output shows, for example the speeds the card is able to negotiate to, if it used auto-negotiation or not and the current negotiated speed.

6.6.13.1. Auto-negotiation and link speed

In this output, the NIC is configured for auto-negotiation. During the negotiation, it has advertised 10 Mbps, 100 Mbps and 1000 Mbps speeds and the negotiation came up with 1000 Mbps. Because of the gigabit speed, it was also able to negotiate the Ethernet pause feature, which will allow the NIC to alert the connected device to stop sending new Ethernet frames.

Figure 6.74. Ethtool section for an auto-negotiated Ethernet link

==================================================
Output of 'ethtool wan0_0':

Settings for wan0_0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Supports auto-negotiation: Yes
        Advertised link modes:  10baseT/Half 10baseT/Full
                                100baseT/Half 100baseT/Full
                                1000baseT/Full
        Advertised auto-negotiation: Yes
        Speed: 1000Mb/s
        Duplex: Full
        MDI: mdi
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: on
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000001 (1)
        Link detected: yes

==================================================

In this output, the auto-negotiation is not enabled but its speed and duplex are set to 100 Mbps and Full Duplex:

Figure 6.75. Ethtool section for a fixed speed and duplex Ethernet link

==================================================
Output of 'ethtool lan0_0':

Settings for lan0_0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supports auto-negotiation: Yes
        Advertised link modes:  Not reported
        Advertised auto-negotiation: No
        Speed: 100Mb/s
        Duplex: Full
        MDI: mdi
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: yes

==================================================

In this example, the NIC didn't manage to negotiate an Ethernet link:

Figure 6.76. No Ethernet link negotiated

==================================================
Output of 'ethtool lan0_0':

Settings for lan0_0:
        Supported ports: [ TP ]
        Supported link modes:   10baseT/Half 10baseT/Full 
                                100baseT/Half 100baseT/Full 
                                1000baseT/Full 
        Supports auto-negotiation: Yes
        Advertised link modes:  Not reported
        Advertised auto-negotiation: No
        Speed: Unknown! (65535)
        Duplex: Unknown! (255)
        MDI: Unknown! (0)
        Port: Twisted Pair
        PHYAD: 1
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: d
        Wake-on: d
        Current message level: 0x00000007 (7)
        Link detected: no

==================================================

6.6.13.2. Ethernet flow control

The Ethernet flow control allows a receiver of Ethernet frames to indicate that the sender should pause sending temporary. It is possible that the flow control is allowed only in one direction.

Figure 6.77. Ethernet flow control

==================================================
Output of 'ethtool -a lan0_0':

Pause parameters for lan0_0:
Autonegotiate:  on
RX:             on
TX:             on

==================================================

6.6.13.3. Offloading features

Various offloading features like TCP checksum generation and TCP segmentation offloading.

Figure 6.78. TCP segmentation offloading

==================================================
Output of 'ethtool -a lan0_0':

Offload parameters for lan0_0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: off
large-receive-offload: off
==================================================