To setup a new optimized TCP session, the Steelhead appliance at the client-side needs to know if there is another Steelhead in the path and where to setup the inner channel to.
The setup of a TCP session happens with the three way handshake[SOURCE:RFC 793] where the client sends a TCP packet with the SYN flag set in the TCP header, the server replies with a TCP packet with the SYN and ACK flags set in the TCP header and the client replies with a TCP packet with the ACK flag set in the TCP header.
Figure 2.5. The setup of a normal TCP session without Steelhead appliances
State Client Server Time Send TCP SYN S >----- '----- | Receive TCP SYN '-----> | | Send TCP SYN/ACK -----< SA | -----' | Receive TCP SYN/ACK <-----' v Send TCP ACK A >----- '----- Receive TCP ACK '-----> Data can be exchanged >-----------------> forwards and backwards <-----------------< >-- initiated by --> received by
With a pair of Steelhead appliances in the path, they will see this setup of the TCP session:
Figure 2.6. The setup of a normal TCP session seen by Steelhead appliances
Line State Client CSH SSH Server Time 1 Send TCP SYN S >-> 2 --> | 3 Receive TCP SYN --> | 4 | 5 Send TCP SYN/ACK <-< SA | 6 <-- | 7 Receive TCP SYN/ACK <-- v 8 9 Send TCP ACK A >-> 10 --> 11 Receive TCP ACK --> 12 Data can be exchanged >-- --- --> 13 forwards and backwards <-- --- --< >-- initiated by --> received by
When the Steelhead appliance sees a TCP packet with the SYN flag set coming in on the LAN side, it will know that this is a new TCP session initiated by a local client. The Steelhead appliance will check the In-path Rules table to determine what should happen with the TCP session:
It can be passed through, the Steelhead appliance does not intercept it.
It can be optimized to another Steelhead appliance configured by a Fixed Target rule. This happens unconditionally, the other Steelhead appliances cannot ignore this.
It can be tagged for optimization via the auto-discovery protocol: The Steelhead appliance adds an auto-discovery probe in the TCP options part of the TCP header so any other Steelhead appliance in the path will know that this TCP session can become an optimized TCP session. The other Steelhead appliance will check its Peering Rules table to determine if it wants to optimize with this Steelhead appliance or for this TCP session.
A TCP SYN packet without the auto-discovery probe is called a naked SYN. A TCP SYN packet with the auto-discovery probe is called a SYN+. A TCP SYN/ACK with the auto-discovery probe is called a SYN/ACK+.
If a naked SYN is seen on the WAN side of the Steelhead appliance, it will ignore it and mark the TCP session as pass-through Note that this is only valid for true in-path designs. For virtual in-path designs where the traffic is forwarded via WCCP, PBR or Interceptors, all traffic is received via the WAN interface and thus it will accept and process a naked SYN packet on the WAN side.
When two Steelhead appliances see each other for the first time, either via auto-discovery or via a fixed target rule, they will start with the setup an Out-of-Band (OOB) Splice. This is a control TCP session between the two Steelhead appliances, used to test the connectivity towards between the two Steelhead appliances.
Figure 2.7. Setup of a new OOB Splice
SH sport[24798]: [splice/oob.INFO] 1 {- -} New OOB Splice created at 10.0.1.6 for peer 192 \ .168.1.6 SH sport[24798]: [mgmt.INFO] - {- -} mgmtd notified that remote peer 192.168.1.6 was disco \ vered SH sport[24798]: [splice/oob.INFO] 1 {- -} Establishing OOB Splice from 10.0.1.6:0 to 192. \ 168.1.6:7800 SH mgmtd[3766]: [mgmtd.INFO]: EVENT: /rbt/sport/peer/event/added SH sport[24798]: [splice/oob.INFO] 1 {- -} OOB Splice connected from 10.0.1.6:40271 to 192 \ .168.1.6:7800 SH sport[24798]: [splice/oob.INFO] 1 {- -} Negotiated sport protocol version: 8 for peer: \ 192.168.1.6:7800
It has the following behaviour:
After the setup of the OOB Splice, the two Steelhead appliances will exchange system information and a list of capabilities. This information contains the hostname, RiOS version and IP addresses. This capabilities list will contain the latency optimizations supported and help the two Steelhead appliances to match the level of these capabilities.
When a Steelhead appliance goes into admission control or the optimization service gets restarted, the OOB Splice will be closed.
Figure 2.8. Termination of an OOB Splice due to admission control
SH sport[6693]: [splice/oob.NOTICE] 8 {- -} Received disconnect cause="Admission control" \ laddr=10.0.1.6:7800 raddr=192.168.1.6:44264 SH sport[6693]: [splice/oob.NOTICE] 8 {- -} Lost OOB Splice between laddr=10.0.1.6:7800 an \ d raddr=192.168.1.6:7800
The first line is the message that the other side goes into Admission Control. The second line is that the OOB Splice is about to be terminated.
The OOB Splice has a TCP Keep Alive interval of 20 seconds. This means that if there is a network routing issue towards the other side, within one minute the Steelhead appliances will know that the other side is unreachable. Except for these TCP keep-alive packets there is no other data being transferred during normal operations.
Once the OOB Splice is setup, the optimization service will setup a Connection Pool towards the remote Steelhead appliance. This is a pool of 20 TCP sessions which can later be used to convert to inner channels. Every time one of these TCP sessions is used to create an inner channel, a replacement TCP session for the pool will be setup. The TCP Keep Alive interval is very large set to 20 minutes.
The Connection Pool sessions is maintained as long as the OOB Splice is established. If the OOB Splice gets terminated, the TCP sessions in the unused Connection Pool will be terminated too.
Figure 2.9. Termination of the Connection Pool because of the loss of the OOB Splice
SH sport[6693]: [splice/oob.NOTICE] 8 {- -} Lost OOB Splice between laddr=10.0.1.6:7800 an \ d raddr=192.168.1.6:7800 SH sport[6693]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6 SH sport[6693]: [spawnsplice.NOTICE] - {- -} (from 192.168.1.6:23210) End of stream readin \ g splice hdr info. Peer maybe down.
The default set of in-path rules consist of three pass-through ("don't try to optimize this traffic") statements and a default auto-discovery rule:
Pass-through of encrypted or secure traffic.
Pass-through of interactive traffic.
Pass-through of Riverbed protocols related to the optimization process.
Auto-discovery of everything else.
Figure 2.10. In-path rules on a Steelhead appliance.
SH # show in-path rules Rule Type P O L N W K VLAN Source Addr Dest Addr Port ----- ---- - - - - - - ---- ------------------ ------------------ -------------- 1 pass - - - - - - all all all Secure 2 pass - - - - - - all all all Interactive 3 pass - - - - - - all all all RBT-Proto def auto N F F A C N all all all all 3 user-defined rule(s) (P) Preoptimization Policy: O=Oracle-Forms S=SSL +=Oracle-Forms-over-SSL N=None (O) Optimization Policy: F=Full S=SDR-only C=Compression-only M=SDR-M N=None (L) Latency Optimizations: F=Full H=HTTP-only O=Outlook-anywhere N=None (N) Neural Framing: A=Always D=Dynamic T=TCP hints N=Never (W) WAN Visibility Mode: C=Correct-Addressing P=Port-Transparency F=Full-Transparency R=Full-Transparency w/Reset (K) Auto Kickoff: Y=Enabled N=Disabled
In-path rules can be used to prevent optimization towards certain machines or services. They can also be used to set specific options for specific optimized TCP session with regarding to WAN visibility, data reduction behaviour, pre-optimization features and so on.
The in-path rules are parsed in order of definition. Global pass-through, auto-discovery and fixed target in-path rules should be added after the default in-path rules. Specific auto-discovery in-path rules should be added to the end of the list of in-path rules. This way ensures that you prevent optimization before you start allowing it.
Figure 2.11. Additional fixed target in-path rule on a Steelhead appliance.
SH # show in-path rules Rule Type P O L N W K VLAN Source Addr Dest Addr Port ----- ---- - - - - - - ---- ------------------ ------------------ -------------- 1 pass - - - - - - all all all Secure 2 pass - - - - - - all all all Interactive 3 pass - - - - - - all all all RBT-Proto 4 fixe N F F A - N all all 192.168.1.1/32 all Target: 192.168.1.6 7810 def auto N F F A C N all all all all
Fixed Target rules specify the remote Steelhead appliance to be used for optimization of a certain set of traffic. It is an administrative task of the order of O(N!) if you would do it for every Steelhead appliance in the network.
Fixed Target rules are often used in environments where auto-discovery isn't possible because other devices in the network are stripping the auto-discovery probe or where the auto-discovery process fails because of policies on firewalls and NAT gateways.
Figure 2.12. The setup of an optimized TCP session using Fixed Target rules.
Line State Client CSH SSH Server Time 1 Send TCP SYN S >-> 2 Setup inner channel >-> | via Connection Pool ( 3 Setup of new TCP S >-> ) | ( 4 session for the <-< SA ) | ( 5 Connection Pool A >-> ) | 6 Forward TCP SYN S >-> v 7 Receive TCP SYN/ACK <-< SA 8 Send TCP ACK A >-> 9 <-< 10 Receive TCP SYN/ACK <-< SA 11 Send TCP ACK A >-> 12 Data can be exchanged >-> >-> >-> 13 forwards and backwards <-< <-< <-< >-- initiated by --> received by
When a client-side Steelhead appliance adds an auto-discovery probe the TCP header of a TCP SYN packet, it will inform any other Steelhead appliances in the path that this TCP session can be optimized. The server-side Steelhead appliance will reply with a SYN/ACK+ packet containing its IP address and TCP port number. When the client-side Steelhead appliance sees the SYN/ACK+ packet, it will setup an inner channel towards the server-side Steelhead appliance.
Figure 2.13. The setup of an optimized TCP session using the default auto-discovery process
Line State Client CSH SSH Server Time 1 Send TCP SYN S >-> 2 Forward SYN+ S+ >-> | 3 Receive SYN/ACK+ <-< SA+ | 4 Setup inner channel >-> | ( 5 Setup of new TCP S >-> ) | ( 6 session for the <-< SA ) | ( 7 Connection Pool A >-> ) | 8 Forward TCP SYN S >-> v 9 Receive TCP SYN/ACK <-< SA 10 Send TCP ACK A >-> 11 Server side TCP is setup <-< 12 Receive TCP SYN/ACK <-< SA 13 Send TCP ACK A >-> 14 Data can be exchanged >-> >-> >-> 15 forwards and backwards <-< <-< <-< >-- initiated by --> received by
In the default auto-discovery process the inner channel towards the first Steelhead appliances seen gets setup before the outer channel to the server is setup.
Figure 2.14. The setup of an optimized TCP session using the enhanced auto-discovery process
Line State Client CSH SSH Server Time 1 Send TCP SYN S >-> 2 Forward SYN+ S+ >-> | 3 Tell CSH that a SH was seen <-< SA* | 4 Forward SYN+ S+ >-> | 5 Receive TCP SYN/ACK <-< SA | 6 Receive SYN/ACK+ <-< SA+ | 7 Setup inner channel >-> | ( 8 Setup of new TCP S >-> ) | ( 9 session for the <-< SA ) | (10 Connection Pool A >-> ) | 11 Send TCP ACK A >-> v 12 Server side TCP is setup <-< 13 Receive TCP SYN/ACK <-< SA 14 Send TCP ACK A >-> 15 Data can be exchanged >-> >-> >-> 16 forwards and backwards <-< <-< <-< >-- initiated by --> received by
In the enhanced auto-discovery process the outer channel towards the server is determined before the inner channel is setup, making it possible to auto-discover the Steelhead appliance furthest in the network.
Peering rules define what the Steelhead appliance should do when it receives a SYN+ packet. It can chose to accept the auto-discovery attempt and reply with a SYN/ACK+ or to ignore it and pass it on.
Figure 2.15. Peering rules on a Steelhead appliance.
SSH (config) # show in-path peering rules Rule Type Source Destination Port Peer SSL-Cap ----- ---- ------------------ ------------------ ------ ---------------- ------- 1 pass all all all all in-cap desc: Default rule to passthrough connections destined to SSL servers 2 auto all all 443 all cap desc: Default rule to attempt to optimize connections destined to port 443 as SSL def auto all all all all no-chk -------------------------------------------------------------------------------- 2 user added rule(s)
Peering rules cannot be used to block optimized TCP sessions from Fixed Target in-path rules coming from other Steelhead appliances.