9.4. OOB Splice and Connection Pool

This section is about the setup and termination of the Out-of-Band splice and the TCP sessions from the Connection Pool.

9.4.1. OOB Splice lost due to admission control

When a Steelhead appliance goes into admission control, it will send the OOB disconnect message Admission Control to the connected Steelhead appliances and terminate the TCP sessions of the connection pool.

Figure 9.18. Local Steelhead appliance into admission control

CSH statsd[7333]: [statsd.NOTICE]: Alarm triggered for rising error for event admission_co \
    nn 
CSH sport[8298]: [admission_control.NOTICE] - {- -} Connection limit achieved. Total Conne \
    ctions 136, Branched Warmed Connections 0 
CSH sport[8298]: [admission_control.WARN] - {- -} Pausing intercept: Connection limit achi \
    eved;  
CSH sport[8298]: [admission_control.NOTICE] - {- -} Memory Usage: 296 Current Connections: \
     136 TCP Memory Usage: 29; 
CSH sport[8298]: [splice/oob.NOTICE] 11 {- -} Resetting state for oob splice: laddr=10.0.1 \
    .6:43437 raddr=192.168.1.6:7800 cflag=0x8 
CSH sport[8298]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6

Figure 9.19. Remote Steelhead appliance into admission control

CSH sport[6693]: [splice/oob.NOTICE] 8 {- -} Received disconnect cause="Admission control" \
     laddr=10.0.1.6:7800 raddr=192.168.1.6:26539
CSH sport[6693]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6
CSH sport[6693]: [splice/oob.NOTICE] 8 {- -} Lost OOB Splice between laddr=10.0.1.6:7800 a \
    nd raddr=192.168.1.6:7800
CSH sport[6693]: [spawnsplice.NOTICE] - {- -} (from 192.168.1.6:26607) End of stream readi \
    ng splice hdr info. Peer maybe down.
CSH sport[6693]: [spawnsplice.NOTICE] - {- -} (from 192.168.1.6:26582) End of stream readi \
    ng splice hdr info. Peer maybe down.
CSH sport[6693]: [spawnsplice.NOTICE] - {- -} (from 192.168.1.6:26578) End of stream readi \
    ng splice hdr info. Peer maybe down.
[...]
CSH sport[6693]: [spawnsplice.NOTICE] - {- -} (from 192.168.1.6:26545) End of stream readi \
    ng splice hdr info. Peer maybe down.
CSH sport[6693]: [spawnsplice.NOTICE] - {- -} (from 192.168.1.6:26556) End of stream readi \
    ng splice hdr info. Peer maybe down.

9.4.2. Inner channel setup failures

When an inner channel gets setup and fails, the existing OOB Splice gets torn down:

Figure 9.20. Remote Steelhead appliance failed to setup an inner channel

CSH sport[26658]: [splice/oob.NOTICE] 12 {- -} Received disconnect cause="TproxyClientSpli \
    ce connect failure" laddr=10.0.1.6:57962 raddr=192.168.1.6:7800 
CSH sport[26658]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6
CSH sport[26658]: [splice/oob.NOTICE] 12 {- -} Lost OOB Splice between laddr=10.0.1.6:5796 \
    2 and raddr=192.168.1.6:7800 

9.4.3. High Availability switch

In a High Availability setup, where one Steelhead appliance is the primary and takes all connections and the second Steelhead appliance is the backup, the switch in roles shows like this:

Figure 9.21. Remote Steelhead becomes the backup in a High Availability cluster

CSH sport[26658]: [splice/oob.NOTICE] 12 {- -} Received disconnect cause="FailoverSplice r \
    eleased ownership" laddr=10.0.1.6:57962 raddr=192.168.1.6:7800 
CSH sport[26658]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6
CSH sport[26658]: [splice/oob.NOTICE] 12 {- -} Lost OOB Splice between laddr=10.0.1.6:5796 \
    2 and raddr=192.168.1.6:7800 

9.4.4. Voluntary takedown of OOB Splice

The keep-alive packets going over the OOB Splice and the Connection Pool can be expensive with for example satellite links. It is possible for the OOB Splice to be taken down when there are no active optimized TCP sessions:

Figure 9.22. Local Steelhead appliance disconnects

CSH sport[15982]: [splice/oob.NOTICE] 1 {- -} Resetting state for oob splice: laddr=10.0.1 \
    .6:37964 raddr=192.168.1.6:7800 cflag=0x200 
CSH sport[15982]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6
CSH sport[15982]: [splice/OOBHandler.NOTICE] - {- -} Dropping oob because of no sessions t \
    o 192.168.1.6:7800 

Figure 9.23. Local Steelhead appliance disconnects

CSH sport[15982]: [splice/oob.NOTICE] 1 {- -} Received disconnect cause="No connections to \
     Peer" laddr=10.0.1.6:37957 raddr=192.168.1.6:7800 
CSH sport[15982]: [connect_pool.NOTICE] - {- -} Destroying pool to peer: 192.168.1.6
CSH sport[15982]: [splice/oob.NOTICE] 1 {- -} Lost OOB Splice between laddr=10.0.1.6:37957 \
     and raddr=192.168.1.6:7800