HTTP latency optimization works in various ways:
HTTP meta-data caching.
HTTP object prefetching.
HTTP latency optimization doesn't work when:
When the HTTP request method is not POST nor GET, for example WEBDAV related requests.
When the format of the requested URL or the referrer field is invalid.
When there are multiple HTTP requests in a single TCP session and one of them fails, the rest of the HTTP requests won't be optimized.
A HTTP Reply header can contain the expected life-time of an object:
Figure 8.37. An HTTP Reply header
HTTP/1.0 200 OK Content-Type: image/png Last-Modified: Fri, 15 Jul 2011 23:21:33 GMT Date: Mon, 14 Nov 2011 21:41:50 GMT Expires: Mon, 15 Nov 2011 21:41:50 GMT Cache-Control: public, max-age=86400 X-Content-Type-Options: nosniff Server: example.mavetju.org Content-Length: 35636 X-XSS-Protection: 1; mode=block
The following meta-data can be seen on this response:
The modification time of the object is Fri, 15 Jul 2011 23:21:33 GMT.
The date on the server is Mon, 14 Nov 2011 21:41:50 GMT.
The object can be used in public caches until Mon, 15 Nov 2011 21:41:50 GMT. This is an old notation style.
The object can be used in public caches for 86400 seconds (or 1 day). This is the new notation style.
There are several ways for a web-browser to get an object:
From its browser cache. From the last time the object was requested, the browser was told that it could use it for 86400 seconds.
After the 86400 seconds, the object in the browser-cache has become stale and the web-browser will ask the web-server to send the object again if it has been changed since its Last-Modified date. If it has been changed, the web server will send the new object. If it has not been changed, it will respond with a HTTP 304 response and a new Cache-Control line which states that the stale object can be used for another 86400 seconds. This is a low-bandwidth browser-cache mechanism for refreshing objects.
This answer can either go all the way to the web-server or can be answered by a local web-cache or a server-side web-cache.
Figure 8.38. A 304 response code
GET /foo.png HTTP/1.1 Host: example.mavetju.org User-Agent: Mozilla/5.0 (X11; FreeBSD i386; rv:22.0) Gecko/20100101 Firefox/22.0 Referer: http://example.mavetju.org/ Connection: keep-alive If-Modified-Since: Mon, 10 Mar 2008 18:47:44 GMT HTTP/1.1 304 Not Modified Date: Sun, 29 Sep 2013 17:06:08 GMT Server: Apache/2.2.3 (FreeBSD) Connection: close Expires: Sun, 29 Sep 2013 21:08:31 GMT Cache-Control: public, max-age=86400
The web-browser can ask the web-server to send the object forcing it to come from the web-server without allowing it to be served from any caches.
Figure 8.39. Network with Web-browsers, web-caches and web-servers
browser -- SH -- WAN -- SH -- cache -- WAN -- SH -- server cache -- server
When the web-browser asks for an object and the Steelhead appliance optimizes the HTTP session, it can keep track of the meta-data. If the client asks for an object refresh via an If-Modified-Since and the Steelhead appliance has already learned from a request via another client that the object hasn't changed and a new expiration time has been given, the Steelhead appliance can answer the web-browser that the object is still valid and provide a new expiration time. That doesn't save only bandwidth but also reduces the round trip time.
If the object requested by the web-server is of the type text/html, it can contain embedded objects like images, style sheet and JavaScript files.
If the Steelhead appliance parses the returned text/html object and fetches the embedded objects from the web-server before the web-browser has a chance to request them, then the Steelhead appliance can serve the objects to the web-browser locally instead of the web-browser having to go all the way to the web-server for it.
Unlike proxy servers where the TCP session is terminated on the proxy server, an optimized TCP session is setup to the remote server first before the latency optimization kicks in. Thus the first request will take longer to setup, especially if the Full Transparency WAN visibility is used for the inner channel.
The most common issue is that a requested URL or given referrer contains a space. While technically this is a violation of the syntax, the space should be replaced by a %20, a lot of scripts or web-browsers send it, while web-servers and proxies have been taught to deal with the issue.
Figure 8.40. An invalid URL error messages due to spaces in the URL
CSH sport[24250]: [http/client.INFO] 3277788 {10.0.1.1:24750 192.168.1.1:80} Request parse \ error, code = HTTP_ERR_HEADER_LINE GET /foo /bar.png HTTP:/1.1\013\010Referrer: http: \ //example.mavetju.org/\013\010Host: example.mavetju.org\013\010
By default, the Steelhead appliances will throw an error and not
perform HTTP latency on this TCP session anymore, but with the
command
protocol http space-in-uri enable
it will allow this and process the request anyway.
Figure 8.41. Output of the command "show protocol http"
CSH (config) # protocol http space-in-uri enable You must restart the optimization service for your changes to take effect. CSH (config) # show protocol http internal [..] Allow Parsing Space in URI: yes
When performing the HTTP latency optimization, the client-side Steelhead will add a line in the HTTP response header:
The first part of the string is the name of the client-side Steelhead performing the latency optimization, the middle part contains the RiOS version running on the Steelhead appliance and the third part contains the optimization scheme code.
Table 8.1. HTTP Optimization Scheme Code
UL | URL Learning | The optimization service learns the order of URLs being requested for a certain base URL and will request the next ones once a base URL gets requested |
PP | Parse-and-Prefetch | The optimization service is parsing the return HTML code for objects it can request in advance |
PT | Object Prefetch Table (OPT) | Response is served from the local Steelhead HTTP cache |
MC | Metadata Response | |
RA | Reuse-Auth | |
FN | Force-NTLM | |
SA | Strip-Auth-hdr | |
GR | Gratuitous-401 | The client-side Steelhead appliance replied with a 401 or 403 |
SC | Strip-Compress | The header field "Accept-Encoding: gzip, deflate" has been removed |
IC | Insert-Cookie | |
IK | Insert-Keep-alive | Despite that the client didn't request it, the optimization service inserted a "Connection: Keep-Alive" in the request |
IF | Inflight-Cache (Stream Splitting) |
Further information about the internals of the HTTP optimization
can be enabled with the command
protocol http x-debug enable
.
Figure 8.43. Enabling the HTTP debugging feature
CSH (config) # protocol http x-debug enable CSH (config) # show protocol http internal Insert Pre-Fetch X-Header: yes Insert X-RBT-Debug header: yes [...]
In the HTTP response, this will add a new line:
X-RBT-Debug
.
The first field, 0x81, is the optimization scheme and its value represents a bitmask which explains which features are enabled for the request:
Table 8.2. Optimization scheme bit mask
0x0001 | URL Learning |
0x0002 | Parse and Prefetch |
0x0004 | Object Prefetch Table |
0x0008 | Reuse NTLM Authentication |
0x0010 | Force Negotation NTLM |
0x0020 | Strip authentication header |
0x0040 | Gratuitous 401 |
0x0080 | Strip Compression |
0x0100 | Insert Cookie |
0x0200 | Insert keep-alive |
0x010000 | Frontpage Extensions |
0x020000 | WebDAV support |
0x040000 | FSS HTTP |
So for the value of 0x81 the features used are Strip Compression and URL Learning.
This value is also seen in the logs for that TCP session:
Figure 8.45. HTTP Optimization scheme
CSH sport[16611]: [http/client.INFO] 6629561 {10.0.1.1:65310 192.168.1.1:80} codec flow co \ ntrol is disabled CSH sport[16611]: [http/client.INFO] 6629561 {10.0.1.1:65310 192.168.1.1:80} opt_scheme_ u \ pdated to 129 from auto config table
In this example, the 129 decimal is 0x81 hexadecimal, relates to the same features as seen earlier.
The next fields are the actions and decisions of the HTTP optimization code. The four letters are part of a code:
The first letter is the status:
S from Success
F from Failure
I from Information
The second letter is the category:
C from Cache
P from Prefetch
M from Miscellaneous
The third and the fourth letter are the action and specific reason codes.
So the code SPIP would be a success from the prefetching functionality and the code FPSF would be a failure for the prefetching functionality.
Table 8.3. Cache related category
FCAA | Failed to cache because the content is authenticated but cache_without_auth is off |
FCAE | Failed to cache because the extension is not present in the cache extension list |
FCAI | Invalid request, e.g., absence of host header |
FCAL | The body is too big to cache |
FCAS | The remaining lifetime is too short to cache |
FCAU | Not allowed to cache, e.g., cache-control header |
FCS2 | The current cache entry only has the header |
FCSA | Failed to serve due to the absence of Accept-Encoding header |
FCSC | The Steelhead appliance is not configured to serve full body cache |
FCSD | Failed to serve from cache because of a different encoding |
FCSE | Failed to serve from cache because the extension is not present in the cache extension list |
FCSI | Invalid request, e.g., absence of host header |
FCSM | Miscellaneous reason |
FCSN | The object is not present in cache |
FCSU | The object is not allowed to cache, e.g., cache-control header |
FCSX | Failed to serve cached object because it has been expired |
SCAN | The object is added to cache |
SCAR | The object has been refreshed in the cache |
Table 8.4. Prefetch related category
FPIA | Failed to initiate prefetching because authentication is required. OR, no response has been received through this connection so the Steelhead appliance is not yet aware if authentication is needed or not |
FPID | Failed to add the URL to the APT database |
FPIE | Didn't initiate prefetching because inner channel is being terminated |
FPIH | Failed to initiate prefetching because a header is missing (most likely cookie header) |
FPIM | Reached the maximum prefetch limit |
FPLF | Failed to learn the object because APT is full |
FPLH | Failed to learn the object due to the absence of the header |
FPLM | Failed to learn the object because the Steelhead appliance is occupying its MAX MEM |
FPSA | Authentication is required |
FPSE | APT request was EOF'ed |
FPSF | APT was not found for the object |
FPSN | APT request was not sent |
FPSR | Received error status code for APT request |
IPGP | APT is generated via PnP |
IPGU | APT is generated via URL Learning |
IPLP | The object has been purged from UL APT tree |
IPSC | The prefetched object is served through callback, i.e., the request was halted for the APT response to come. |
SPIP | This object triggered prefetching |
SPLN | The object was learned by UL |
SPPP | The page is parsed for PnP |
Table 8.5. Miscellaneous category
FM4S | Received a 401 despite that strip authentication took the strip header off |
FMGS | Failed to store the 401 response because the body is shorter than specified |
IMAA | Expecting an authenticaton header in the request but it was absent |
IMAC | Stopped storing the body for processing because the Steelhead appliance is in a LOW MEM state |
IMBR | This is a base request |
IMIR | The Steelhead appliance inserted a referer header |
SMGN | The 401/407 response is newly added |
SMGR | The 401/407 response cache entry is refreshed |