3.6. Traceroute

Traceroute is a basic command in the networking world to determine the path the traffic takes to a remote host. It sends a number of UDP packets with different source ports and with increasing TTL value. The source address of the returned ICMP TTL Exceeded replies will show the IP addresses of the path.

Besides the traceroute command, used for the IPv4 transport layer, there is also the traceroute6 command, used for the IPv6 transport layer. The options used for the IPv4 version can also be used for the IPv6 version unless noted differently.

With the returned ICMP packets, the IP addresses on it are often the IP address of the outgoing interface on that router which might not be the IP address of the incoming interface of that router. So to confirm the path, the traceroute program should to be run on both the client and the server.

Figure 3.30. WAN router receives packet via a different path than it send it

WAN router receives packet via a different path than it send it

3.6.1. Traceroute options

The following options are available for the traceroute command:

  • The option -n prevents name-resolution of the IP addresses displayed.

  • The option -s <IP address> changes the source IPv4 or IPv6 address to use for and thus the outgoing interface.

  • The option -S # changes the size of the payload of the outgoing UDP packet.

  • The option --mtu can be used to identify the maximum MTU size towards the remote host.

3.6.2. Traceroute examples

The traceroute command uses by default the IP address of the primary interface:

Figure 3.31. The command to traceroute to a remote host via the primary interface

SH # traceroute -n 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 172.16.2.2  90.657 ms  90.245 ms  90.342 ms 
 4 172.16.3.2  130.657 ms  130.245 ms  130.342 ms 
 5 192.168.1.1  130.857 ms  130.545 ms  130.642 ms 

The -s option can be used to specify the IP address of an in-path interface:

Figure 3.32. Traceroute to a remote host via an in-path interface

SH # traceroute -n -s 10.0.1.5 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1) from 10.0.1.5, 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 172.16.2.2  90.657 ms  90.245 ms  90.342 ms 
 4 172.16.3.2  130.657 ms  130.245 ms  130.342 ms 
 5 192.168.1.1  130.857 ms  130.545 ms  130.642 ms 

3.6.3. MTU Path Detection with traceroute

The ping command can be used to determine that there are MTU size issues in a network, the traceroute command can be used to determine where it is happening.

If the network is properly informing with ICMP Fragmentation Needed packets, the --mtu option can be used to see it:

Figure 3.33. Determining the MTU size with the --mtu option

CSH # traceroute -n --mtu 192.168.1.1
traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 65000 byte packets
 1 10.0.1.9  0.657 ms F=1422  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 172.16.2.2  90.657 ms  90.245 ms  90.342 ms 
 4 172.16.3.2  130.657 ms  130.245 ms  130.342 ms 
 5 192.168.1.1  130.857 ms  130.545 ms  130.642 ms 

The detected maximum MTU size here is 1422 bytes.

If the network is not sending the ICMP Fragmentation Needed packets, the same method by increasing and decreasing the packet size as used with the ping command should be used.

First the path between towards the remote host needs to be determined:

Figure 3.34. Successful traceroute between two hosts

SH # traceroute -n 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 172.16.2.2  90.657 ms  90.245 ms  90.342 ms 
 4 172.16.3.2  130.657 ms  130.245 ms  130.342 ms 
 5 192.168.1.1  130.857 ms  130.545 ms  130.642 ms 

Now the same trace but with a packet size larger than the MTU size of the link:

Figure 3.35. Failed traceroute due to MTU size issues

SH # traceroute -n -S 1480 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 1480 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 * * *
 4 * * *
 5 * * *

The router with the IP address of 172.16.1.2 did forward the packet, but the router with the IP address of 172.16.2.2 did not return any answers. Use a smaller option for the size to determine the right maximum MTU size.

3.6.4. Determine failure scenarios

3.6.4.1. The trace doesn't show data for certain hops.

In this example the traceroute command returned no data for hop 3:

Figure 3.36. Missing reply for hop three.

SH # traceroute -n 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 * * *
 4 172.16.3.2  130.657 ms  130.245 ms  130.342 ms 
 5 192.168.1.1  130.857 ms  130.545 ms  130.642 ms 

This could happen when for example:

  • A firewall doesn't allow ICMP TTL Expired messages from the IP addresses on hop 3.

  • The gateway in hop 3 is instructed not to send out ICMP TTL Expired messages.

As the end-to-end communication works, the missing hop may be ignored.

3.6.4.2. The trace doesn't show data after a certain hop.

The traceroute command could have returned only *'s after a certain hop:

Figure 3.37. Missing reply for hop three and following.

SH # traceroute -n 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 * * *
 4 * * *
 5 * * *

This can happen when:

  • The network after hop 2 doesn't know the way back to the source IP subnet of the trace of 10.0.1.0/24.

  • The network after hop 2 doesn't know the way to 192.168.1.1 but doesn't send back ICMP Network Unreachable. If it did, the output would have ended with:

    Figure 3.38. Network Unreachable response

       3  192.168.1.1  3000.123 ms !N  3000.231 ms !N  3000.349 ms !N
    


  • A firewall between hop 2 and 3 is blocking the traffic.

3.6.4.3. The trace doesn't return anything for the last hop.

The traceroute command could have returned *'s for the latest hop:

Figure 3.39. Missing reply for the latest hop.

SH # traceroute -n 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 172.16.2.2  90.657 ms  90.245 ms  90.342 ms 
 4 172.16.3.2  130.657 ms  130.245 ms  130.342 ms
 5 * * *

This could happen when for example:

  • When the firewall on the remote host is configured not to send ICMP Port Unreachable.

  • When the destination IP address is not configured on that IP subnet and the final router did not send back ICMP Host Unreachable. If it did, the output would have ended with:

    Figure 3.40. Host Unreachable response

       5  192.168.1.1  3000.123 ms  !H 3000.231 ms !H  3000.349 ms !H
    


3.6.4.4. The trace did contain multiple IP addresses for a hop

This happens when a hop is doing load balancing over multiple links. In that case the outgoing interface is pretty much random.

Figure 3.41. Hop 4 has multiple outgoing interfaces

SH # traceroute -n 192.168.1.1
Traceroute to 192.168.1.1 (192.168.1.1), 30 hops max, 38 byte packets
 1 10.0.1.9  0.657 ms  0.245 ms  0.342 ms 
 2 172.16.1.2  30.657 ms  30.245 ms  30.342 ms 
 3 172.16.2.2  90.657 ms  90.245 ms  90.342 ms 
 4 172.16.3.2  130.657 ms
   172.16.3.10  130.245 ms
   172.16.3.2  130.342 ms 
 5 192.168.1.1  130.857 ms  130.545 ms  130.642 ms