Putting It All Together: Host Discovery Strategies
Choosing and Combining Ping Options
Effective scanning requires more than knowing all of the options described in this and previous sections. Users must understand how and when to use them to suit the target network topology and scanning goals.
Most valuable probes
In May 2009 we did some research to find the most effective ping probes. We tested dozens of different probes and host discovery options against thousands of Internet hosts. The results of this research are shown in Table 3.2, which ranks the different probes by effectiveness. Percentages are out of the number of hosts that responded to any of the probes. We tested 90 different probes; this is a subset.
Hosts found | Probe |
---|---|
62.47% | -PE |
44.17% | -PS443 |
43.28% | -PA80 |
43.01% | -PA443 |
42.47% | -PS80 |
40.65% | -PA110 |
40.42% | -PA3389 |
40.41% | -PS110 |
39.89% | -PA22 |
39.62% | -PS21 |
39.62% | -PA21 |
38.75% | -PS22 |
37.50% | -PS3389 |
36.66% | -PP |
31.17% | -PU40125 --source-port 53 --data-length 24 |
29.96% | -PU31338 --source-port 53 --data-length 24 |
29.05% | -PU631 --source-port 53 --data-length 24 |
26.38% | -PU40125 |
26.09% | -PS25 |
25.69% | -PA25 |
25.35% | -PU31338 |
24.71% | -PU631 |
24.15% | -PU53 --source-port 53 --data-length 24 |
22.20% | -PU53 |
9.09% | -PO2 |
9.03% | -PO150 |
7.20% | -PO4 |
4.21% | -PM |
We can pick out some interesting features from
Table 3.2.
The best single probe by far is -PE
, ICMP echo.
Following that are various SYN and ACK probes (-PS
and
-PA
) to common ports. ICMP timestamp,
-PP
, does quite well. The best UDP probes
(-PU
) are those to random high ports, and they are more
effective when combined with --source-port 53 --data-length
24
. SYN and ACK to port 25 (SMTP) are notably worse than other
TCP probes, perhaps because that port is commonly filtered. Various IP
protocol pings (-PO
) are not very effective, and
-PM
, the ICMP address mask request, comes in last.
The best multi-probe combinations are not found simply by picking entries from the top of Table 3.2. That's because some probes find many of the same hosts as another, so using both of them is little better than using just one. We had a computer program find the optimum combination for different numbers of probes, by examining how many hosts each probe found and how much they overlapped with other probes. The best combinations are in Table 3.3. Nmap's default four-probe combination was selected from this list as a compromise between comprehensiveness and speed.
Hosts found | Probe combination | |
---|---|---|
1 probe | 62.47% | -PE |
2 probes | 77.61% | -PE -PA80 |
3 probes | 83.83% | -PE -PA80 -PS443 |
4 probes | 88.64% | -PE -PA80 -PS443 -PP |
5 probes | 91.12% | -PE -PA80 -PS443 -PP -PU40125 --source-port 53 |
6 probes | 92.42% | -PE -PS80 -PS443 -PP -PU40125 -PA3389 --source-port 53 |
7 probes | 93.10% | -PE -PS80 -PS443 -PP -PU40125 -PS3389 -PA21 --source-port 53 |
8 probes | 93.69% | -PE -PS80 -PS443 -PP -PU40125 -PS3389 -PA21 -PU161 --source-port 53 |
TCP probe and port selection
The TCP ping options are some of the most powerful discovery techniques in Nmap. An administrator may be able to get away with blocking ICMP echo request packets without affecting most users, but a server absolutely must respond to SYN packets sent to the public services it provides. Meanwhile, ACK packets often get through non-stateful firewalls. I would recommend using both of SYN and ACK probes, using lists of ports based on any knowledge you might have of the target networks as well as more generally popular ports. Table 3.4 lists the ports that are most likely to be responsive (open or closed) based on empirical testing (see the section called “What Are the Most Popular Ports?”). On hosts with a default-drop filter (the hardest type to reach), these are the ports most likely to elicit a response.
In addition to popular ports such as the ones in the list above, choosing at least one high-numbered port is recommended. Many poorly configured firewalls only have default-deny for the privileged ports, meaning those below 1,024. I usually pick a high numbered port out of the air, such as 40,000 or 10,042, to catch machines behind this sort of firewall.
In choosing the ports to probe, remember to emphasize platform diversity. If you are limiting your ping scan to two ports, HTTP (80) and SSH (22) are probably better than HTTP (80) and HTTPS (443) because the latter two are related web services, and many machines that have HTTPS will often have HTTP available anyway. Finding two accessible ports on the same machine is no better for ping scanning purposes than finding one. The goal is to choose ports so that a broad set of hosts will match at least one of them.
Note that the valuable port table does not include many client-oriented ports such as the ubiquitous Windows SMB port 135. The primary reason is that this table only looked at hosts behind default-deny firewalls, where the vast majority of ports are filtered. In those situations, Windows ports such as 135–139 and 445 are usually blocked. When these machines are not behind a firewall, the open ports are unimportant for ping scanning because the thousands of closed ports work just as well.
UDP port selection
In selecting UDP ports, remember that an open port is unlikely to respond to the probes. Unfiltered ports are desired. To avoid open ports, you might consider excluding common UDP services like DNS (port 53) and SNMP (161). On the other hand, firewall rules are often so broad that those probes (particularly to port 53) might get through and hit a closed port. So I would recommend choosing at least port 53 and an arbitrarily selected high-numbered port such as 37,452.
ICMP probe selection
For ICMP, the standard ping (echo request) is usually worth trying. Many administrators specifically allow this because it is useful for debugging or because RFC 1122 requires it. I would also use at least one of the address mask or timestamp requests. These are valuable for networks where administrators intentionally block echo request packets, but forget about other ICMP queries.
Designing the ideal combinations of probes
How all of these ping types are combined into a ping scan
strategy depends on characteristics of the target network and on the
scan goals. For internal networks, the default ping type usually
works well. The default is also fine for most casual scanning, where
missing an occasional host is no big deal. Adding more probes can
help catch those occasional stealthy machines, at the expense of
making the ping scan take a bit longer. Time taken is roughly
proportional to the number of probes sent to each machine. For
security scans of target networks over the Internet, adding more
probes is usually advisable. Try to include a diverse set of the
techniques discussed previously. Here is a set of ping options that
should catch the vast majority of hosts: -PE -PP
-PS21,22,23,25,80,113,443,31339 -PA80,113,443,10042
. Adding in
--source-port 53
might be worthwhile as well. How
much better will the results be, and how much longer will it take?
That depends on the target network, of course, but the Nmap random
target selection option (-iR
) makes it easy to
perform a quick test. Example 3.13 shows Nmap
generating 50,000 random IP addresses and then performing a default
ping scan. You should remember that the default is four probes: an ICMP
echo request, a TCP SYN to port 443, a TCP ACK to port 80, and an ICMP
timestamp request.
#nmap -n -sL -iR 50000 | awk '/^Host / {print $2}' | sort -n > 50K_IPs
#head -5 50K_IPs
3.0.47.195 3.100.112.251 3.100.98.39 3.10.89.120 3.101.183.79 #nmap -n -sn -T4 -iL 50K_IPs -oA 50KHosts_DefaultPing
Starting Nmap ( https://nmap.org ) Host 4.178.9.27 is up (0.27s latency). Host 12.135.202.138 is up (0.20s latency). Host 12.151.172.161 is up (0.089s latency). [thousands of lines cut] Host 222.99.77.130 is up (0.20s latency). Nmap done: 50000 IP addresses (3927 hosts up) scanned in 2532.05 seconds
Scanning the 50,000 addresses took just over 42 minutes, and 3,927 hosts were detected. To determine the effects of using a wider range of ping techniques, the same 50K hosts were rescanned with 14 probes per port rather than the default of four. As shown in Example 3.14, Nmap was able to detect 785 (20%) more hosts. It took about 147 minutes, which is almost 3.5 times as long. Given all the new hosts detected, that extra time was well spent. Note that not all of the new hosts will be legitimate. Increasing the number of ping probes increases the chances that Nmap will hit network artifacts that make a non-existent host appear to be active. Firewalls that return a RST for SYN or ACK packets to port 113 are one example of this.
# nmap -n -sn -PE -PP -PS21,22,23,25,80,113,443,31339 -PA80,113,443,10042 \
-T4 --source-port 53 -iL 50K_IPs -oA 50KHosts_ExtendedPing
Starting Nmap ( https://nmap.org )
Host 4.238.177.186 is up (0.44s latency).
Host 12.135.202.138 is up (0.13s latency).
Host 12.151.172.161 is up (0.092s latency).
[thousands of hosts cut]
Host 222.94.94.113 is up (0.23s latency).
Nmap done: 50000 IP addresses (4712 hosts up) scanned in 8842.31 seconds
When performing security audits for clients, I normally start
TCP analysis with a port scan against the most common 1,000 ports (the
default) with comprehensive ping scan options like those shown in
Example 3.14, “Repeating
ping scan with extra probes”. Such a scan
does not take particularly long, allowing me to quickly start working.
I also launch -Pn
(ping disabled) scans against all
65K TCP ports in the background while I work. When they finish, which may
be days later, I compare them to my initial quick scan and investigate
any new ports or machines found.