Scan Time Reduction Techniques
The ideal solution to long scan times is to reduce them. This section offers many high-level tips for doing so. Unlike many circumstances in life, tuning your Nmap command line can make a huge difference. Hot-rodding your Honda Accord with a coffee-can exhaust tip, a three-foot-high spoiler, and a big red “type R” sticker won't reduce your 0–60 time much. Yet the section called “Scanning 676,352 IP Addresses in 46 Hours” describes how Jack Mogren shaved days off his Nmap runtime by simply adding a few stickers (I mean options) to his Nmap command line.
Omit Non-critical Tests
The electronic equivalent to buying a Hummer when you never leave the pavement or carry more than groceries is to launch an intense and comprehensive Nmap scan to obtain a relatively trivial amount of information. Wasting a few seconds per host rarely matters on a home network, but can make daily WAN scans infeasible for large enterprises. The following list details ways to avoid common over-scanning mistakes, starting with the most egregious problems and followed by more subtle optimizations that even advanced users often forget.
-
Skip the port scan (
-sn
) when you only need to determine what hosts are online. Some people determine whether a host is online using the command nmap
<hostname>
. While this works, it is overkill. Nmap will send four packets to determine that the host is up, then at least 1,000 to port scan the host. The problem is amplified when a whole network is scanned this way to find all online hosts, or one particular host.Rather than waste time port scanning, specify
-sn
to do a ping scan when all you wish to know is what hosts are up or what their MAC addresses are.- Limit the number of ports scanned.
By default, Nmap scans the most common 1,000 ports. On a fast network of responsive machines, this may take a fraction of a second per host. But Nmap must slow down dramatically when it encounters rate limiting or firewalls that drop probe packets without responding. UDP scans can be agonizingly slow for these reasons. Yet the vast majority of open ports fall into just a few hundred port numbers. A port scan will be about 10 times as fast if you only scan 100 ports instead of the default 1,000. You can scan just the most popular 100 ports with the
-F
(fast scan) option, specify an arbitrary number of the most commonly open ports with--top-ports
, or provide a custom list of ports to-p
.- Skip advanced scan types (
-sC
,-sV
,-O
,--traceroute
, and-A
). Some people regularly specify the
-A
Nmap option, which gives them the works. It causes Nmap to do OS detection, version detection, script scanning (NSE), and traceroute as well as the default port scan. Version detection can be extraordinarily useful, but can also bog down a large scan. So can NSE. When pressed for time, you can always skip-sC
and-sV
on the large scale scan and then perform them on individual ports as necessary later.OS detection is not nearly as slow as version detection, but it can still easily take up 5–10 seconds per online host. Even without this, you can often guess the OS based on the name, open ports, and MAC address on a LAN. And in many cases you may not care about the OS. So
-O
is another candidate for only-as-necessary use. As a compromise, you can specify--osscan-limit --max-os-tries 1
which tells Nmap not to retry OS detection attempts which fail to match, and also to skip OS detection against any online hosts that don't have at least one open TCP port and one closed TCP port. OS detection isn't as accurate against such hosts anyway.- Remember to turn off DNS resolution when it isn't necessary.
By default, Nmap performs reverse-DNS resolution against every host that is found to be online. It is done against all hosts if you skip the ping step with
-Pn
or specify-R
. This was a major bottleneck when host DNS libraries were used to look up one IP at a time.While Nmap now has a fast parallel reverse-DNS system to speed queries, they still can take a substantial amount of time. Disable them with the
-n
option when you don't need the data. For simple scans (such as ping scans) against a large number of hosts, omitting DNS can sometimes reduce scan time by 20% or more. DNS time is not a major factor in more involved scans which probe thousands of ports or utilize intensive features such as version detection. If you want the Nmap host machine to handle name resolution (using thegethostbyaddr
function), specify the--system-dns
option. Doing so can slow scans down dramatically.
Optimize Timing Parameters
Nmap offers dozens of options for providing hints and rules to
control scan activity. These range from high level timing
aggressiveness levels provided by the
-T
option
(described in the section called “Timing Templates (-T
)”) to the
finer-grained controls described in
the section called “Low-Level Timing Controls”. You can even combine the
two. These options are particularly useful when scanning highly
filtered networks where Nmap receives few responses to determine its
own timing estimates. Scan time can often be safely cut in half.
Most of these options will have little effect against a local LAN
filled with responsive hosts, as Nmap can determine optimal values
itself in that case.
Separate and Optimize UDP Scans
Scanning UDP ports is important because many vulnerable services use that protocol, but the timing characteristics and performance requirements of UDP scans are much different than TCP scans. Of particular concern is ICMP error rate-limiting, which is extremely common and affects UDP scans far more often than TCP.
For these reasons, I don't recommend combining TCP and UDP scans
when performance is critical, even though Nmap supports doing so with
options such as -sSU
. You often want
different timing flags for each protocol, requiring separate command lines.
the section called “Speeding Up UDP Scans” provides valuable tricks
and real-life examples for improving UDP scan performance.
Upgrade Nmap
There have been many cases where I have investigated reports of
poor Nmap performance only to find that the reporter used an ancient
version that was many years out of date. The newest versions of Nmap
have important algorithmic improvements, bug fixes,
performance-enhancing features such as local network ARP scanning, and
more. The first response to performance problems should be to compare
your version of Nmap (run nmap -V) with the latest
version available from https://nmap.org
.
Upgrade if necessary. If it is still not fast enough, try the other
techniques in this chapter.
Execute Concurrent Nmap Instances
Some people try to speed up Nmap by executing many copies in
parallel against one target each. For example, the Nessus scanner
used to do this by default. This is usually much less efficient and slower
than letting Nmap run against the whole network. Nmap has its own
parallelization
system that is customized to its needs, and Nmap is
able to speed up as it learns about network reliability when it scans
a large group. Further, there is substantial overhead in asking the
OS to fork 65,536 separate Nmap instances just to scan a class B.
Having dozens of copies of Nmap running in parallel is also a memory
drain since each instance loads its own copy of the data files such as
nmap-services
and
nmap-os-db
.
While launching single-host Nmap scans in parallel is a bad idea, overall speed can usually be improved by dividing the scan into several large groups and executing those concurrently. Don't go overboard though. Five or ten Nmap processes are fine, but launching 100 Nmap processes at once is not recommended. Launching too many concurrent Nmap processes leads to resource contention. Another sort of concurrency is to run Nmap from different hosts at once. You can have cron (or At on Windows) schedule local hosts on each of your networks to start scanning machines local to them at the same time, then mail the results to a central data server. Scanning your Australian network from the U.S. will be slower than scanning it from a local machine on that network. The difference will be even greater if the U.S. machine must traverse extra firewalls to reach the distant network.
Scan From a Favorable Network Location
Restrictive firewalls can turn a five-second scan into a multi-hour chore. The latency and packet loss associated with some Internet routes doesn't help either. If you can run Nmap from host(s) local to the target network, do so. Of course if the goal is to view the network as an external attacker would, or to test the firewall, external scanning is required. On the other hand, scanning and securing the internal network provides defense in depth which is critical against internal threats and those wily attackers who circumvent the firewall (see Chapter 10, Detecting and Subverting Firewalls and Intrusion Detection Systems).
When doing reverse DNS resolution, especially if you have a heavily burdened local nameserver, it can help to use a less busy nameserver or directly query the authoritative nameservers. This gain is usually slight and only worth doing for repeated or enormous scans. Of course, there are sometimes non-performance reasons for choosing nameservers.
Increase Available Bandwidth and CPU Time
You can occasionally improve Nmap scan times by increasing your available bandwidth or CPU power. This may be done either by installing a new data line or CPU, or by halting concurrently running applications which compete for these resources. For example, Nmap will run slower if you concurrently saturate your DSL line by downloading a pirate torrent of The Matrix Reloaded.
It is far more common that Nmap is constrained by its own
congestion control algorithms
than being CPU-bound or limited by the
available local bandwidth. These controls help prevent network
flooding and increase accuracy. Increasing CPU power and local
bandwidth won't help this sort of self-limiting by Nmap—timing
options must be adjusted instead. You can test whether Nmap is CPU
constrained by monitoring your CPU load with an application such as
top on Unix or the Task
Manager on Windows. If your CPU spends most of its time
idle, then upgrading won't help much. To test Nmap's bandwidth usage,
run it in verbose mode (-v
). Nmap will then report
the number of bytes sent and received and its execution time, as shown
in Example 6.1.
# nmap -v -n -p- sec.titan.net
Starting Nmap ( https://nmap.org )
[10 lines cut]
Nmap scan report for 192.168.0.8
Not shown: 65534 closed ports
PORT STATE SERVICE
22/tcp open ssh
MAC Address: 00:1A:6B:C1:33:37 (USI)
Nmap done: 1 IP address (1 host up) scanned in 2.20 seconds
Raw packets sent: 65536 (2.884MB) | Rcvd: 65536 (2.621MB)
Multiply the byte values by eight and divide by the execution time to get the average bandwidth usage in bits per second. In Example 6.1, Nmap received 2,621,000 bytes (Nmap considers 1,000,000 bytes to be a MB) in 2.20 seconds. So receive traffic was about 9.5 Mbps (send rate was 10.5 Mbps). Therefore the 100 Mbps ethernet link isn't likely constraining Nmap, and upgrading to gigabit ethernet won't help much.
Some consumer broadband devices and other equipment struggles to handle the rate of packets sent by Nmap, even though the small packet size (usually Nmap sends empty headers) keeps bandwidth low. In Example 6.1, “Bandwidth usage over local 100 Mbps ethernet network”, Nmap sent about 30,000 packets per second and received a similar number. Such high packet rates can cause problem with low-quality devices. In this case, we see that both send and receive packet counts were 65,536, which is the number of scanned ports (65,535) plus one for the initial ARP ping probe. Therefore Nmap did not encounter any packet drops requiring retransmission. This suggests again that the networking equipment was not a limiting factor—Nmap was probably CPU bound.