Nmap checks to see if the port is one of the ports to be excluded, as specified by the Exclude directive in nmap-service-probes. If it is, Nmap will not scan this port for reasons mentioned in the section called “nmap-service-probes File Format”.

If the port is TCP, Nmap starts by connecting to it. If the connection succeeds and the port had been in the open|filtered state, it is changed to open. This is rare (for TCP) since people trying to be so stealthy that they use a TCP scan type which produces open|filtered ports (such as FIN scan) generally know better than to blow all of their stealth by performing version detection.

Once the TCP connection is made, Nmap listens for roughly five seconds. Many common services, including most FTP, SSH, SMTP, Telnet, POP3, and IMAP servers, identify themselves in an initial welcome banner. Nmap refers to this as the “NULL probe” because Nmap just listens for responses without sending any probe data. If any data is received, Nmap compares it to about 3,000 NULL probe signatures in its nmap-service-probes file (described in the section called “nmap-service-probes File Format”). If the service is fully identified, we are done with that port! The regular expression in signatures can includes substring matches to pick version numbers out of the response. In some cases, Nmap gets a “soft match” on the service type, but no version info. In that case, Nmap continues but only sends probes that are known to recognize the soft-matched service type.

This point is where Nmap starts for UDP probes, and TCP connections continue here if the NULL probe described above fails or soft-matches. Since the reality is that most ports are used by the service they are registered to in nmap-services, every probe has a list of port numbers that are considered to be most effective. For example, the probe called GetRequest that recognizes web servers (among many other services) lists 80-85, 8000-8010, and 8080-8085 among its probable ports. Nmap sequentially (in the order they appear in the file) executes the probe(s) that match the port number being scanned.

Each probe includes a probe string (which can be arbitrary ASCII text or \xHH escaped binary), which is sent to the port. Responses that come back are compared to a list of signature regular expressions of the same type as discussed in the NULL probe description above. As with the NULL probe, these tests can either result in a full match (ends processing for the remote service), a soft match (limits future probes to those which match a certain service), or no match at all. The exact list of regular expressions that Nmap uses to test for a match depends on the probe fallback configuration. For instance, the data returned from the X11Probe is very unlikely to match any regular expressions crafted for the GetRequest probe. On the other hand, it is likely that results returned from a Probe such as RTSPRequest might match a regular expression crafted for GetRequest because the two protocols being tested for are closely related. So the RTSPRequest probe has a fallback to GetRequest matches. For a more comprehensive explanation, see the section called “Cheats and Fallbacks”.

If any response during version detection is ever received from a UDP port which was in the open|filtered state, that state is changed to open. This makes version detection an excellent complement to UDP scan, which is forced to label all scanned UDP ports as open|filtered when some common firewall rules are in effect. While combining UDP scanning with version detection can take many times as long as a plain UDP scan, it is an effective and useful technique. This method is described in the section called “Distinguishing Open from Filtered UDP Ports”.

In most cases, the NULL probe or the probable port probe(s) (there is usually only one) described above match the service. Since the NULL probe shares its connection with the probable port probe, this allows service detection to be done with only one brief connection in most cases. With UDP only one packet is usually required. But should the NULL probe and probable port probe(s) fail, Nmap goes through other existing probes sequentially. In the case of TCP, Nmap must make a new connection for each probe to avoid having previous probes corrupt the results. This worst-case scenario can take a bit of time, especially since Nmap must wait about five seconds for the results from each probe because of slow network connections and otherwise slowly responding services. Fortunately, Nmap utilizes several automatic techniques to speed up scans:

One of the probes tests whether the target port is running SSL. If so (and if OpenSSL is available), Nmap connects back via SSL and restarts the service scan to determine what is listening behind the encryption. A special directive allows different probable ports for normal and SSL tunneled connections. For example, Nmap should start against port 443 (HTTPS) with an SSL probe. But after SSL is detected and enabled, Nmap should try the GetRequest probe against port 443 because that port usually has a web server listening behind SSL encryption.

Another generic probe identifies RPC-based services. When these are found, the Nmap RPC grinder (discussed later) is initiated to brute force the RPC program number/name and supported version numbers. Similarly, an SMB post-processor for fingerprinting Windows services is available as part of the Chapter 9, Nmap Scripting Engine.

If at least one of the probes elicits some sort of response, yet Nmap is unable to recognize the service, the response content is printed to the user in the form of a fingerprint. If users know what services are actually listening, they are encouraged to submit the fingerprint to Nmap developers for integration into Nmap, as described in the section called “Submit Service Fingerprints”.