Last modified: Tuesday, 30-January-2001 04:13:05 PDT


Network Reconnaissance Techniques
 (Latest version: https://nmap.org/presentations/CanSecWest01/OSDEM_Presentation_Details.html)
  1. Introduction

  2. Finding machines that are up on the network Before you can do much else, you need to identify the machines on the target network. There are many ways to do this:
    • Obvious answer: send an ICMP echo-request (ping) packet to each IP address and wait for a reply to determine which hosts are up.
    • But many hosts filter out ping requests or replies! Example: 
      amy~> ping microsoft.com
PING microsoft.com (207.46.230.219) from 208.184.74.98 : 56(84) bytes of data.

--- microsoft.com ping statistics ---
8 packets transmitted, 0 packets received, 100% packet loss
      The first time I pinged Microsoft like this, I thought, well, it is Windows. The box is probably crashed and not responding. But they are actually filtering the pings as we will soon see.
    • Solution: "TCP" ping. By default, Nmap sends a TCP ACK (acknowledgement) packet to port 80 in parallel with an ICMP ping request. If a RST packet (or a ping reply) comes back, we know the host exists. And just in case anyone isn't very familiar with TCP, let me give a really brief description of what I mean by "ACK" packet and "RST" packet. The TCP protocl has 6 bits, called flags, you can set set in the header of each packet. One of them is the SYN flag, which is used to request a connection, FIN is used to finish a connection, ACK is used to acknowledge data, and RST is often sent by machines which receive packets they don't expect. For a much more complete description, download RFC 793 or buy the book "TCP/IP Illustrated Volume 1" by W. Richard Stevens. So when I say something like "ACK" packet, I generally mean an TCP header with the ACK bit set and no data. Of course the TCP header is carried by an IP packet.
    • In some cases you may want to probe machines with a TCP SYN packet instead of an ACK. This is done with -PS. We will discuss one reason for doing this later.
    • -P0 tells Nmap to scan each machine without even checking if it is up first. This can be very slow (if scanning thousands of ports), but is the ultimate technique for paranoid (experienced) security admins.

  3. Determining the ports that are open . Now that we have determined using the techniques above what machines are available on the network, our next step is to determine what services they are offering to the world.
    • What ports are, why they are important. As I am sure all of you know, most servers and even some client programs listen on TCP or UDP ports to enable communication with other systems.
    • The golden rule of port management is to close all of them you do not need.Close all ports you don't need! Even if there aren't any known vulnerabilities you should do so, for the following reasons::
      • attacks are often distributed underground for a long while before becoming publicly available. They can sometimes float around for years before being discovered due to a botched exploit attempt or when someone leaves it around somewhere and it is discovered..
      • attackers make databases they can refer to when a new hole is discovered and publicized. Typical script kiddie behavior is to scan hundres of thousands of machines and make a list of the operating systems, open ports, and service version numbers. Then when someone posts, say, a remote root hole in Bind 8.2, our little cracker could grep his list for 'TCP/53' and 'Linux' and whatever OS his exploit is for. And in seconds he has a list of exploitable machines. This can lead to systems being compromised well before the admin learns about the bug and fixes it.
      • Even if there are no holes in (say) your socks daemon, people scanning for port 1080 will see your machine in their logs and may come back to check if your socks server is open. This is undesirable attention. Also be sure you close these ports (and apply all your other security measures) before you connect the machine to the Internet. Don't assume that your machine will be safe for a couple hours while you secure it. I am a member of a research group called the Honeynet project (http://project.honeynet.org). We set up fake networks, often using default installations of various operating systems and we study intruder techniques and methods when they are compromised. We don't lure anyone to the networks or otherwise advertise them. We just stick them out there and I am always amazed at how quickly they are discovered and compromised.
    • Open TCP ports can be determined by a SYN scan. This is the preferred general-purpose TCP scan type. Also known as half-open scanning. Give Nmap the -sS argument to perform this kind of scan. The idea is that we send a SYN packet to the target host & port. If the port is open, the target host will send back a SYN|ACK. If closed, the target returns a RST. Even if we get a SYN|ACK (meaning the port is open), we RST the partial connection rather than completing it. That is why it is called half-open scanning.
    • Don't forget UDP scanning! (Nmap option: -sU) A lot of people forget this whole protocol. They close all there TCP ports and think they are secure, but do not realize they have SunRPC, syslogd, lpd, and mountd all waiting to be exploited via UDP.
    • Other scan types: FIN, XMAS, and NULL scans (-sF, -sX, -sN). FIN scan, as you can imagine, uses TCP packets with just the FIN flag set. NULL uses no flags at all, and XMAS scan uses every flag except SYN, ACK and RST.. More details on the mechanics of these scans is available in the Nmap manpage.
    • Advanced technique: Roll your own SCAN type! The 3 scans above are almost the same except they use different flags ( FIN vs FIN|PSH|URG vs 0 ). A little known fact is that any combination of FIN|PSH|URG will work for this type of scan. And some IDS vendors haven't figured this out yet since they took the lazy route and generated product signatures by studying the Nmap man documentation rather than RFC 793.. To roll your own scan, go to scan_engine.c in the latest Beta of Nmap and fine the line:
      else if (scantype == NULL_SCAN) scanflags = 0;
      Then change the 0 to something like 'TH_URG|TH_PSH' and now you have a very special version of -sN
    • Advanced scan type: ACK scan (-sA) for probing firewalls/filtering systems.The idea here is pretty simple. In response to an unexpected ACK packet, a compliant TCP stack will send back a RST. But if we get nothing back on some ports, or we get an ICMP administratively prohibited host unreachable, we know a filter is blocking our packet. Now SYN scan tells you this too, with the added bonus of determining whether the port is open at the same time. But an ACK is more likely to get through to the destination machine since it is difficult to block with stateless filters like ipchains, router ACLs, etc). So doing an SYN scan followed by an ACK scan can tell you a lot about what type of firewall is being used and how it is configured. (FIXME: Demo -- hping Microsoft with ACK packet to port 80 and SYN packet to port 80)
    • Advanced scan type: IP Protocol scan -sP. Nmap usually focuses on TCP, UDP, and ICMP, but there is a whole World of other protocols available for advanced attacks and information gathering. The Protocol Scan cycles through the 8-bit protocol field sending raw IP headers without any data. An ICMP Protocol Unreachable error means the target does not accept packets for the given protocol.

      For example, here is a SYN scan of a high-end CISCO router: 

      amy~# nmap -sS 64.56.192.19 
(Explain syntax then note: scanning single machine.  
To scan the class C subnet containing this IP you would add /24 to the 
end to scan the 24-bit subnet.  Or you could use 64.56.192.0-255.
Starting nmap V. 2.54BETA19 ( insecure.org/nmap/ )
Interesting ports on dcr01-g6-0.sntc05.exodus.net (64.56.192.19):
(The 1537 ports scanned but not shown below are in state: closed) 
As a side note, many people ignore this line (above),
but it is actually quite important [explain it].
Port       State       Service
514/tcp    open        shell                   

Nmap run completed -- 1 IP address (1 host up) scanned in 7 seconds
      As you can see, this doesn't leave a whole lot of room for attack because there is only 1 open TCP port. In a real audit, UDP would be scanned too.
      Next we try an IP Proto scan: 
      amy~#nmap -sO 64.56.192.19

Starting nmap V. 2.54BETA19 ( insecure.org/nmap/ )
Interesting protocols on dcr01-g6-0.sntc05.exodus.net (64.56.192.19):
(The 238 protocols scanned but not shown below are in state: closed)
Protocol   State       Name
1          open        icmp                    
4          open        ip                      
6          open        tcp                     
8          open        egp                     
9          open        igp                     
17         open        udp                     
47         open        gre                     
53         open        swipe                   
54         open        narp                    
55         open        mobile                  
77         open        sun-nd                  
80         open        iso-ip                  
88         open        eigrp                   
89         open        ospfigp                 
94         open        ipip                    
103        open        pim                     

Nmap run completed -- 1 IP address (1 host up) scanned in 149 seconds
      Looking over this list we see a bunch of potential opportunities, such as [ go through list and note interesting ones ]. These protocols are not hammered on by attackers nearly as frequently as TCP, UDP, and ICMP. So it may be easier to find and exploit bugs and design flaws in them.

      IP Protocol scanning support was sent in by Gerhard Rieger (rieger@iue.tuwien.ac.at) last year. And since this is an Open Source and Free Software development conference, I will take an aside here and mention that I consider this to be a great example of where Open Source really shines. Even though this may seem like an obvious feature now, it wasn't on my radar last year because I hadn't thought of it and nobody had asked for it. Then out of the blue one day, Gerhard sent his complete patch to nmap-hackers. I am very pleased to report that this sort of grass roots innovation is not an isolated occurance. Literally hundreds of people have helped out in important ways. For example, just this last Thursday someone named Rob Braun sent me MacOS X (um, I think they call it Darwin now) portability patches. I integrated those in so that the next version of Nmap will support Darwin right out of the box. Closed source competitors like the ISS Security Scanner and NAI Cybercop completely miss out on this opportunity.

  4. Determining network architecture
    • Frequently vulnerabilities are specific to certain operating systems or OS release versions. Nmap (with -O) can usually determine the OS in use via a technique known as TCP/IP fingerprinting. The idea is to send various valid and invalid IP packets to the remote host and study the characteristics of the response very closely. A paper I wrote which describes these techniques is available at https://nmap.org/nmap-fingerprinting-article.html.
    • Determining network layout and routing. At some point Nmap will some advanced traceroute-like functionality. But for now I recommend traceroute or, for more flexibility, hping2. The advantage of hping2 if that you are not restricted to using a ping or high-port UDP packet. You can craft your own packet which is more likely to get through. Example: 
      hping2 --traceroute -t 1 -2 --baseport 53 -keep -V -p 5023 gw.target.com
      This means do a traceroute, starting with ttl=1 using UDP packets with a source port of 53 (dns) and a desination port of 5023 against gw.target.com. -V just turns on verbosity.
    • Determining application protocol (eg smtp vs named), name (eg qmail vs sendmail) and version number (bind 8.2.2 vs 8.2.3) listening on the open ports is often the next step once you have created a network map. Currently Nmap does not do this because I am still researching the best way to approach the problem. But if you are impatient you can download Nmap+V from ftp.saurik.com. This patch to Nmap was written by Jay Freeman ( Saurik ).
    • And the step after determining application versions is often to pinpoint the ones which are vulnerable to attack. Nmap probably won't be doing that any time soon, but there ar several good application-level vulnerability scanners out there. One I recommend is Nessus ( http://www.nessus.org) which uses Nmap for some low-level reconnaissance work and then uses that information to help search for hundreds of application-level security vulnerabilities. Renaud Deraison, the author of Nessus, is speaking in this room right after me. So I recommend you stick around. I certainly will be.

  5. Locating Firewall Misconfigurations
    • Vary the scan source port to look for common holes
      • Lazy admins sometimes unwittingly compromise security when they open holes as quick-hacks to make various services work.
      • Source port 53/udp is sometimes allowed in as a quick (INSECURE) hack to allow DNS replies (the ZoneAlarm personal "firewall" V. 2.0.26 had this problem).
      • Source port 20/tcp for FTP-DATA (active FTP) connections
      • Port 67/udp (BOOTP/DHCP) -- ZoneAlarm 2.1.10 is vulnerable.
      • Try both UDP & TCP for these ports.
      • The Nmap -g option allows for changing the source port of a scan.
    • Exploit identd to determine username running services (-I option). (Describe the problem; Open shell and do "nmap -I localhost"; first of all, note that my box is not following the golden rule of closing all the ports you don't need. Those are actually only opened to localhost for demos. Also note that this gives attackers an easy way to prioritize their attacks. They will go after the root services first. Additionally, it gives them a chance to spot common misconfigurations, such as httpd servers running as root.)
    • Filtering out SunRPC portmapper alone does not prevent exploits of RPC services. See docs for Nmap "Direct" RPC scan (-sR). We'll give an example of this soon.
    • Bouncing through the firewall
      • Socks proxy
      • FTP Server (active or passive attacks)
      • Web proxy
      • Source routed packets
      • Fragmentation holes Nmap has an option that exploits one particular fragmentation problem, but there are many more that various hosts are vulnerable to. Search the Bugtraq archives securityfocus.com.
    • Vary source addresses using IP.ID scanning (not yet implemented in Nmap). Primitive IP.ID scanners are available on the Internet (for example: Idlescan at http://www.hackers-pt.org/ptstuff ). This is truly a neat scan technique as it allows you to scan a system without sending a single packet to that box from your real source address. It also allows you to better understand filters by noting how packet filters may changed depending on the spoofed source address you are using. I am hoping to add this scan type relatively soon. This clever technique was invented by Antirez, who also wrote the hping2 tool we saw a little while ago.

  6. Optimizing your scan for speed . While a lot of work has been put into optimizing Nmap for performance, scanning huge networks can still take a long time. Here are some tips for speeding those scans up.
    • Frequently you may want to scan a large number of addresses for a single service (for example, to find all of the Bind servers in your organization after the last weeks exploit). Nmap has a little-known shortcut which can make this significantly faster. For example, here is a way to scan 16 million 10.* internal addresses for (tcp) named: 
      nmap -sS -PS53 -p53 10.0.0.8/8
      Recall from earlier in this presentation that -PS means "spew SYN packet probes to the given port of each target IP and watch for replies". But if a SYN packet has already been sent to the targeted port of a machine, doing the actuaql "SYN scan" is redundant, so Nmap just interprets the results of the initial SYN probe to deterine whether the port is open or not This shortcut can make a scan go two or three times faster than it would otherwise. All three options above must be given, and the port numbers must match up.
    • If you are scanning a very sparse IP space (like the example above), give Nmap a few available IPs to start with so Nmap can guage the speed of your network. Otherwise Nmap will go very slowly until it starts receiving packets back from an available machine. Nmap has to do this because, for all it knows, you could be on some 300bps, high-latency, low-reliability wireless link. So in the example above you might really want to do "nmap -sS -PS53 -p53 10.2.3.0/24 10.0.0.8/8"" (assuming 10.2.3.* contains some machines).
    • Nmap has a large number of timing options which allow you can use to give Nmap a high level timing policy (eg -T Aggressive or to muck with the low level timing parameters (eg --max_rtt_timout, --host_timeout, etc.). These can make a dramatic difference if you scan thousands of hosts and learn to use them appropriately. Usually they are most important if the hosts being scanned are behind a firewall. Otherwise Nmap usually does a good job at determining the optimum scan speed.

  7. Optimizing your scan for stealth It is occasionally desirable to perform your network reconnaissance without alerting the target network admins that you are doing so.
    • Avoid connect() scans (which are the default). These often leave a page of errors in syslog from services which got confused when Nmap connected to them and then immediately disconnected. As long as you are root, SYN scan is the way to go.
    • The decoy option (-D) uses source address spoofing to forge scans against the target "from" the machines given as decoys. So even if the target network has special software to log scans, they will see scans from dozens of addresses and are unlikely to be able to fish out the true scan origin from all of the decoys. Now this feature caused a bit of a stir when I first released it. For example, the Naval Surface Warfare Center put out an alarming report shortly afterward saying that they were seeing "coordinated, multi-national attacks" where it appeared that computers all over the world wore working together the Navy network. In reality, it was probably just some 15-year-old using Nmap -D. This caused some people to ask me why I even add such an option, which they considered to have no legitimate purpose. Well, first of all, there can be legitimate purposes to use decoys. And more importantly, I was noticing several disturbing trends:
      • People taking SYN scan logs at face value without realizing or caring that they are trivially forgeable.
      • Even worse, authors were releasing scan detection tools which could take offensive action by automatically running attack commands when a scan is detecting. I wrote some of these people saying don't you realize that someone could forge a scan and cause your program to winnuke or pingflood an innocent party? And I would get response like "we don't see forged scans very often". I think that answer is a disaster waiting to happen.
      • Progress seemed pretty slow on solutions that truly do help prevent source address spoofing such as egress filtering by ISPs and security protocols like IPSec which can provide cryptographic assurances that the source address really is the machine specified in the packet header.

    • Example scan To make this more concrete, here is an example TCP scan (hostname sanitized):
      Show via xterm! First give quick description of what the scan is. Also note the "filtered" ports and talk about people who think they have Backorifice when it is really filtered. 
      amy~#nmap -p- -sR -I -O www.secret.com
Starting nmap V. 2.52 by ( insecure.org/nmap/ )

Interesting ports on foo.bar.com (42.43.44.45):
(The 65514 ports scanned but not shown below are in state: closed)
Port       State       Service (RPC)           Owner
21/tcp     open        ftp                     root
22/tcp     open        ssh                     root
23/tcp     open        telnet                  root
25/tcp     open        smtp                    mail
80/tcp     open        http                    http
110/tcp    open        pop-3                   root
111/tcp    filtered    sunrpc                  
113/tcp    open        auth                    root
220/tcp    filtered    imap3                   
443/tcp    open        https                   http
512/tcp    filtered    exec                    
513/tcp    filtered    login                   
514/tcp    filtered    shell                   
515/tcp    filtered    printer                 
516/tcp    filtered    videotex                
517/tcp    filtered    talk                    
518/tcp    filtered    ntalk                   
635/tcp    filtered    unknown                 
939/tcp    open        (status V1)             root
2049/tcp   filtered    nfs                     
3600/tcp   open        unknown                 root

TCP Sequence Prediction: Class=random positive increments
                         Difficulty=4797787 (Good luck!)

Sequence numbers: 702A7726 70243971 7059255D 70F3C86B 710DC518 704EBFEE
Remote operating system guess: Linux 2.1.122 - 2.2.14

Nmap run completed -- 1 IP address (1 host up) scanned in 467 seconds
      Note that this is the traditional command line version. Several X Window versions are also available, and one of them is included with the base Nmap distribution. In addition, web front ends are available so that you can control it from a remote browser.
      What could be done Better?
      • Default-deny filtering rules
      • Eliminate the insecure applications (telnet, pop3, possibly FTP)
      • Better RPC filtering
      • Turn off identd or fix username disclosure problem.

    • Final Notes and Warnings
      • If you wish to contribute to Nmap development, you can join the dev list by sending a blank email to nmap-dev-subscribe@insecure.org. This is a reasonably low-volume list where we discuss Nmap future direction and development issues.
      • For announcements of new Nmap versions, nmap-related projects, and occasional, low-volume moderated discussion of technical, social, and political developments related to scanning, you can join the nmap-hackers list by sending a blank email to nmap-hackers-subscribe@insecure.org.
      • Nmap (like computer security in general) is frequently updated. The most current information is always available at https://insecure.org/nmap.
      • Read the manual thoroughly before using Nmap.
      • Nmap does not intentionally try to crash machines, but there are many poorly written applications, TCP/IP stacks, Intrusion detection systems, and even operating systems out there. Some have been known to occasionally crash during an Nmap scan. Be prepared for this possibility. But also remember that people are likely to Nmap scan your servers anyway, so it might be best to try it yourself whether than wonder what is causing mysterious crashes.
      • Be wary of scanning other people's networks.