Nmap API

NSE scripts have access to several Nmap facilities for writing flexible and elegant scripts. The API provides target host details such as port states and version detection results. It also offers an interface to the Nsock library for efficient network I/O.

Information Passed to a Script

An effective Nmap scripting engine requires more than just a Lua interpreter. Users need easy access to the information Nmap has learned about the target hosts. This data is passed as arguments to the NSE action method. The arguments, host and port, are Lua tables which contain information on the target against which the script is executed. The following list describes each variable in the host and port tables.

host

This table is passed as a parameter to the rule and action functions. It contains information on the operating system run by the host (if the -O switch was supplied), the IP address and the host name of the scanned target.

host.os

The os entry in the host table is an array of strings. The strings (maximally 8) are the names of the operating systems the target is possibly running. Strings are only entered in this array if the target machine is a perfect match for one or more OS database entries. If Nmap was run without the -O option, then host.os is nil.

host.ip

Contains a string representation of the IP address of the target host. If the scan was run against a host name and the reverse DNS query returned more than one IP addresses then the same IP address is used as the one chosen for the scan.

host.name

Contains the reverse DNS entry of the scanned target host represented as a string. If the host has no reverse DNS entry, the value of the field is an empty string.

host.targetname

Contains the name of the host as specified on the command line. If the target given on the command line contains a netmask or is an IP address the value of the field is nil.

host.directly_connected

A Boolean value indicating whether or not the target host is directly connected (i.e. on the same network segment).

host.mac_addr

MAC address of the destination host (6-byte long binary string) or nil, if the host is not directly connected.

host.mac_addr_src

Our own MAC address, which was used to connect to the host (either our network card's, or (with --spoof-mac) the spoofed address).

host.interface

A string containing the interface name (dnet-style) through which packets to the host are sent.

host.bin_ip

The target host's IPv4 address as 4 byte long binary value.

host.bin_ip_src

Our host's (running Nmap) source IPv4 address as 4 byte long binary value.

port

The port table is passed to the Lua script in the same fashion as the host table. It contains information about the port against which the script is running. If the script is run according to a host rule, then no port table is passed to the script. Port states on the target can still be requested from Nmap using the nmap.get_port_state() call.

port.number

Contains the number of the currently scanned port.

port.protocol

Defines the protocol of the port. Valid values are tcp and udp.

port.service

Contains a string representation of the service running on port.number as detected by the Nmap service detection. If the port.version field is nil then Nmap has guessed the service based only on the port number. Otherwise this field is equal to port.version.name.

port.version

This entry is a table which contains information retrieved by the Nmap version scanning engine. Some of the values (like service name, service type confidence, RPC related values) may be retrieved by Nmap even if a version scan was not required. Values which were not retrieved default to nil. The meaning of each value is given in the following table:

Table 9.2. port.version values

Name	Description
`name`	Contains the service name Nmap will use for the port.
`name_confidence`	Evaluates how confident the version detection is about the accuracy of `name`, from 1 (least confident) to 10.
`product`, `version`, `extrainfo`, `hostname`, `ostype`, `devicetype`	These five variables are described in `<versioninfo>`.
`service_tunnel`	Contains the string `none` or `ssl` based on whether or not Nmap used SSL tunneling to detect the service.
`service_fp`	The service fingerprint, if any, is provided in this value. This is described in the section called “Community Contributions”.
`rpc_status`	Contains a string value of `good_prog` if we were able to determine the program number of an RPC service listening on the port, `unknown` if the port appears to be RPC but we couldn't determine the program number, `not_rpc` if the port doesn't appear be RPC, or `untested` if we haven't checked for RPC status.
`rpc_program`, `rpc_lowver`, `rpc_highver`	The detected RPC program number and the range of version numbers supported by that program. These will be `nil` if `rpc_status` is anything other than `good_prog`.

port.state

Contains information on the state of the port. Service scripts are only run against ports in the open or open|filtered states, so port.state generally contains one of those values. Other values might appear if the port table is a result of the get_port_state function. You can adjust the port state using the nmap.set_port_state() call. This is normally done when an open|filtered port is determined to be open.

Scripts also have access to some of Nmap’s functions and state variables that are exposed through functions in the nmap table.

nmap.debugging()

Returns the debugging level as a non-negative integer. The debugging level can be set with the -d option.

nmap.have_ssl()

Returns true if Nmap was compiled with SSL support, false otherwise. This can be used to avoid sending SSL probes when SSL is not available.

nmap.verbosity()

Returns the verbosity level as a non-negative integer. The verbosity level can be set with the -v option.

nmap.fetchfile(filename)

Allows access to Nmap's data files. fetchfile() searches for the specified file and returns a string containing it's path if it is found and readable (to the process). If the file is not found, not readable, or is a directory, nil is returned. The call

nmap.fetchfile("nmap-rpc")

will search for the data file nmap-rpc and, assuming it's found (which it should be), return a location like /usr/local/share/nmap/nmap-rpc.

nmap.timing_level()

Returns the timing level as a non-negative integer. Possible return values vary from 0 to 5, corresponding to the six built-in Nmap timing templates. The timing level can be set with the -T option.

Target Information Retrieval by a Script

Often the information passed to the script is not enough. Sometimes a script might want to correct target information or set it in the first place. The following API methods handle this.

nmap.get_port_state(host, port, protocol)

The get_port_state() call takes a host table, a port table and a protocol (tcp or udp) and returns a port table for the queried port. The host and port table are similar in structure to the ones passed to the rule and action functions. The host table should have an IP address field. The port table needs a port number and a protocol field. A call could look like this:

nmap.get_port_state({ip="127.0.0.1"}, {number="80", protocol="tcp"})

You can of course reuse the host and port tables passed to the port rule function. The purpose of this call is to be able to match scripts against more than one open port. For example if the target host has an open port 22 and a running identd server, then you can write a script which will only fire if both ports are open and there is an identification server on port 113. While it is possible to specify IP addresses different to the currently scanned target, the result will only be correct if the target is in the currently scanned group of hosts.

nmap.set_port_state(host, port, state)

The set_port_state() call takes a host table, a port table, and a port state (open or closed). Using this method the final port state, reflected in Nmap's results, can be changed for a target. This is useful when Nmap detects a port as open|filtered (i.e. unable to determine which), but the script successfully connects to that port. In this case the script can set the port state to open. Note that the port.state value, which was passed to the script's action function will not be changed by this call.

nmap.set_port_version(host, port, probestate)

NSE scripts are sometimes able to determine the service name and application version listening on a port. A whole script category (version) was designed for this purpose, as described in the section called “Version Detection Using NSE”. The set_port_version function is used to record version information when it is discovered.

This method takes a host and a port table as arguments. The third argument describes the state in which the script completed. It is a string which is one of: hardmatched, softmatched, nomatch, tcpwrapped, or incomplete. The hardmatched argument is almost always used, as it signifies a successful match. The other possible states are generally only used for standard version detection rather than the NSE enhancement.

The host and port arguments to this function should either be the tables passed to the action method or they should have the same structure. The version detection fields this function looks at are name, product, version, extrainfo, hostname, ostype, devicetype, and service_tunnel. All values in this table are optional. It is possible to pass a table in which all these values are set to nil or not to set the values at all.

Various Utility Functions for Raw Packet Support

NSE has support for sending raw ethernet frames and capturing packets. The following two functions may be handy in this context:

nmap.clock_ms(): Returns a number representing the current time as milliseconds since the start of the epoch (on most systems this is 01/01/1970).
nmap.get_interface_link("interface_name"): For the provided dnet-style interface_name, nmap.get_interface_link() returns what kind of link level hardware the interface belongs. Return values are: ethernet, loopback or p2p. If the provided interface_name is not one of those types, nil is returned.

Network I/O API

To allow for efficient and parallelizable network I/O, NSE provides an interface to Nsock, the Nmap socket library. The smart callback mechanism Nsock uses is fully transparent to NSE scripts. The main benefit of NSE's sockets is that they never block on I/O operations, allowing many scripts to be run in parallel. The I/O parallelism is fully transparent to authors of NSE scripts. In NSE you can either program as if you were using a single non blocking socket or you can program as if your connection is blocking. Seemingly blocking I/O calls still return once a specified timeout has been exceeded. Two flavors of Network I/O are supported:

Connect-style network I/O

This part of the network API should be suitable for most classical network uses: Users create a socket, connect it to a remote address, send and receive data and close the socket again. Everything up to the Transport layer (which is either TCP, UDP or SSL) is handled by the library. The following socket API methods are supported:

nmap.new_socket()

The new_socket() Nmap call returns an NSE socket object which is the recommended method for network I/O. It provides facilities to perform communication using the UDP, TCP and SSL protocol in a uniform manner.

status, error = socket_object:connect(hostid, port, [protocol])

The connect method of NSE socket objects will put the socket in a state ready for communication. It takes as arguments a host descriptor (either an IP address or a host name), a port number and optionally a protocol. The protocol must be one of tcp, udp or ssl. By default the connect call will attempt to open a TCP connection. On success the returned value of status is true. If the connection attempt has failed, the error value contains a description of the error condition stored as a string. Those strings are taken from the gai_strerror() C function. They are (with the errorcode in parentheses):

“Address family for hostname not supported” (EAI_ADDRFAMILY)
“Temporary failure in name resolution” (EAI_AGAIN)
“Bad value for ai_flags” (EAI_BADFLAGS)
“Non-recoverable failure in name resolution” (EAI_FAIL)
“ai_family not supported” (EAI_FAMILY)
“Memory allocation failure” (EAI_MEMORY)
“No address associated with hostname” (EAI_NODATA)
“Name or service not known” (EAI_NONAME)
“Servname not supported for ai_socktype” (EAI_SERVICE)
“ai_socktype not supported” (EAI_SOCKTYPE)
“System error” (EAI_SYSTEM)

In addition to these standard system error based messages are the following two NSE-specific errors:

“Sorry, you don't have OpenSSL.” occurs if ssl is passed as third argument, but Nmap was compiled without OpenSSL support.
“invalid connection method” occurs if the second parameter is not one of tcp, udp, ssl.

status, error = socket_object:send(data)

The send method sends the data contained in the data string through an open connection. On success the returned value of status is true. If the send operation has failed, the error value contains a description of the error condition stored as a string. The error strings are:

“Trying to send through a closed socket”—if there was no call to socket_object:connect before the send operation.
“TIMEOUT”—if the operation took longer than the specified timeout for the socket.
“ERROR”—if an error occurred inside the underlying Nsock library.
“CANCELLED”—if the operation was cancelled.
“KILL”—if for example the script scan is aborted due to a faulty script.
“EOF”—if an EOF was read—will probably not occur for a send operation.

status, value = socket_object:receive()

The receive method does a non-blocking receive operation on an open socket. On success the returned value of status is true and the received data is stored in value. If receiving data has failed, value contains a description of the error condition stored as a string. A failure occurs for example if receive is called on a closed socket. The receive call returns to the NSE script all the data currently stored in the receive buffer of the socket. Error conditions are the same as for the send operation.

status, value = socket_object:receive_lines(n)

Tries to receive at least <n> lines from an open connection. A line is a string delimited with “\n” characters. If it was not possible to receive at least <n> lines before the operation times out a TIMEOUT error occurs. On the other hand, if more than <n> lines were received, all are returned, not just <n>. Use stdnse.make_buffer to guarantee one line is returned per call. On success the returned value of <status> is true and the received data is stored in <value>. If the connection attempt has failed, <value> contains a description of the error condition stored as string. Error conditions are the same as for the send operation.

status, value = socket_object:receive_bytes(n)

Tries to receive at least <n> bytes from an open connection. On success the returned value of <status> is true and the received data is stored in <value>. If operation fails, <value> contains a description of the error condition stored as a string. Similarly to receive_lines() <n> is the minimum amount of characters we would like to receive. If more arrive, we get all of them. If fewer than <n> characters arrive before the operation times out, a TIMEOUT error occurs. Other error conditions are the same as for the send operation.

status, value = socket_object:receive_buf(func/"string", keeppattern)

The receive_buf method reads data from the network until it encounters the given delimiter string (or matches the function passed in). Only data which occurs before the delimiter is returned, and the delimiter is then erased. This function continues to read from the network until the delimiter is found or the function times out. If data is read beyond the delimiter, that data is saved in a buffer for the next call to receive_buf. This buffer is cleared on subsequent calls to other Network I/O API functions.

The receive_buf method takes two arguments. The first one is either a string or a function. Strings are passed to Lua's string.find function as the second (pattern) parameter, with the buffer data being searched. If the first receive_buf argument is a function, it is expected to take exactly one parameter (the buffer) and its return values must be in the same format as string.find (offsets to the start and the end of the delimiter inside the buffer, or nil if the delimiter is not found). The nselib match.lua module (see the section called “Buffered Network I/O Helper Functions”) provides functions for matching against regular expressions or byte counts. These functions are suitable as arguments to receive_buf.

The second argument to receive_buf is a Boolean value which indicates whether the delimiting pattern should be returned along with the received data or discarded. The delimiter is included if true is passed as the keeppattern argument.

The return values of receive_buf are the same as the other receive* functions. A (status, val) tuple is returned. The status is true if the request was successful. The val variable contains the returned data, or an error message if the call failed.

Possible error messages include those described previously for the other receive* functions as well as the following:

“Error inside splitting-function”—if the first argument was a function which caused an error while being called.
“Error in string.find (nsockobj:receive_buf)!”—if a string was provided as the first argument, and string.find() yielded an error while being called.
“Expected either a function or a string!”—if the first argument was neither a function nor a string.
“Delimiter has negative size!”—if the returned start offset is greater than the end offset.

status, err = socket_object:close()

Closes an open connection. On success the returned value of status is true. If the connection attempt has failed, value contains a description of the error condition stored as a string. Currently the only error message is: “Trying to close a closed socket”, which is issued if the socket has already been closed. Sockets are subject to garbage collection. Should you forget to close a socket, it will get closed before it gets deleted (on the next occasion Lua's garbage collector is run). However since garbage collection cycles are difficult to predict, it is considered good practice to close opened sockets.

status,localip,localport,remoteip,remoteport=socket_object:get_info()

This function returns information about the socket object. It returns 5 values. If an error occurred, the first value is nil and the second value describes the error condition. Otherwise the first value describes the success of the operation and the remaining 4 values describe both endpoints of the TCP connection. If you put the call in a try() statement the status value is consumed. The call can be used for example if you want to query an authentication server.

socket_object:set_timeout(t)

Sets the time, in milliseconds, after which input and output operations on a socket should time out and return. The default value is 30,000 (30 seconds). The lowest allowed value is 10 ms, since this is the granularity of NSE network I/O.

Raw packet network I/O

For those cases where the connection oriented approach is too inflexible, NSE provides script developers with a more powerful option: raw packet network I/O. The greater flexibility comes, however, at the cost of a slightly more complex API. Receiving raw packets is accomplished via a wrapper around Libpcap inside the Nsock library. In order to keep the capturing efficient it works in a three tiered approach: Opening a device for capturing, registering listeners to it and receiving packets. With each call to pcap_open() you have to provide a callback function, which receives the packet (along with it's layer 2 and 3 headers) and is used to compute a so-called packet hash. Each call to pcap_register() takes a binary string as argument. For every packet captured the computed hash is matched against all registered strings. Those scripts for which the compare yields true are then provided with the packet as a return value to pcap_receive(). The more general the packet hash computing function is kept, the more scripts may receive the packet and proceed with their execution. To use the packet capturing inside your script you have to create a socket with nmap.newsocket() and later close the socket with socket_object:close()—just like with the connection-based network I/O. A more detailed description of the functions for packet capturing follows:

socket_object:pcap_open(device, snaplen, promisc, test_function, bpf)

The pcap_open() call opens the socket for packet capturing. The parameters are:

device—the dnet-style interface name of the device you want to capture from
snaplen—defines the length of each packet you want to capture (similar to the -s option to tcpdump)
promisc—should be set to 1 if the interface should activate promiscuous mode, and zero otherwise
test_function—callback function used to compute the packet hash
bpf—a string describing a Berkeley packet filter expression (like those provided to tcpdump)

socket_object:pcap_register(packet-hash)

Starts the listening for incoming packages. The provided packet-hash is a binary string which has to match the hash returned by the test_function parameter provided to pcap_open(). If you want to receive all packets, just provide the empty string (""). There has to be a call to pcap_register() before a call to pcap_receive().

status, packet_len, l2_data, l3_data = socket_object:pcap_receive()

Receives a captured packet. If successful, the return values are:

status—a Boolean with the value true
packet_len—the length of the captured packet which was received (this may be smaller than the actual packet length since packets are truncated when the Libpcap snaplen parameter is smaller than the total packet length)
l2_data—data from the second OSI layer (e.g. ethernet headers)
l3_data—data from the third OSI layer (e.g. IPv4 headers)

Should an error or timeout occur, while waiting for a packet the return values are: nil,error_message,nil,nil, where error_message describes the occurred error.

socket_object:pcap_close()

Closes the pcap device.

Receiving raw packets is a great feature, but it is also only the half job. Now for sending raw packets: To accomplish this NSE has access to a wrapper around the dnet library. Currently NSE has the ability to send raw ethernet frames via the following API:

dnet_object=nmap.new_dnet(): Creates and returns a new dnet_object, which can be used to send raw packets.
dnet_object:ethernet_open(interface_name): Opens the interface defined by the provided <interface_name> for sending ethernet frames through it. An error (“device is not valid ethernet interface”) is thrown in case the provided argument is not valid.
dnet_object:ethernet_send(packet): Sends the provided data as ethernet frame across the previously opened interface. Note that you have to provide the packet including IP header and ethernet header. If there was no previous valid call to ethernet_open() an error is thrown (“dnet is not valid opened ethernet interface”).
dnet_object:ethernet_close(): Closes the interface. The only error which may be thrown is the same as for the ethernet_send() operation.

Thread Mutexes

Each thread made for a script (e.g. anonFTP.nse) will yield to other scripts whenever it makes a call on network objects (sending/receiving data). Some scripts need finer control over threads' execution. An example is the whois.nse script which queries whois servers for each target. Because many concurrent queries often result in getting one's IP banned for abuse and a query may return additional information for targets other threads are running against, it is useful to have other threads pause while one thread is conducting a query.

To solve this problem, there is an nmap function, mutex, that provides a mutex usable by scripts. The mutex allows for only one thread to be working on an object. Threads waiting to work on this object are put in the waiting queue until they can get a "lock" on the mutex. A solution for the whois.nse problem above is to have each thread block on a mutex for the section called “id Field”, thus ensuring only one thread is working so its results can be shared with other scripts which may not need to run and so queries to the whois servers are staggered.

mutex = nmap.mutex(object)

Returns a function that works on a mutex for the object passed. This object can be any Lua data type except nil, booleans, and numbers. The returned function allows you to lock, try to lock, and release the mutex. Its first and only parameter must be one of the following:

"lock"—Make a blocking lock on the mutex. If the mutex is busy (another thread has a lock on it), then the thread will yield and wait. The function returns with the mutex locked.
"trylock"—Makes a non-blocking lock on the mutex. If the mutex is busy then it immediately returns with a return value of false. Otherwise the mutex locks the mutex and returns true.
"done"—Releases the mutex and allows another thread to lock it. If the thread does not have a lock on the mutex, an error will be raised.
"running"—Returns the thread locked on the mutex or nil if the mutex is not locked. This should only be used for debugging as it interferes with finished threads from being collected.

Example 9.2. Mutex manipulation

id = "My Script's Unique ID";

local mutex = nmap.mutex(id);
function action(host, port)
  mutex "lock";
  -- do stuff
  mutex "done";
  return script_output;
end

Exception Handling

NSE provides an exception handling mechanism not present in the plain Lua language. The exception handling is tailored specifically for network I/O operations. The mechanism follows a functional programming paradigm rather than an object oriented programming paradigm. To create an exception handler the nmap.new_try() API method is used. This method returns a function, which takes a function as an argument. If the function passed as an argument raises an exception, then the script execution is aborted and no output is produced. Optionally you can pass a function to the new_try() method which will be called if an exception is caught. In this function you can perform required clean up operations.

Example 9.3 shows cleanup exception handling at work. A new function named catch is defined to simply close the newly created socket in case of an error. It is then used to protect connection and communication attempts on that socket. If no catch function is specified, execution of the script aborts without further ado—open sockets will remain open. If the verbosity level is at least one or if the scan is performed in debugging mode a description of the uncaught error condition is printed on standard output. Note that it is currently not easily possible to group several statements in one try block. It is also important to remember that if the socket is not closed it will occupy memory until the next run of Lua's garbage collector.

Example 9.3. Exception handling example

local result, socket, try, catch

result = ""
socket = nmap.new_socket()
catch = function() 
socket:close() 
end
try = nmap.new_try(catch)

try(socket:connect(host.ip, port.number))
result = try(socket:receive_lines(1));
try(socket:send(result))

Writing a function which is treated properly by the try/catch mechanism is straightforward. The function should return multiple values. The first value should be a Boolean which is true upon successful completion of the function and false otherwise. If the function completed successfully the try construct consumes the indicator value and returns the remaining values. If the function failed then the second returned value must be a string describing the error condition. Note that if the value is not nil it is treated as true so you can return your value in the normal case and return nil, <error description> if an error occurs.

The Registry

The registry is a normal Lua table. What is special about it is that it is visible by all scripts and it retains its state between script executions. Nmap does not scan every host specified on the command line at the same time, it puts them in smaller groups and these groups are scanned in parallel. The registry is rebuilt for every group, so information stored there is only deleted after NSE finishes processing the current target group. This implies of course that the registry is transient—it is not stored between Nmap executions. Every script can read the registry and write to it. If a script is running after another script, it can read some information in the registry which was left by the first script. This feature is particularly powerful in combination with the run level concept. A script with a higher run level can rely on entries left behind for it by scripts with lower run levels. Remember however that the registry can be written by all scripts equally, so choose the keys for your entries wisely. The registry is stored in nmap.registry. The behavior of the registry allows caching of already calculated data. The cache can be seen by all scripts until the registry is rebuilt with the next target group.