Merge pull request #26 from guanana/improve_nmap

Improve nmap

README.md

@@ -23,15 +23,15 @@ After installation, use the `netbox-scanner.conf` file as an example to create y
 ## Quick Start
 
 0. Clone the repo and install the dependencies as shown above.
-1. Move the `netbox-scanner.conf` file to your Netbox directory (`/opt/netbox`) and fill out the variables according to your setup. Don't forget to change the path to match where you put this repo under `[NMAP].path`.
+1. Use the `netbox-scanner.conf` file that you find in the project directory or create your own under (`/opt/netbox/netbox-scanner.conf` or `~/.netbox-scanner.conf`) and fill out the variables according to your setup.
-2. Go to the `samples` subdirectory of this repo and execute `./nmap-scan.sh` to get a first look at the behavior of this project.
+2. Fill the networks you want to scan (one per line) under `networks.txt` and then run `python netbox-scanner.py nmap` to get a first look at the behavior of this project.
 
 ## Basics
 netbox-scanner reads a user-defined source to discover IP addresses and descriptions, and insert them into NetBox.  To control what was previously inserted, netbox-scanner adds tags to each record, so it will know that that item can be handled.  In order to guarantee the integrity of manual inputs, records without such tags will not be updated or removed.
 
 It is important to note that if netbox-scanner cannot define the description for a given host, then it will insert the string defined in the `unknown` parameter.  Users can change those names at their own will.
 
-For NetBox access, this script uses [pynetbox](https://github.com/digitalocean/pynetbox) --worth saying that was tested under NetBox v2.6.7.
+For NetBox access, this script uses [pynetbox](https://github.com/digitalocean/pynetbox) --worth saying that was tested under NetBox v2.9.11.
 
 ### Garbage Collection
 If the user marked the `cleanup` option to `yes`, then netbox-scanner will run a garbage collector after the synchronization finishes.  Basically, it will get all IP addresses recorded in NetBox under the same tag.  Then, both lists will be compared: the one just retrieved from NetBox and the list that was synced.  Hosts in the first list that don't appear in the second list will be deleted.
@@ -40,9 +40,9 @@ If the user marked the `cleanup` option to `yes`, then netbox-scanner will run a
 ## Configuration
 Users can interact with netbox-scanner by command line and configuration file.  The latter is pretty simple and straight forward: the only parameter accepted is the module you want to use.
 
-The configuration file (`netbox-scanner.conf`) is where netbox-scanner looks for details such as authentication data and path to files.  This file can be stored on the user's home directory or on `/opt/netbox`, but if you choose the first option, it must be a hidden file --`.netbox-scanner.conf`.
+The configuration file (`netbox-scanner.conf`) is where netbox-scanner looks for details such as authentication data and path to files.  This file can be stored on the user's home directory, on `/opt/netbox` or under the project root directory, but if you choose the first option, it must be a hidden file --`.netbox-scanner.conf`.
 
-> Remember that netbox-scanner will always look for this file at home directory, then at `/opt/netbox`, in this order.  The first occurrence will be considered.
+> Remember that netbox-scanner will always look for this file at home directory, then at `/opt/netbox`, and finally in the project root dir, in this order.  The first occurrence will be considered.
 
 
 ## Modules
@@ -54,10 +54,8 @@ Since version 2.0, netbox-scanner is based on modules.  This way, this program i
 
 
 ## Nmap Module
-Performing the scans is beyond netbox-scanner features, so you must run [Nmap](https://nmap.org/) and save the output as an XML file using the `-oX` parameter.  Since this file can grow really fast, you can scan each network and save it as a single XML file.  You just have to assure that all files are under the same directory before running the script --see `samples/nmap.sh` for an example.
+Performing the scans is incorporated now under netbox-scanner features, so you don't need to run [Nmap](https://nmap.org/) yourself. 
-
+In this version the scan doesn't need root privileges since it only checks for hosts up.
-To properly setup this module, you must inform the path to the directory where the XML files reside, define a tag to insert to discovered hosts, and decide if clean up will take place.  Tested on Nmap v7.80.
-
 
 ## Prime Module
 This script accesses [Prime](https://www.cisco.com/c/en/us/products/cloud-systems-management/prime-infrastructure/index.html) through RESTful API and all routines are implemented here.  Users only have to point to Prime's API, which looks like `https://prime.domain/webacs/api/v4/`, inform valid credentials allowed to use the API, and fill the other variables, just like in Nmap.

nbs/__init__.py

@@ -1,12 +1,12 @@
 import configparser
 import logging
-
 import pynetbox.models.dcim
 import requests
 import docker
 import ipaddress
 from collections import deque
 
+import requests
 from pynetbox import api
 
 
@@ -53,7 +53,12 @@ class NetBoxScanner(object):
             return False
 
         if nbhost:
-            if (self.tag in nbhost.tags):
+            tag_update = False
+            for tag in nbhost.tags:
+                if (self.tag == tag.name):
+                    tag_update = True
+                    break
+            if tag_update:
                 if (host[1] != nbhost.description):
                     aux = nbhost.description
                     nbhost.description = host[1]
@@ -65,7 +70,7 @@ class NetBoxScanner(object):
                     logging.info(f'unchanged: {host[0]}/32 "{host[1]}"')
                     self.stats['unchanged'] += 1
             else:
-                logging.info(f'unchanged: {host[0]}/32 "{host[1]}"')
+                logging.info(f'no-tag(unchanged): {host[0]}/32 "{host[1]}"')
                 self.stats['unchanged'] += 1
         else:
             self.netbox.ipam.ip_addresses.create(

nbs/nmap.py

@@ -1,30 +1,32 @@
-import os
+import nmap3
-import xml.etree.ElementTree as ET
-
 
 class Nmap(object):
 
-    def __init__(self, path, unknown):
+    def __init__(self, unknown, networks):
         self.unknown = unknown
-        self.path = path
+        self.networks = networks
         self.hosts = list()
+        self.scan_results = {}
+
+    def scan(self):
+        nmap = nmap3.NmapHostDiscovery()  # instantiate nmap object
+        for item in self.networks:
+            temp_scan_result = nmap.nmap_no_portscan(item.replace('\n', ''), args="-R --system-dns")
+            self.scan_results = {**self.scan_results, **temp_scan_result}
+            self.scan_results.pop("stats")
+            self.scan_results.pop("runtime")
+        return self.scan_results
 
     def run(self):
-        for f in os.listdir(self.path):
+        self.scan()
-            if not f.endswith('.xml'):
+        for k,v in self.scan().items():
-                continue
+            try:
-            abspath = os.path.join(self.path, f)
+                self.hosts.append((
-            tree = ET.parse(abspath)
+                    k,
-            root = tree.getroot()
+                    v['hostname'][0]['name']
-
+                ))
-            for host in root.findall('host'):
+            except (IndexError, KeyError):
-                try:
+                self.hosts.append((
-                    self.hosts.append((
+                    k,
-                        host.find('address').attrib['addr'],
+                    self.unknown
-                        host.find('hostnames').find('hostname').attrib['name']
+                ))
-                    ))
-                except AttributeError:
-                    self.hosts.append((
-                        host.find('address').attrib['addr'],
-                        self.unknown
-                    ))

netbox-scanner.conf

@@ -8,6 +8,7 @@ tls_verify = no
 
 [NMAP]
 path     = ./
+networks = networks.txt
 unknown  = autodiscovered:netbox-scanner
 tag      = nmap
 cleanup  = no

netbox-scanner.py

@@ -26,12 +26,15 @@ if argument == 'prime':
 
 local_config = expanduser('~/.netbox-scanner.conf')
 global_config = '/opt/netbox/netbox-scanner.conf'
+dir_config = './netbox-scanner.conf'
 config = ConfigParser()
 
 if isfile(local_config):
     config.read(local_config)
 elif isfile(global_config):
     config.read(global_config)
+elif isfile(dir_config):
+    config.read(dir_config)
 else:
     raise FileNotFoundError('Configuration file was not found.')
 
@@ -80,9 +83,11 @@ logging.getLogger().addHandler(logging.StreamHandler())
 # useful if you have tls_verify set to no
 disable_warnings(InsecureRequestWarning)
 
+with open(nmap['networks'], 'r') as file:
+    networks = file.readlines()
 
 def cmd_nmap(s):  # nmap handler
-    h = Nmap(nmap['path'], nmap['unknown'])
+    h = Nmap(nmap['unknown'], networks)
     h.run()
     s.sync(h.hosts)
 

networks.txt

@@ -0,0 +1,5 @@
+192.168.2.0/24
+192.168.3.0/24
+192.168.4.0/24
+192.168.5.0/24
+192.168.15.0/24

requirements.txt

@@ -1,9 +1,10 @@
-certifi==2020.4.5.1
+certifi==2020.12.5
 chardet==3.0.4
-idna==2.9
+idna==2.10
-pynetbox
+pynetbox==5.1.0
-requests
+requests==2.25.0
-six
+six==1.15.0
-urllib3
+urllib3==1.26.2
+python3-nmap==1.4.9
 setuptools
-docker
+docker

tests/test_nmap.py

@@ -1,16 +1,17 @@
 import unittest
-from os import environ
+
 from nbs.nmap import Nmap
 
 
 class TestRequest(unittest.TestCase):
     def test_api(self):
-        path = environ.get('NMAP_PATH')
 
-        nmap = Nmap(path, 'test')
+        nmap = Nmap("test", ["127.0.0.1/32"])
         self.assertIsInstance(nmap, Nmap)
         nmap.run()
         self.assertIsInstance(nmap.hosts, list)
+        self.assertEqual(nmap.hosts[0][0], "127.0.0.1")
+        self.assertEqual(nmap.hosts[0][1], "localhost")
 
 
 if __name__ == '__main__':