Receiving XML exception from nmap.scan() results.

Receiving XML exception from nmap.scan() results. - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Receiving XML exception from nmap.scan() results. (/thread-23840.html)

Receiving XML exception from nmap.scan() results. - PythonNmap - Jan-20-2020

Hi All,

I'm making use of the python-nmap library.

However, running a basic scan against 10.0.0.17/255.255.255.0 or more precisely 10.0.0.17/24, yields the following exception on the returned XML results:

Error:Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/nmap/nmap.py", line 322, in analyse_nmap_xml_scan
    dom = ET.fromstring(self._nmap_last_output)
  File "/usr/lib64/python3.6/xml/etree/ElementTree.py", line 1314, in XML
    parser.feed(text)
  File "<string>", line None
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 4, column 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./get_address", line 94, in nmapScan
    raw = nm.scan(hosts=vlan+'/'+str(cidr), arguments=' -v -sn -n -oG - ')
  File "/usr/local/lib/python3.6/site-packages/nmap/nmap.py", line 266, in scan
    nmap_warn_keep_trace=nmap_warn_keep_trace
  File "/usr/local/lib/python3.6/site-packages/nmap/nmap.py", line 327, in analyse_nmap_xml_scan
    raise PortScannerError(self._nmap_last_output)

The command is:

raw = nm.scan(hosts=vlan+'/'+str(cidr), arguments=' -v -sn -n -oG - ')

which translates to:

nmap -v -sn -n -oG - 10.0.0.117/24

Raw output would be similar to this:

# nmap -v -sn -n -oG - 10.0.0.117/24
# Nmap 6.40 scan initiated Sun Jan 19 23:22:18 2020 as: nmap -v -sn -n -oG - 10.0.0.117/24
# Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Host: 10.0.0.0 ()       Status: Down
Host: 10.0.0.1 ()       Status: Up
Host: 10.0.0.2 ()       Status: Down
Host: 10.0.0.3 ()       Status: Down
.
.
.
Host: 10.0.0.252 ()     Status: Down
Host: 10.0.0.253 ()     Status: Down
Host: 10.0.0.254 ()     Status: Down
Host: 10.0.0.255 ()     Status: Down
# Nmap done at Sun Jan 19 23:22:22 2020 -- 256 IP addresses (7 hosts up) scanned in 3.62 seconds

When running the resulting XML through some parsing, I get this (snippet):

Error:[root@one01 ~]# cat text.txt | awk '{ gsub(/\\n/, "\n", $0); print; }'
<?xml version="1.0"?>
<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24 -->
# Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24
<nmaprun scanner="nmap" args="nmap -oX - -v -sn -n -oG - 10.0.0.117/24" start="1579495098" startstr="Sun Jan 19 23:38:18 2020" version="6.40" xmloutputversion="1.04">
<verbose level="1"/>
<debugging level="0"/>
# Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;)
<taskbegin task="Ping Scan" time="1579495099"/>
<taskend task="Ping Scan" time="1579495102" extrainfo="256 total hosts"/>
<host><status state="down" reason="no-response" reason_ttl="0"/>
<address addr="10.0.0.0" addrtype="ipv4"/>
Host: 10.0.0.0 ()\tStatus: Down
</host>
<host><status state="up" reason="echo-reply" reason_ttl="255"/>
<address addr="10.0.0.1" addrtype="ipv4"/>
<hostnames>
</hostnames>
Host: 10.0.0.1 ()\tStatus: Up
<times srtt="3292" rttvar="5000" to="100000"/>

The unaltered error:

Error:
ERROR:root:'<?xml version="1.0"?>\n<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>\n<!-- Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24 -->\n# Nmap 6.40 scan initiated Sun Jan 19 23:38:18 2020 as: nmap -oX - -v -sn -n -oG - 10.0.0.117/24\n<nmaprun scanner="nmap" args="nmap -oX - -v -sn -n -oG - 10.0.0.117/24" start="1579495098" startstr="Sun Jan 19 23:38:18 2020" version="6.40" xmloutputversion="1.04">\n<verbose level="1"/>\n<debugging level="0"/>\n# Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;)\n<taskbegin task="Ping Scan" time="1579495099"/>\n<taskend task="Ping Scan" time="1579495102" extrainfo="256 total hosts"/>\n<host><status state="down" reason="no-response" reason_ttl="0"/>\n<address addr="10.0.0.0" addrtype="ipv4"/>\nHost: 10.0.0.0 ()\tStatus: Down\n</host>\n<host><status state="up" reason="echo-reply" reason_ttl="255"/>\n<address addr="10.0.0.1" addrtype="ipv4"/>\n<hostnames>\n</hostnames>\nHost: 10.0.0.1 ()\tStatus: Up\n<times srtt="3292" rttvar="5000" to="100000"/>\n</host>\n<host><status state="down" reason="no-response" reason_ttl="0"/>\n<address addr="10.0.0.2" addrtype="ipv4"/>\nHost: 10.0.0.2 ()\tStatus: Down\n</host>\n

Perhaps the -- in the above is causing the XML to throw an exception? Or the #'s ?

Thx,
TK

RE: Receiving XML exception from nmap.scan() results. - buran - Jan-20-2020

I think it's the # - traceback is clear - line 4, column1.
And you can validate your snippet against xml validators and they also raise error on #.
Now - should it be there is another question.
According to their sample output - no, it shouldn't be there
https://nmap.org/book/output-formats-xml-output.html

Note that there is another one on line 8 - # Ports scanned: TCP(0;) UDP(0;) SCTP(0;) PROTOCOLS(0;)
Maybe ask nmap support why it's in their xml output?

RE: Receiving XML exception from nmap.scan() results. - PythonNmap - Jan-20-2020

Yeah, after I posted I used a validator. Confirmed the same message.

However, the # is expected. There's a few formats nmap can output in, including XML. Meaning, I don't need to use python-nmap's XML generator, using instead an XML reader on nmap results captured from a system command. The option is -oX instead of -oG to nmap. I'll need to research this.

Wouldn't this make it more of a python-nmap coding question then? The #'s invalidate a ton of examples.

Thanks Buran!

Another clue. Looking closely though, line 3, that's the comment on line 4 encoded in HTML comments.

What this tells me is that the python-nmap logic did work converting a comment to HTML format but left the original comment in place as well. Why leave it twice? This suggests the #'s should have been removed once converted to HTML comments. Perhaps I should reach out to the developers of python-nmap to see if there's a way around this?

Thx,
TK

RE: Receiving XML exception from nmap.scan() results. - buran - Jan-20-2020

Maybe you shouldn't supply -oG option in arguments, Nowhere in the examples and the docs they force -oG (i.e. Grepable output format).
Looking at the source code, they force -oX option.

RE: Receiving XML exception from nmap.scan() results. - PythonNmap - Jan-21-2020

Hm. Tried to use -oX . python-nmap tried to wrap XML into XML.

Now I just tried using without -oG and without -oX. Now it works.

Cheers,
TK