topic Re: How to write regex to extract Port numbers and status from nmap output? in Splunk Search

How to write regex to extract Port numbers and status from nmap output?

sina_shafaei — Mon, 28 Sep 2020 17:26:52 GMT

this took at least three days of my life but I still couldn't handle it, please, I need help

I have a data file includes output of nmap traces (couple of traces on different times on each IP) in our subnet like this:

host starttime="1402600202" endtime="1402601288" status state="up" reason="echo-reply" address addr="128.150.254.114" addrtype="ipv4" hostnames hostname name="sunvirt01.srv.be" type="PTR" hostnames ports extraports state="filtered" count="985" extrareasons reason="no-responses" count="985" extraports: port protocol="tcp" portid="20" state state="closed" reason="reset" reason_ttl="63" service name="ftp-data" method="table" conf="3" port port protocol="tcp" portid="21" state state="open" reason="syn-ack" reason_ttl="63"/ service name="ftp" product="ProFTPD" method="probed" conf="10" cpe cpe:/a:proftpd:proftpd cpe service port port protocol="tcp" portid="22" port protocol="tcp" portid="222" state state="open" reason="syn-ack" reason_ttl="63" service name="tcpwrapped" method="probed" ...
(os)portused state="open" proto="tcp" portid="21"/ portused state="closed" proto="tcp" portid="20"/ osmatch name="Linux 3.0" accuracy="99" line="47080" (...)

what I need is a table to show the IP address and "open" Ports on that machine beside the service(s) are using that open port, something like this(e.g):

------ -----IP ----- ------- Ports ----- ----- Service

128.150.254.114 -------21------------------ ftp

(if there are more open ports o one host they have to be listed below "Ports" by their services)

I'm using this regex: (?i) portid="(?P[^"]+) to extract Port numbers but unfortunately in some cases I face with values like 21/ or duplicate values like this:

21 --- ftp

80 --- http

1718

or like this one:

20 ---

443 --- http

20/

21 ---- ftp

( I mean I need the duplicate values to be removed (or not detected) or values with "/" at the end which I have no idea about them!) can you give me some hint or the right regex I have to used to extract only open ports (once without considering duplicates at the rest)?

I've even tried this one but still doesn't work properly... :

rex max_match=10 "(?i) portid=\"(?P[^\"]+)"|rename FIELDNAME as OpenPorts|rex max_match=10 "(?im)^<\w+><\w+\s+\w+=\"(?P[^\"]+)" |rename FIELDNAME as status

I think this is related to part after (os) or at least tell me what's the meanin of that part that repeats the result? or how can I ignore that part in my search?

I'm waiting for any kind of help,

Thanks in advance

Re: How to write regex to extract Port numbers and status from nmap output?

pradeepkumarg — Sat, 30 Aug 2014 12:03:08 GMT

Check if this works (?i)portid=\"(?P.*?\d+)\W

Re: How to write regex to extract Port numbers and status from nmap output?

gokadroid — Fri, 07 Oct 2016 07:31:35 GMT

Assuming that for each port mentioned in log we have "portid=", "state=" and "service name=" then this might work

your base search
| rex field=_raw "addr=\"(?< ip>[^\"]+)" max_match=10 
| rex field=_raw "portid=\"(?< port>[^\"]+)" max_match=10
| rex field=_raw "state=\"(?< state>[^\"]+)" max_match=10 
| rex field=_raw "service name=\"(?< service>[^\"]+)" max_match=10
| eval allFields= if(match(state, "open"), mvzip(port, service), "")
| table ip, allFields

Note: Due to formatting issue, each field **< ip>, < port>, < state>, < service_name> ** has a space between "<" and first letter of field name. Remove those spaces when runnign the query
Keep max_match according to your need

Re: How to write regex to extract Port numbers and status from nmap output?

AnthonyPhipps — Thu, 21 Sep 2017 17:27:24 GMT

This is how I parse .gnmap files.

index="gnmap" 
| eval ports=split(_raw,",")
| mvexpand ports
| Search ports!="*host*"
| rex field=_raw "Host:\s(?<dest_ip>\d+.\d+.\d+.\d+)\s+\((?<fqdn>.*)\)"
| rex field=ports "(?<port>\d+)\/+(?<status>\w+)\/+(?<proto>\w+)\/+(?<desc>\w+|\/)"
| rex field=_raw "OS:\s(?<os>\w+)"
| table dest_ip fqdn port status proto desc os

Re: How to write regex to extract Port numbers and status from nmap output?

amorgado — Thu, 21 Sep 2017 21:37:07 GMT

what does this line do ?
| foreach %* [eval <>=mvindex(<>,0)]

Re: How to write regex to extract Port numbers and status from nmap output?

AnthonyPhipps — Fri, 22 Sep 2017 11:53:34 GMT

Extraneous, I will remove it.