topic Re: Help with Regex...? in Splunk Search

Help with Regex...?

prakash007 — Tue, 18 Oct 2016 14:18:14 GMT

I'm trying to extract the following from this regex...somehow i am not able to get the browser agent and status...

some of the data doesn't have a user agent..any help to get IP, UserAgent(optional), status

(?P<method>\w+)\s(?P<URL>.*)?\s\-\s(?P<port>\d+)\s\-\s(?P<IP>\d+\.\d+\.\d+\.\d+)\s?\-?\s(?P<UserAgent>.*\))?\s?(?P<status>\d+)


GET /user/stell_static/legacy/css/share.css - 443 - 192.188.2.0 - 404

GET / - 443 - 192.168.28.0 - 200 0 0 0

GET /public/about.html - 443 - 192.168.26.7 - 403

POST /data/willer/testsample.php - 443 - 192.168.20.1 Mozilla/4.0+(Windows;+U;+Windows+NT+5.1;+en-US;+rv:1.8.1.14) 404 0 2 0

GET /ajax/real/setsample.asp - 443 - 192.168.21.0 Mozilla/5.0+(Windows+NT+5.1;+WOW64;+rv:35.0)+Gecko/20980000+Firefox/45.0 200 0 0 31 #apache webserver services #version5.5

Re: Help with Regex...?

skoelpin — Tue, 18 Oct 2016 14:23:48 GMT

You should create 2 separate fields for the user agent string and the status code

User gent string regex
(?P<UserAgent>\d+\.\d+\.\d+\.\d+\s(\S+))

Re: Help with Regex...?

gcusello — Tue, 18 Oct 2016 14:47:21 GMT

Try this one:

(?P<method>\w+)\s(?P<URL>.*)?\s\-\s(?P<port>\d+)\s\-\s(?P<IP>\d+\.\d+\.\d+\.\d+)\s(?P<UserAgent>[^ ]*)?\s?(?P<status>\d\d\d)

If you want to modify it test on https://regex101.com/

Bye.
Giuseppe

Re: Help with Regex...?

somesoni2 — Tue, 18 Oct 2016 15:11:41 GMT

This should do it

(?P<method>\w+)\s+(?P<URL>\S+)\s+\S+\s+(?P<port>\d+)\s+\S+\s+(?P<IP>\d+\.\d+\.\d+\.\d+)\s+(?P<UserAgent>\S+)\s+(?P<status>\d+)

https://regex101.com/r/Vg9lbg/1

Re: Help with Regex...?

prakash007 — Tue, 18 Oct 2016 15:15:26 GMT

somehow it's not working well for this kind of data

GET /ajax/real/setsample.asp - 443 - 192.168.21.0 Mozilla/5.0+(Windows+NT+5.1;+WOW64;+rv:35.0)+Gecko/20980000+Firefox/45.0 200 0 0 31 950 #softwate#apache 560 731

UserAgent=Mozilla/5.0+(Windows+NT+5.1;+WOW64;+rv:35.0)+Gecko/20980000+Firefox/45.0 200 0 0 31 950 #softwate#apache 560
 expected(UserAgent=Mozilla/5.0+(Windows+NT+5.1;+WOW64;+rv:35.0)+Gecko/20980000+Firefox/45.0)

status=731
(expected status=200)

GET / - 443 - 192.168.67.1 - 200 0 0 202

UserAgent= -200
(expected UserAgent = -)

status=202
(expected status=200)

Re: Help with Regex...?

inventsekar — Tue, 18 Oct 2016 15:15:47 GMT

tested and its working good..

sourcetype="post" 
| rex field=_raw "(?P<method>\w+)\s(?P<URL>.*)?\s\-\s(?P<port>\d{3})\s\-\s(?P<IP>\d+\.\d+\.\d+\.\d+)\s(?P<UserAgent>.*)\s(?P<status>\d{3})"
| table method URL port IP UserAgent status _raw

Re: Help with Regex...?

gokadroid — Tue, 18 Oct 2016 15:33:10 GMT

This will work for all you cases:

(?P<method>[\w]+)\s+(?P<URL>[^\s]+)\s+\S+\s+(?P<port>[\d]+)\s+\S+\s+(?P<IP>[\d\.]+)\s+(?P<UserAgent>[\S]+)\s+(?P<status>[\d]+)