topic Help with extracting specific data in Splunk Search

Help with extracting specific data

Reosoul — Wed, 23 Mar 2016 23:44:02 GMT

Given bunch of results in a format like:

6d2112effbe814f41ef6a6b984221c2490ef5112b70d394c074bb1427561556c some.site.com REST.GET.OBJECT text/2015/xml/somefile.xml "GET /some.site.com/text/2015/xml/somefile.xmll?Expires=1479754790 HTTP/1.1" 200 - 24583 24583 27 26 "-" "-" -

How can I return only:

/some.site.com/text/2015/xml/somefile.xmll?Expires=1479754790

(in other words from: /some.site.com/ all the way till the next white space)?

Thanks.

Re: Help with extracting specific data

maciep — Thu, 24 Mar 2016 00:22:33 GMT

Well, if the format is consistent, then something like this should work

your base search | rex "GET\s+(?<cool_field>\S+)"

Essentially find in the raw data where there is GET followed by a space (or spaces), then capture everything that is not a space after that and store it in a new field called "cool_field"

If the events are not in a consistent enough format where that would work, then maybe include some of the events where it doesn't work too and I'm sure we can adjust

Re: Help with extracting specific data

Reosoul — Thu, 24 Mar 2016 14:11:23 GMT

Great that worked! If within that cool_field I have some events that end with .jpg or .jpeg, is there a way to filter those out?

Re: Help with extracting specific data

maciep — Thu, 24 Mar 2016 16:33:06 GMT

Not sure I understand exactly where they would be since the field ends with the expires part, but something like this should remove anything .jpg in that field

 your base search | rex "GET\s+(?<cool_field>\S+)" | rex field=cool_field mode=sed "s/\.jpe?g//g"

The sed mode of the rex command can be used for replacing strings. In this case, I'm looking for anything .jp(e)g and replacing it with nothing...and that's done (g)lobally in that field.