topic Re: Conditional Field Extraction via props and transforms in Splunk Search

Conditional Field Extraction via props and transforms

KarunK — Wed, 27 Mar 2013 05:56:33 GMT

Hi All,

I have a couple searches like below to extract field based on a condition of existence of a string in the log.

index="access_log" protocol="http" | rex field=url "<field_extraction_01>"
index="access_log" protocol="rtsp" | rex field=url "<field_extraction_02>"
index="access_log" protocol="rtmp" | rex field=url "<field_extraction_03>"

How will I implement this search time field extraction using props OR transforms ?

Thanks in Advance.

Cheers

Re: Conditional Field Extraction via props and transforms

kristian_kolb — Wed, 27 Mar 2013 06:07:25 GMT

I guess that you'd have to EXTRACT them for all events, or attempt to. Naturally they would fail in the case of a different protocol.

If you could provide a few sample events, it would be easier to see if you need one or more EXTRACTs. Depending on the look of your events you could perhaps do something like;

[sourcetype]
EXTRACT-woo = proto=(http|rtsp|rtmp)\s+url=(?<field>\w+)[\w.-]+\s

Re: Conditional Field Extraction via props and transforms

KarunK — Thu, 04 Apr 2013 03:58:36 GMT

As requested I have added the props and sample events. Based on protocol I need two regex to extract the asset value from the field "URL".

EXTRACT-router=^(?P\S+?)\s+(?P\S+?)\s+(?P\S+?)\s+(?P\S+?)\s+(?P\S+?)\s+(?P\S+?)\s+(?P\S+?)\s+(?P\S+?)\s*$

10.11.12.13 WMPlayer 2013-04-04 03:49:59 rtsp://XXXXX.com/channelA?SIGV=adbfvbavbsbmsdbsvbha67v8776avavkjah89 RTSP 404 -
10.11.12.14 Mozilla 2013-04-04 03:49:59 http://XXXXX.com/ABC/123/456/DEFGHGJGJG/DragonsGate-270x390.jpg HTTP 302 -

Re: Conditional Field Extraction via props and transforms

_d_ — Thu, 04 Apr 2013 04:15:56 GMT

You can use either props or transforms. But in any case make sure that you scope the extraction on the field of interest. If you can't produce a regex that alternates on protocol, you may need to have separate regexes for each case. Here's an example using EXTRACT-xxx from props.conf. The "in url" string after the regex, narrows the scope of the extraction only on the url field.

[my_stanza] EXTRACT-router= ... EXTRACT-url_asset_01 = my_regex_01 in url EXTRACT-url_asset_02 = my_regex_03 in url EXTRACT-url_asset_03 = my_regex_03 in url