I have a couple searches like below to extract field based on a condition of existence of a string in the log.
index="access_log" protocol="http" | rex field=url "<field_extraction_01>" index="access_log" protocol="rtsp" | rex field=url "<field_extraction_02>" index="access_log" protocol="rtmp" | rex field=url "<field_extraction_03>"
How will I implement this search time field extraction using props OR transforms ?
Thanks in Advance.
I guess that you'd have to EXTRACT them for all events, or attempt to. Naturally they would fail in the case of a different protocol.
If you could provide a few sample events, it would be easier to see if you need one or more EXTRACTs. Depending on the look of your events you could perhaps do something like;
[sourcetype] EXTRACT-woo = proto=(http|rtsp|rtmp)\s+url=(?<field>\w+)[\w.-]+\s
As requested I have added the props and sample events. Based on protocol I need two regex to extract the asset value from the field "URL".
10.11.12.13 WMPlayer 2013-04-04 03:49:59 rtsp://XXXXX.com/channelA?SIGV=adbfvbavbsbmsdbsvbha67v8776avavkjah89 RTSP 404 -
10.11.12.14 Mozilla 2013-04-04 03:49:59 http://XXXXX.com/ABC/123/456/DEFGHGJGJG/DragonsGate-270x390.jpg HTTP 302 -
You can use either props or transforms. But in any case make sure that you scope the extraction on the field of interest. If you can't produce a regex that alternates on protocol, you may need to have separate regexes for each case. Here's an example using
EXTRACT-xxx from props.conf. The "
in url" string after the regex, narrows the scope of the extraction only on the
EXTRACT-url_asset_01 = my_regex_01 in url
EXTRACT-url_asset_02 = my_regex_03 in url
EXTRACT-url_asset_03 = my_regex_03 in url