Index-time extraction of multiple timestamps fields within a single event

[robert_vincent]

Need to set up searching and alerting for batch-job logging. Each log line will have the following format:

timestamp|uuid|appname|next-timestamp|max-execution-time|start-stop-code

So I need to set up a search that starts with:

index="batch" | transaction fields=uuid

And then add logic that tests for:

• linecount=1 | where ( timestamp + max-execution-time > current time )
• Most recent "next-timestamp" for each appname is less than current time

Problem is, I don't know how to set up index-time field extraction for multiple timestamps within a single event. Help?

[kristian_kolb]

You do not normally need to extract anything at index-time. You can make your comparisons with search-time extracted data.

Don't know what you really want to do, and what the transaction is used for, but if max-execution-time is in seconds, the logic/math will be rather simple. Current time (when the search starts) can be found via now().

...| eval XXX = _time + max-execution-time 
| eval YYY = if(XXX > now() AND next-timestamp < now(),"apple", "orange")

Perhaps you want to also look at the dedup command to let you only get the most recent event for some field.

See;

http://docs.splunk.com/Documentation/Splunk/5.0.4/SearchReference/CommonEvalFunctions

Some more explanation and a few sample events would let people here understand your problem better and be able to help you more.

/K

[lukejadamec]

Is your data showing up as a file's worth of data in one event? Or, is each line showing up as a single event with it's own timestamp?

[linu1988]

Finally one timestamp will be recognized. Better to do leave it as it is, the first timestamp field will be recorded as _time field. Then you can go for the field extraction on the UI, which will be easier for you. it will add those entries to props.conf, then you can refer them to do the manual extraction entry yourself.

http://docs.splunk.com/Documentation/Splunk/5.0.4/admin/Propsconf