topic Re: props.conf entries in Getting Data In

props.conf entries

DTERM — Wed, 15 Aug 2012 18:29:00 GMT

I need to perform some date calculations in my app. Every entry in my logs will have multiple custom dates that I need to query. The time formats in the logs look like:

firsttime=2012/08/10 22:14:13
actiontime=2012/08/11 10:25:03
lasttime=2012/08/12 12:23:26

I've entered the following in props.conf

[lasttime]
TIME_PREFIX = lasttime
FORMAT = %Y/%m/%/d %H:%M:%S

[firsttime]
TIME_PREFIX = firsttime
FORMAT = %Y/%m/%/d %H:%M:%S

[actiontime]
TIME_PREFIX = actionTime
FORMAT = %Y/%m/%/d %H:%M:%S

Is this format accurate? If so, how to run a query for all lines where (for example) firsttime is older then 5 days?

Thanks

Re: props.conf entries

Ayn — Wed, 15 Aug 2012 18:47:21 GMT

Wait, are you saying that these events all have their own sourcetype, so the one with "firsttime" actually has sourcetype "firsttime" and so on? Because the stanzas you've defined in props.conf seem to suggest that.

Re: props.conf entries

lguinn2 — Wed, 15 Aug 2012 18:51:17 GMT

Can you provide a few lines of one of the logs?

Re: props.conf entries

DTERM — Thu, 16 Aug 2012 14:24:37 GMT

A single line, the rest would be the same, with different times. I'd like to be able to run a query to find all lines in a log where firstTime is older then 5 days. I know this can be accomplished using regular expressions, but I want to get away from that. Thanks.

Re: props.conf entries

Ayn — Thu, 16 Aug 2012 14:40:34 GMT

I think you're confusing some concepts here.

First of all, stanzas in props.conf refer to what type of events something should apply to - most commonly this is a sourcetype, like in your props.conf where you have for instance a stanza [actiontime]. That's telling Splunk that the settings below the stanza should apply to the sourcetype actiontime.

Second, there is no FORMAT configuration directive in props.conf. There IS a directive called TIME_FORMAT, but that only applies to how Splunk should parse a timestamp in an event for creating its own event timestamp (_time).

If you want to make use of strftime/strptime format strings for extracting time values from timestamp strings, you should use eval's strftime and strptime functions, or possibly the convert command. More information on eval, its functions and finally convert, respectively, can be found at the following sections in the docs:

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Eval
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/CommonEvalFunctions
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Convert

Re: props.conf entries

dwaddle — Thu, 16 Aug 2012 14:56:17 GMT

What you have configured so far is probably not what you want. You have told Splunk how to do timestamp recognition for three different sourcetypes, lasttime, firsttime, and actiontime. I doubt that your sample event is any of these three sourcetypes, and am 100% confident that it is not all three at once.

I am going to make an assumption that your firsttime, lasttime, and actiontime are all within a single event and that, thanks to auto-kv extraction, they are already pulled out as fields. To do your comparison, the easiest thing to do is yank them into time_t values.

sourcetype=mysourcetype 
| eval firsttime_t=strptime(firsttime,"%Y/%m/%/d %H:%M:%S")
| eval lasttime_t=strptime(lasttime,"%Y/%m/%/d %H:%M:%S")
| eval actiontime_t=strptime(actiontime,"%Y/%m/%/d %H:%M:%S")
| where firsttime_t <= (now() - (86400 * 5))

Re: props.conf entries

DTERM — Fri, 17 Aug 2012 16:07:48 GMT

Thanks for the reply.