topic Time stamp for index time extraction in Splunk Dev

Time stamp for index time extraction

sreejith2k2 — Mon, 06 Mar 2017 12:20:16 GMT

Following are the different time stamp we are getting from different sources and trying to write a time stamp for the index time extraction. Your help is much appreciated.

10.10.10.10 - - [06/Mar/2017:11:45:30 +0000] "GET /service....."
2017-03-05T16:03:50.457678+00:00 HOSTNAME
17/3/5@13:03:01: EXIT
Mar 3 16:01:34
Fri Mar 3 15:54:59 2017
2017-03-05 13:14:39+00000
2017-03-05 15:22:39,849

Re: Time stamp for index time extraction

adonio — Mon, 06 Mar 2017 13:42:41 GMT

Hi sreejith2k2,
you can use this doc as a reference: https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Commontimeformatvariables
for line number 1 it will be %d/%b/%Y:%H:%M %z
Hope it helps

Re: Time stamp for index time extraction

lakshman239 — Tue, 29 Sep 2020 13:06:52 GMT

Are you having more than one time format in an event for a given data source or the logs from different sources have diff time format? ( in the former, you can specific which timestamp to use for TIME_FORMAT and TIME_PREFIX. In the later, how about giving a different sourcetype to each data source and define its timestamp as per the format in the event).

Pls let me know if I am missing something.

Re: Time stamp for index time extraction

richgalloway — Mon, 06 Mar 2017 14:02:01 GMT

Each source should have its own config settings, including timestamp and sourcetype.

Re: Time stamp for index time extraction

muebel — Mon, 06 Mar 2017 14:13:23 GMT

Hi sreejith2k2, First of all, you'll want to make sure that the events with these different time formats are partitioned out to their own sources and/or sourcetypes. I'd guess that Splunk can probably make sense of the timestamp for at least some of these formats.

For the sources that Splunk can't recognize the timestamp for (the "Add Data" wizard is great for determining this, take a sample set of events and run it through that to immediately find out if Splunk can figure it out), you can set Props configuration on the source/sourcetype to tell Splunk some attributes concerning the timestamp in the events. See this for more details : http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Configuretimestamprecognition

Essentially, you can tell Splunk the strptime format ( strptime ) , you can give it a regex for a pattern that precedes the timestamp ( TIME_PREFIX ), and you can tell it how many characters either into the event, or from the prefix it should look for the timestamp ( MAX_TIMESTAMP_LOOKAHEAD )

Also, see the "Timestamp extraction configuration" section of the props.conf spec for a full list of available configuration directives.

Please let me know if this answers your question!