Building for the Splunk Platform

Time stamp for index time extraction

sreejith2k2
Explorer

Following are the different time stamp we are getting from different sources and trying to write a time stamp for the index time extraction. Your help is much appreciated.

10.10.10.10 - - [06/Mar/2017:11:45:30 +0000] "GET /service....."
2017-03-05T16:03:50.457678+00:00 HOSTNAME
17/3/5@13:03:01: EXIT
Mar 3 16:01:34
Fri Mar 3 15:54:59 2017
2017-03-05 13:14:39+00000
2017-03-05 15:22:39,849

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi sreejith2k2, First of all, you'll want to make sure that the events with these different time formats are partitioned out to their own sources and/or sourcetypes. I'd guess that Splunk can probably make sense of the timestamp for at least some of these formats.

For the sources that Splunk can't recognize the timestamp for (the "Add Data" wizard is great for determining this, take a sample set of events and run it through that to immediately find out if Splunk can figure it out), you can set Props configuration on the source/sourcetype to tell Splunk some attributes concerning the timestamp in the events. See this for more details : http://docs.splunk.com/Documentation/Splunk/6.5.2/Data/Configuretimestamprecognition

Essentially, you can tell Splunk the strptime format ( strptime ) , you can give it a regex for a pattern that precedes the timestamp ( TIME_PREFIX ), and you can tell it how many characters either into the event, or from the prefix it should look for the timestamp ( MAX_TIMESTAMP_LOOKAHEAD )

Also, see the "Timestamp extraction configuration" section of the props.conf spec for a full list of available configuration directives.

Please let me know if this answers your question!

lakshman239
SplunkTrust
SplunkTrust

Are you having more than one time format in an event for a given data source or the logs from different sources have diff time format? ( in the former, you can specific which timestamp to use for TIME_FORMAT and TIME_PREFIX. In the later, how about giving a different sourcetype to each data source and define its timestamp as per the format in the event).

Pls let me know if I am missing something.

0 Karma

adonio
Ultra Champion

Hi sreejith2k2,
you can use this doc as a reference: https://docs.splunk.com/Documentation/Splunk/6.5.2/SearchReference/Commontimeformatvariables
for line number 1 it will be %d/%b/%Y:%H:%M %z
Hope it helps

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Each source should have its own config settings, including timestamp and sourcetype.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Observability Highlights | January 2023 Newsletter

 January 2023New Product Releases Splunk Network Explorer for Infrastructure MonitoringSplunk unveils Network ...

Security Highlights | January 2023 Newsletter

January 2023 Splunk Security Essentials (SSE) 3.7.0 ReleaseThe free Splunk Security Essentials (SSE) 3.7.0 app ...

Platform Highlights | January 2023 Newsletter

 January 2023Peace on Earth and Peace of Mind With Business ResilienceAll organizations can start the new year ...