Splunk Search

How to configure Splunk to extract the correct year from the date stamp in my sample log?

daniel_augustyn
Contributor

I am not sure how to fix the date extraction from a raw log which is done by default by Splunk. Splunk extracts date by default and it's not doing the year correctly.

This is the raw log:

Jun 21 00:00:32 10.20.14.12 Jun 20 17:00:32 : 2016/06/20 17:00:32 PDT,1,7016505,L2 Poll Failed,0,10596,,LAB,10.18.8.1,,L2 Poll failed to read hosts from LAB.

And this is date that is getting extracted:

6/20/12 5:00:32.000 PM

Anyone knows how to fix it?

0 Karma
1 Solution

ryanoconnor
Builder

Do you have any way of modifying the format of these logs? Ideally the first portion of your log would be a valid timestamp. In this case Splunk is getting confused because it sees a portion of a valid date at the beginning of the log.

You may be able to work around it using the following, assuming this is your timestamp:

2016/06/20 17:00:32 PDT

You'll need to configure a props.conf file to recognize this.

[your_sourcetype] 
TIME_PREFIX = ^.*\s:\s
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y/%m/%d %T %Z

http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Configuretimestamprecognition

View solution in original post

ryanoconnor
Builder

Do you have any way of modifying the format of these logs? Ideally the first portion of your log would be a valid timestamp. In this case Splunk is getting confused because it sees a portion of a valid date at the beginning of the log.

You may be able to work around it using the following, assuming this is your timestamp:

2016/06/20 17:00:32 PDT

You'll need to configure a props.conf file to recognize this.

[your_sourcetype] 
TIME_PREFIX = ^.*\s:\s
MAX_TIMESTAMP_LOOKAHEAD = 24
TIME_FORMAT = %Y/%m/%d %T %Z

http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/Configuretimestamprecognition

daniel_augustyn
Contributor

I added this to the props.conf stanza on the search head under system/local/ and it didn't help. I am still getting logs with wrong year in them.

0 Karma

MuS
Legend

@daniel_augustyn , theses setting need to be done where the parsing is happening, usually an indexer or a heavyweight forwarder. See this http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F to learn more about this topic.

daniel_augustyn
Contributor

Awesome, it totally fixed it.

0 Karma
Get Updates on the Splunk Community!

Splunk App for Anomaly Detection End of Life Announcment

Q: What is happening to the Splunk App for Anomaly Detection?A: Splunk is officially announcing the ...

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...