Getting Data In

Splunk takes the wrong timestamp from the log

jorjiana88
Path Finder

I have a log that has multiple timestamps like this inside, but not all lines have such a date entry.

NOTE: 24DEC17:09:05:53.121 start executig macro main() syscc=0

The log creation date is 2017-12-24 9:05.

Some of the lines in the log are indexed with today's date (it seems to take creation date of the file), and some are indexed as if they were yesterday and at 17:09 instead of 9:05 a.m,: 12/23/17 5:09:05.570 PM

How can I make sure that Splunk takes the correct date ?

1 Solution

niketn
Legend

@jorjiana88, would it be possible to post the raw sample data of the event where timestamp recognition is not working? What is the format of timestamp on these events (is it date time or just time)?

You can get one of your sample data file and choose Settings --> Add Data --> Upload to Splunk for data preview. Note only first 1000 events in 50 pages will be displayed in the data preview mode. So make sure raw events with incorrect timestamp are in first 1000 events (you can create your own dummy file with such with few correct/incorrect log events sampled from original log files to ingest).

Under the first step in the Data Preview Mode the Set Source Type screen you should verify whether the correct timestamp is getting assigned to events or not. You can use the Timestamps option in this screen to make sure that correct timestamp gets picked up for data being ingested. Once your data preview displays correct timestamp, no need to continue with data ingestion. Under the Advanced section there should be an option to Copy to Clipboard from where you can pick up Timestamp related props.conf configuration and update to your props.conf file in production. Refer to the following few Splunk documentation to understand and configure Timestamp recognition:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Modifyeventprocessing
http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

0 Karma

niketn
Legend

@jorjiana88, would it be possible to post the raw sample data of the event where timestamp recognition is not working? What is the format of timestamp on these events (is it date time or just time)?

You can get one of your sample data file and choose Settings --> Add Data --> Upload to Splunk for data preview. Note only first 1000 events in 50 pages will be displayed in the data preview mode. So make sure raw events with incorrect timestamp are in first 1000 events (you can create your own dummy file with such with few correct/incorrect log events sampled from original log files to ingest).

Under the first step in the Data Preview Mode the Set Source Type screen you should verify whether the correct timestamp is getting assigned to events or not. You can use the Timestamps option in this screen to make sure that correct timestamp gets picked up for data being ingested. Once your data preview displays correct timestamp, no need to continue with data ingestion. Under the Advanced section there should be an option to Copy to Clipboard from where you can pick up Timestamp related props.conf configuration and update to your props.conf file in production. Refer to the following few Splunk documentation to understand and configure Timestamp recognition:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Modifyeventprocessing
http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@jorjiana88,were you able to try out the suggestion? Is your issue resolved?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

jorjiana88
Path Finder

actually we made changes to the software that was generating the logs in order to fix it.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...