Getting Data In

Splunk takes the wrong timestamp from the log

jorjiana88
Path Finder

I have a log that has multiple timestamps like this inside, but not all lines have such a date entry.

NOTE: 24DEC17:09:05:53.121 start executig macro main() syscc=0

The log creation date is 2017-12-24 9:05.

Some of the lines in the log are indexed with today's date (it seems to take creation date of the file), and some are indexed as if they were yesterday and at 17:09 instead of 9:05 a.m,: 12/23/17 5:09:05.570 PM

How can I make sure that Splunk takes the correct date ?

1 Solution

niketn
Legend

@jorjiana88, would it be possible to post the raw sample data of the event where timestamp recognition is not working? What is the format of timestamp on these events (is it date time or just time)?

You can get one of your sample data file and choose Settings --> Add Data --> Upload to Splunk for data preview. Note only first 1000 events in 50 pages will be displayed in the data preview mode. So make sure raw events with incorrect timestamp are in first 1000 events (you can create your own dummy file with such with few correct/incorrect log events sampled from original log files to ingest).

Under the first step in the Data Preview Mode the Set Source Type screen you should verify whether the correct timestamp is getting assigned to events or not. You can use the Timestamps option in this screen to make sure that correct timestamp gets picked up for data being ingested. Once your data preview displays correct timestamp, no need to continue with data ingestion. Under the Advanced section there should be an option to Copy to Clipboard from where you can pick up Timestamp related props.conf configuration and update to your props.conf file in production. Refer to the following few Splunk documentation to understand and configure Timestamp recognition:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Modifyeventprocessing
http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"

View solution in original post

0 Karma

niketn
Legend

@jorjiana88, would it be possible to post the raw sample data of the event where timestamp recognition is not working? What is the format of timestamp on these events (is it date time or just time)?

You can get one of your sample data file and choose Settings --> Add Data --> Upload to Splunk for data preview. Note only first 1000 events in 50 pages will be displayed in the data preview mode. So make sure raw events with incorrect timestamp are in first 1000 events (you can create your own dummy file with such with few correct/incorrect log events sampled from original log files to ingest).

Under the first step in the Data Preview Mode the Set Source Type screen you should verify whether the correct timestamp is getting assigned to events or not. You can use the Timestamps option in this screen to make sure that correct timestamp gets picked up for data being ingested. Once your data preview displays correct timestamp, no need to continue with data ingestion. Under the Advanced section there should be an option to Copy to Clipboard from where you can pick up Timestamp related props.conf configuration and update to your props.conf file in production. Refer to the following few Splunk documentation to understand and configure Timestamp recognition:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Modifyeventprocessing
http://docs.splunk.com/Documentation/Splunk/latest/Data/Configuretimestamprecognition
http://docs.splunk.com/Documentation/Splunk/latest/Data/HowSplunkextractstimestamps

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

niketn
Legend

@jorjiana88,were you able to try out the suggestion? Is your issue resolved?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

jorjiana88
Path Finder

actually we made changes to the software that was generating the logs in order to fix it.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...