Getting Data In

TIME_FORMAT in props.conf seems to not be working

brent_weaver
Builder

Hello all... I have events that have a timestamp that starts with:

 

 

 

 

2014-05-07 13:12:27 2910 ...

 

 

 

 

The trailing # (2910) is variable in length and this seems to mess things up. I have been setting this in my props.conf for this sorucetype:

 

 

 

 

TIME_FORMAT = %Y-%m-%d %H:%M:%S

 

 

 

 

 This is not working. What am I missing?

Labels (1)
1 Solution

richgalloway
SplunkTrust
SplunkTrust

I suspect Splunk is reluctant to index something that might be 6 years old.  There should be a message to that effect in splunkd.log.

Try adding 

MAX_DAYS_AGO = 3650

to your props.conf.  That will tell Splunk to accept dates up to 10 years old.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

inventsekar
Super Champion

Hi @brent_weaver Any ideas how this issue was fixed(for future readers reference and for our learnings as well).. maybe, accept Rich's or any post closer to your fix, choose it and accept it as the solution. 

 

Best Regards,

Sekar

PS - your karma points will be my 2 cents!

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

inventsekar
Super Champion

The trailing # (2910) is variable in length and this seems to mess things up///

whats that 2910?

is it the sub-seconds, etc.?!?!

do you want to keep this with the timestamp or not needed?

 

hope you corrected your typo (as @bowesmana mentioned)

PS ... If any post helped you in any way, pls give a hi-five to the author with an upvote. if your issue got resolved, please accept the reply as solution.. thanks.
0 Karma

brent_weaver
Builder

Hey sorry about the type, its fixed. No that trailing # is not sub seconds, its a random integer. I do not want to include this in the format.

 

Thanks for taking the time to respond and help me out!

richgalloway
SplunkTrust
SplunkTrust

The TIME_FORMAT setting should be "%Y-%m-%d %H:%M:%S", but that probably was a typo in the question rather than the code.

Please elaborate at some length on "not working".  What is the expected result?  What is the actual result?  Are there any error messages in splunkd.log?  If so, what do they say?

---
If this reply helps you, Karma would be appreciated.

brent_weaver
Builder

Not working as in the timestamps get really crazy. Where the year is 2014 in the raw event, in Splunk it shows as current time... This clearly indicates that it is not in fact being read. I am certain it is a config issue that I am overlooking. 

My props looks like:

[testst]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S

I feel like there is just some param I am overlooking. Thank you for taking the time to help me out here.

Splunkers rock!

 

richgalloway
SplunkTrust
SplunkTrust

I suspect Splunk is reluctant to index something that might be 6 years old.  There should be a message to that effect in splunkd.log.

Try adding 

MAX_DAYS_AGO = 3650

to your props.conf.  That will tell Splunk to accept dates up to 10 years old.

---
If this reply helps you, Karma would be appreciated.

brent_weaver
Builder

Hey there... Solid catch, that worked. Yes the timestamp was outside of the range and therefore problematic. Thank you very much for helping me out here, and on thanksgiving nonetheless. There is nothing like the Splunk community!

 

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Did you mean 

TIME_FORMAT=%Y-%m-%d...

i.e. d not m?

 

brent_weaver
Builder

Yes I did. 

0 Karma
Get Updates on the Splunk Community!

Happy CX Day to our Community Superheroes!

Happy 10th Birthday CX Day!What is CX Day? It’s a global celebration recognizing innovation and success in the ...

Check out This Month’s Brand new Splunk Lantern Articles

Splunk Lantern is a customer success center providing advice from Splunk experts on valuable data insights, ...

Routing Data to Different Splunk Indexes in the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...