Getting Data In

TIME_FORMAT in props.conf seems to not be working

brent_weaver
Builder

Hello all... I have events that have a timestamp that starts with:

 

 

 

 

2014-05-07 13:12:27 2910 ...

 

 

 

 

The trailing # (2910) is variable in length and this seems to mess things up. I have been setting this in my props.conf for this sorucetype:

 

 

 

 

TIME_FORMAT = %Y-%m-%d %H:%M:%S

 

 

 

 

 This is not working. What am I missing?

Labels (1)
1 Solution

richgalloway
SplunkTrust
SplunkTrust

I suspect Splunk is reluctant to index something that might be 6 years old.  There should be a message to that effect in splunkd.log.

Try adding 

MAX_DAYS_AGO = 3650

to your props.conf.  That will tell Splunk to accept dates up to 10 years old.

---
If this reply helps you, an upvote would be appreciated.

View solution in original post

inventsekar
Super Champion

Hi @brent_weaver Any ideas how this issue was fixed(for future readers reference and for our learnings as well).. maybe, accept Rich's or any post closer to your fix, choose it and accept it as the solution. 

 

Best Regards,

Sekar

PS - your karma points will be my 2 cents!

>>> Happy Splunking !
0 Karma

inventsekar
Super Champion

The trailing # (2910) is variable in length and this seems to mess things up///

whats that 2910?

is it the sub-seconds, etc.?!?!

do you want to keep this with the timestamp or not needed?

 

hope you corrected your typo (as @bowesmana mentioned)

>>> Happy Splunking !
0 Karma

brent_weaver
Builder

Hey sorry about the type, its fixed. No that trailing # is not sub seconds, its a random integer. I do not want to include this in the format.

 

Thanks for taking the time to respond and help me out!

richgalloway
SplunkTrust
SplunkTrust

The TIME_FORMAT setting should be "%Y-%m-%d %H:%M:%S", but that probably was a typo in the question rather than the code.

Please elaborate at some length on "not working".  What is the expected result?  What is the actual result?  Are there any error messages in splunkd.log?  If so, what do they say?

---
If this reply helps you, an upvote would be appreciated.

brent_weaver
Builder

Not working as in the timestamps get really crazy. Where the year is 2014 in the raw event, in Splunk it shows as current time... This clearly indicates that it is not in fact being read. I am certain it is a config issue that I am overlooking. 

My props looks like:

[testst]
DATETIME_CONFIG =
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SHOULD_LINEMERGE = false
TIME_FORMAT = %Y-%m-%d %H:%M:%S

I feel like there is just some param I am overlooking. Thank you for taking the time to help me out here.

Splunkers rock!

 

richgalloway
SplunkTrust
SplunkTrust

I suspect Splunk is reluctant to index something that might be 6 years old.  There should be a message to that effect in splunkd.log.

Try adding 

MAX_DAYS_AGO = 3650

to your props.conf.  That will tell Splunk to accept dates up to 10 years old.

---
If this reply helps you, an upvote would be appreciated.

brent_weaver
Builder

Hey there... Solid catch, that worked. Yes the timestamp was outside of the range and therefore problematic. Thank you very much for helping me out here, and on thanksgiving nonetheless. There is nothing like the Splunk community!

 

0 Karma

bowesmana
SplunkTrust
SplunkTrust

Did you mean 

TIME_FORMAT=%Y-%m-%d...

i.e. d not m?

 

brent_weaver
Builder

Yes I did. 

0 Karma
Get Updates on the Splunk Community!

Should our Deployment Servers have the Search Head server role on them?

all of our stuff is on premcurrently our dedicated Deployment Servers also have the Search Head role on them, ...

Why am I unable to create dropdown static option that is "All" static options ...

Hi, I am trying to get a static option that is "All" the individual static options combined.  The mCode ...

Where can we set the script (host=CSG196) so can we deploy the script in host?

Hi  We face a challenge We have created one alert in which we are monitoring one of the windows service ...