topic Re: problem extracting timestamps in Getting Data In

problem extracting timestamps

StefNighthawk — Mon, 18 Jun 2012 10:20:09 GMT

I'm unable to get correct timestamps for my snmptrap log-file.

This is an example of an snmptrap :

2012-06-18 11:55:12 servername [UDP: [10.10.10.10]:32768->[0.0.0.0]:0]:

Splunk does not take 2012-06-18 11:55:12 as timestamp, instead it takes the last-modified date of the log-file which remains the same as it is an always-edited file.

I've tried with following settings in props.conf :
[snmptrap] SHOULD_LINEMERGE=False TIME_PREFIX= TIME_FORMAT=%Y-%m-%d %H:%M:%S LINE_BREAKER=([\r\n]+)\d{4}-\d{2}-\d{2}\s\d{d}

But still not getting the correct timestamp.
What am I doing wrong?

Re: problem extracting timestamps

Takajian — Mon, 28 Sep 2020 11:57:18 GMT

If you change your props.conf, it will not affect until you reboot splunk and re-index the data. And you do not need "LINE_BREAKER" setting when "SHOULD_LINEMERGE=false".

Re: problem extracting timestamps

StefNighthawk — Mon, 28 Sep 2020 11:57:20 GMT

I've restarted the entire server instead of the splunk service only.
Thus the log file has been recreated and with it it's modification time.
Currently I have no way to tell whether Splunk is using the file modification time or the timestamp.

I already see that timestartpos and timeendpos have values.
The weird thing is that the timestamping went OK for more than two days and then suddenly stopped. Before I configured TIME*FORMAT the timestartpos and timeendpos were (0,20), when timestamping broke their value became null and now with TIME*FORMAT its (0,19).

I hope timestamping will stay correct this time.

Re: problem extracting timestamps

Takajian — Mon, 18 Jun 2012 11:30:37 GMT

I think I can index your timestamp of the file by default setting. How did you log the snmptrap to the file? Does the file contains the timestamp as you described?

Re: problem extracting timestamps

StefNighthawk — Mon, 18 Jun 2012 11:50:34 GMT

I used snmptrapd from net-snmp to create a log-file locally on the splunk server. Each event in the file starts with the timestamp like 2012-06-18 11:55:12. Until yesterday there was no problem using the default setting.
Once splunk started to use the file modification time instead of the timestamp events started to no longer be properly split.

Re: problem extracting timestamps

Takajian — Mon, 18 Jun 2012 12:13:42 GMT

It's strange. Please clean event at all, then try to re-index the file.