topic Re: How to configure proper timestamp recognition to fix syslog date parsing? in Getting Data In

How to configure proper timestamp recognition to fix syslog date parsing?

kenniskoldewyn — Sun, 23 Nov 2014 22:53:04 GMT

We have a firewall sending events to a Splunk indexer via syslog, so we have a section of our inputs.conf file like this:

[tcp://<port over which syslog data is sent>]
connection_host = dns
host = <name of firewall>
index = firewall
sourcetype = syslog

The trouble is that the firewall's date and time format is a bit strange:

<nn>YYYY:MM:DD-HH:mm:ss ...

where nn is a two or three digit number, YYYY is the year with century, MM is a two-digit month, DD is a two-digit day, HH is a two-digit hour, mm is a two-digit minute and ss is a two-digit second. About half the time, Splunk gets the day wrong (perhaps it thinks that the - between the day and the hour is a subtraction?). There are also events that don't start with `` and don't include a date and time.

In order to fix the date parsing, I know I need to create an inputs.conf file, but I'm not clear on exactly what I should be putting into it, given that not all lines start with ``. Any suggestions?

Can anything be done to correct all of the data (about a year's worth) that has had the dates parsed incorrectly?

Thanks for any suggestions!

Re: How to configure proper timestamp recognition to fix syslog date parsing?

changux — Sun, 23 Nov 2014 23:06:55 GMT

Hi.
Check the timestamp recognition.

http://docs.splunk.com/Documentation/Splunk/6.2.0/Data/Configuretimestamprecognition

Bye.

Re: How to configure proper timestamp recognition to fix syslog date parsing?

kenniskoldewyn — Tue, 25 Nov 2014 16:05:13 GMT

I added the following section to props.conf:

[host::<name of the firewall>]
TIME_PREFIX = <\d+>
TIME_FORMAT = %Y:%m:%d-%H:%M:%S

but that didn't help. Any other ideas?

Re: How to configure proper timestamp recognition to fix syslog date parsing?

kenniskoldewyn — Tue, 25 Nov 2014 16:10:52 GMT

Corrections to the above: "I know I need to create an inputs.conf file" should be "I know I need to create a props.conf file", and "given that not all lines start with `" should be "given that not all lines start with

`".

Re: How to configure proper timestamp recognition to fix syslog date parsing?

theouhuios — Tue, 25 Nov 2014 16:38:53 GMT

TIME_PREFIX=^\<\d+\>|^
TIME_FORMAT = %Y:%m:%d-%H:%M:%S

Try this

Re: How to configure proper timestamp recognition to fix syslog date parsing?

kenniskoldewyn — Wed, 26 Nov 2014 18:47:14 GMT

Nope, still doesn't work.