Getting Data In

How to configure proper timestamp recognition to fix syslog date parsing?

kenniskoldewyn
Explorer

We have a firewall sending events to a Splunk indexer via syslog, so we have a section of our inputs.conf file like this:

[tcp://<port over which syslog data is sent>]
connection_host = dns
host = <name of firewall>
index = firewall
sourcetype = syslog

The trouble is that the firewall's date and time format is a bit strange:

<nn>YYYY:MM:DD-HH:mm:ss ...

where nn is a two or three digit number, YYYY is the year with century, MM is a two-digit month, DD is a two-digit day, HH is a two-digit hour, mm is a two-digit minute and ss is a two-digit second. About half the time, Splunk gets the day wrong (perhaps it thinks that the - between the day and the hour is a subtraction?). There are also events that don't start with `` and don't include a date and time.

In order to fix the date parsing, I know I need to create an inputs.conf file, but I'm not clear on exactly what I should be putting into it, given that not all lines start with ``. Any suggestions?

Can anything be done to correct all of the data (about a year's worth) that has had the dates parsed incorrectly?

Thanks for any suggestions!

0 Karma

kenniskoldewyn
Explorer

Corrections to the above: "I know I need to create an inputs.conf file" should be "I know I need to create a props.conf file", and "given that not all lines start with `" should be "given that not all lines start with

`".

0 Karma

changux
Builder
0 Karma

kenniskoldewyn
Explorer

I added the following section to props.conf:

[host::<name of the firewall>]
TIME_PREFIX = <\d+>
TIME_FORMAT = %Y:%m:%d-%H:%M:%S

but that didn't help. Any other ideas?

0 Karma

theouhuios
Motivator
TIME_PREFIX=^\<\d+\>|^
TIME_FORMAT = %Y:%m:%d-%H:%M:%S

Try this

0 Karma

kenniskoldewyn
Explorer

Nope, still doesn't work.

0 Karma
Take the 2021 Splunk Career Survey

Help us learn about how Splunk has
impacted your career by taking the 2021 Splunk Career Survey.

Earn $50 in Amazon cash!