topic Re: Timestamp issue with Fortigate Logs in Getting Data In

Timestamp issue with Fortigate Logs

tjensen — Mon, 17 Sep 2012 13:31:47 GMT

Hello,

I receive Fortigate Firewall Logs via Syslog.
To separte the Logs into different facilities I've enabled the

no_priority_stripping=true

Now Splunk can't determine the correct timestamp, but there are fields named date and time available. For Example here is the beginning of a received message:

<173>date=2012-09-17 time=15:38:14 ........

What are the correct settings for the props.conf ?

I tried things like this:

[mysourcetype] TIME_FORMAT = date=%Y-%m-%d time=%H:%M:%S TIME_PREFIX = date=

Do you have any comments on this ?

Thanks

Torsten

Re: Timestamp issue with Fortigate Logs

ranjyotiprakash — Mon, 17 Sep 2012 13:40:56 GMT

Hi tjensen,

You can refer to following documentation to configure timestamp recognition :

Configure Timestamp Recognition

Thanks !!

Re: Timestamp issue with Fortigate Logs

tjensen — Mon, 17 Sep 2012 14:19:36 GMT

Hi ranjyotiprakash,

thanks for your fast answer.
Meanwhile I take a look at this part of the docs many time, but I don't find a solution for this problem.

I tested also the following settings:
1st: TIME_FORMAT = date=%Y-%m-%d%ttime=%H:%M:%S TIME_PREFIX = ^</d{3}>date= 2nd: TIME_FORMAT = %H:%M:%S TIME_PREFIX = time= 3rd: TIME_PREFIX = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2} TIME_FORMAT = %m/%d/%y %H:%M:%S

but no success at all. So I think, the 2nd try should be works fine, but it don't.

Any comments ?

Thanks

Re: Timestamp issue with Fortigate Logs

MarioM — Tue, 18 Sep 2012 07:21:23 GMT

if you still can't get timestamp recognised you can create a custom datetime.xml:

-by using ./splunk train date

-or configuring the custom datetime.xml: example

Re: Timestamp issue with Fortigate Logs

itinney — Mon, 28 Sep 2020 12:27:26 GMT

Check that the sourcetype has not changed since applying the new setting, Splunk may have learned a new sourcetype.
The TIME_FORMAT string should define the string that follows the TIME_PREFIX string.
Remember that strftime format strings are not regular expressions so you cannot use things like \s+ to represent spaces. However the TIME_PREFIX string is a regular expression.
Is the character between the "date" and "time" string definitely a space? You can check with the od command on linux:

grep "date=" logFileName | head -1 | od -b

and make sure the the space character is octal 40. It could be a tab (octal 11) or Hard-Space character (octal 240)
Be careful copy/pasting from web
pages as some characters (i.e. '-')
can get changed to different,
similar looking characters.

Given the above checks out, the following should work:

TIME_FORMAT = date=%Y-%m-%d time=%H:%M:%S
TIME_PREFIX = ^\s*<\d{3}>

Re: Timestamp issue with Fortigate Logs

tjensen — Tue, 18 Sep 2012 10:00:07 GMT

Hi, thanks for all your help.

I find the following solution for my problem:

The sourcetype was created by a transformation from a other sourcetype.
Splunk try at first to find the timestamp and after that the transforms will be managed.
For this my settings in the props.conf are not parsed during the timestamp creation.

Now I changed my settings from sourcetype to host and it works without any problems.

This issue is solved now!

Thanks again.

Re: Timestamp issue with Fortigate Logs

bfernandez — Tue, 02 Oct 2012 14:07:33 GMT

This worked for me as well, thanks!