Timestamp issue with Fortigate Logs

[tjensen]

Hello,

I receive Fortigate Firewall Logs via Syslog.
To separte the Logs into different facilities I've enabled the

no_priority_stripping=true

Now Splunk can't determine the correct timestamp, but there are fields named date and time available. For Example here is the beginning of a received message:

<173>date=2012-09-17 time=15:38:14 ........

What are the correct settings for the props.conf ?

I tried things like this:

[mysourcetype]
TIME_FORMAT = date=%Y-%m-%d time=%H:%M:%S
TIME_PREFIX = date=

Do you have any comments on this ?

Thanks

Torsten

[tjensen · Accepted solution]

Hi, thanks for all your help.

I find the following solution for my problem:

The sourcetype was created by a transformation from a other sourcetype.
Splunk try at first to find the timestamp and after that the transforms will be managed.
For this my settings in the props.conf are not parsed during the timestamp creation.

Now I changed my settings from sourcetype to host and it works without any problems.

This issue is solved now!

Thanks again.

[bfernandez]

This worked for me as well, thanks!

[itinney]

1. Check that the sourcetype has not changed since applying the new setting, Splunk may have learned a new sourcetype.
2. The TIME_FORMAT string should define the string that follows the TIME_PREFIX string.
3. Remember that strftime format strings are not regular expressions so you cannot use things like \s+ to represent spaces. However the TIME_PREFIX string is a regular expression.

4. Is the character between the "date" and "time" string definitely a space? You can check with the od command on linux:

   grep "date=" logFileName | head -1 | od -b

   and make sure the the space character is octal 40. It could be a tab (octal 11) or Hard-Space character (octal 240)

5. Be careful copy/pasting from web
   pages as some characters (i.e. '-')
   can get changed to different,
   similar looking characters.

Given the above checks out, the following should work:

TIME_FORMAT = date=%Y-%m-%d time=%H:%M:%S
TIME_PREFIX = ^\s*<\d{3}>

[MarioM]

if you still can't get timestamp recognised you can create a custom datetime.xml:

-by using ./splunk train date

or

-or configuring the custom datetime.xml: example

[tjensen]

Hi ranjyotiprakash,

thanks for your fast answer.
Meanwhile I take a look at this part of the docs many time, but I don't find a solution for this problem.

I tested also the following settings:

1st:
TIME_FORMAT = date=%Y-%m-%d%ttime=%H:%M:%S
TIME_PREFIX = ^</d{3}>date=
2nd:
TIME_FORMAT = %H:%M:%S
TIME_PREFIX = time=
3rd:
TIME_PREFIX = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
TIME_FORMAT = %m/%d/%y %H:%M:%S

but no success at all. So I think, the 2nd try should be works fine, but it don't.

Any comments ?

Thanks

[ranjyotiprakash]

Hi tjensen,

You can refer to following documentation to configure timestamp recognition :

Configure Timestamp Recognition

Thanks !!