Getting Data In

Timestamp issue with Fortigate Logs

Explorer

Hello,

I receive Fortigate Firewall Logs via Syslog.
To separte the Logs into different facilities I've enabled the

no_priority_stripping=true

Now Splunk can't determine the correct timestamp, but there are fields named date and time available. For Example here is the beginning of a received message:

<173>date=2012-09-17 time=15:38:14 ........

What are the correct settings for the props.conf ?

I tried things like this:

[mysourcetype]
TIME_FORMAT = date=%Y-%m-%d time=%H:%M:%S
TIME_PREFIX = date=

Do you have any comments on this ?

Thanks

Torsten

Tags (1)
0 Karma
1 Solution

Explorer

Hi, thanks for all your help.

I find the following solution for my problem:

The sourcetype was created by a transformation from a other sourcetype.
Splunk try at first to find the timestamp and after that the transforms will be managed.
For this my settings in the props.conf are not parsed during the timestamp creation.

Now I changed my settings from sourcetype to host and it works without any problems.

This issue is solved now!

Thanks again.

View solution in original post

0 Karma

Explorer

Hi, thanks for all your help.

I find the following solution for my problem:

The sourcetype was created by a transformation from a other sourcetype.
Splunk try at first to find the timestamp and after that the transforms will be managed.
For this my settings in the props.conf are not parsed during the timestamp creation.

Now I changed my settings from sourcetype to host and it works without any problems.

This issue is solved now!

Thanks again.

View solution in original post

0 Karma

Communicator

This worked for me as well, thanks!

0 Karma

Path Finder
  1. Check that the sourcetype has not changed since applying the new setting, Splunk may have learned a new sourcetype.
  2. The TIMEFORMAT string should define the string that follows the TIMEPREFIX string.
  3. Remember that strftime format strings are not regular expressions so you cannot use things like \s+ to represent spaces. However the TIME_PREFIX string is a regular expression.
  4. Is the character between the "date" and "time" string definitely a space? You can check with the od command on linux:

    grep "date=" logFileName | head -1 | od -b

    and make sure the the space character is octal 40. It could be a tab (octal 11) or Hard-Space character (octal 240)

  5. Be careful copy/pasting from web
    pages as some characters (i.e. '-')
    can get changed to different,
    similar looking characters.

Given the above checks out, the following should work:

TIME_FORMAT = date=%Y-%m-%d time=%H:%M:%S
TIME_PREFIX = ^\s*<\d{3}>

Motivator

if you still can't get timestamp recognised you can create a custom datetime.xml:

-by using ./splunk train date

or

-or configuring the custom datetime.xml: example

0 Karma

Explorer

Hi ranjyotiprakash,

thanks for your fast answer.
Meanwhile I take a look at this part of the docs many time, but I don't find a solution for this problem.

I tested also the following settings:

1st:
TIME_FORMAT = date=%Y-%m-%d%ttime=%H:%M:%S
TIME_PREFIX = ^</d{3}>date=
2nd:
TIME_FORMAT = %H:%M:%S
TIME_PREFIX = time=
3rd:
TIME_PREFIX = \d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}
TIME_FORMAT = %m/%d/%y %H:%M:%S

but no success at all. So I think, the 2nd try should be works fine, but it don't.

Any comments ?

Thanks

0 Karma

Communicator

Hi tjensen,

You can refer to following documentation to configure timestamp recognition :

Configure Timestamp Recognition

Thanks !!

0 Karma