How to troubleshoot why I'm not getting any date in the Fortinet Fortigate App for Splunk?

[flgrh]

Hi,

As this add-on makes the data preprocessing for the Fortigate App I think the problem is found here.

My setup:
Syslog-ng takes everything from syslog port and writes the logs into separate files on disk (per device). Splunk is configured to read them.

The problem is that I have no data in the app. If I change the sourcetype to "fgt_traffic" it works, but then I'll miss UTM and normal events as everythings is interpreted as traffic. If I set sourcetype to "fgt_logs" or "fortigate" I have no data again.

I added a SOURCE-KEY statement to the transfrom entries (I read about them somewhere here) but it didn't change anything. I checked the regex with regex101 and my logs and they match correctly.

Any ideas what I'm making wrong?

Many thanks,
Ronald

part of props.conf

[source::*]
#[source::udp:514]
TRANSFORMS-force_sourcetype_fgt = force_sourcetype_fgt_traffic,force_sourcetype_fgt_utm,force_sourcetype_fgt_event
SHOULD_LINEMERGE = false

part of transforms.conf

##sourcetype
[force_sourcetype_fgt_traffic]
SOURCE_KEY = MetaData:Sourcetype
DEST_KEY = MetaData:Sourcetype
REGEX = date=.+time=.+devid=FG.+type=traffic
FORMAT = sourcetype::fgt_traffic

[force_sourcetype_fgt_utm]
SOURCE_KEY = MetaData:Sourcetype
DEST_KEY = MetaData:Sourcetype
REGEX = date=.+time=.+devid=FG.+type=utm
FORMAT = sourcetype::fgt_utm

[force_sourcetype_fgt_event]
SOURCE_KEY = MetaData:Sourcetype
DEST_KEY = MetaData:Sourcetype
REGEX = date=.+time=.+devid=FG.+type=event
FORMAT = sourcetype::fgt_event

[jerryzhao]

Hi flgrh
because you are using syslog-ng's log file as input and tag the logs with sourcetype fgt_logs or fortigate, please add [fgt_logs] or [fortigate] in between.
[source::*]
#[source::udp:514]
[fgt_logs]
TRANSFORMS-force_sourcetype_fgt = force_sourcetype_fgt_traffic,force_sourcetype_fgt_utm,force_sourcetype_fgt_event

[mikaelbje]

There's now an official Fortinet Fortigate add-on: https://splunkbase.splunk.com/app/2846/#/documentation

I suggest you try that one instead.

The add-on I developed was created when there was no app that supports FortiOS 5.x.

[mshumate]

Originally I started to post a very negative response to this as i too was experiencing the same issues mentioned above and as all Splunk docs / answers etc....they lead you in circles ending in frustration. So once I discovered the answer I wanted to come back and respond for the original questions cause apparently it hasn't been answered yet. My situation was in the configuration for the overriding of the sourcetype in the transforms.conf file (Fortinet Fortigate Add On App). The "REGEX = date=.+time=.+devname=FW.+devid=FW.+type=traffic" had to be modified for the devid=FG to FW. Yep! as simple as that. And NOTE both apps are required, the "Fortinet Fortigate Add On for Splunk" and the "Fortinet Fortigate App for Splunk". Nothing details that anywhere. AND if you've came across this too, the README.txt files contains nothing beneficial to installing the app. Hope this help you out.

[jerryzhao]

root causes are case by case although effect might look the same. in your case mshumate, it is because of a bug in the REGEX as you mentioned, it should have allowed for both FG and FW so forti-wifi products can be processed as well.
As for dependency on the add-on, it has been stated in STEP 1 of this documentation, so maybe you missed that. https://splunkbase.splunk.com/app/2800/#/documentation

[flgrh]

Sry, but I'm refering to the offical App called "Fortinet FortiGate App for Splunk".
AFAIK your add-on is called "Fortinet Fortigate with FortiOS 5 Add-On".

[mikaelbje]

The thread was tagged with the FortiOS 5 add-on as well, hence my response.

In any way try to contact Fortinet directly by clicking on the app's author in the Download page if you're not getting any response from them here 🙂