Thank You both @PickleRick @meetmshah   I tried to fw the logs of cisco into a syslog prior to splunk and the situation is the same. So i will try to find if there is a bug with cisco asa Br ... View more

Installed on the SPLUNK we do have Splunk Add-on for Cisco ASA. Maybe there is any misconfiguration here ? ... View more

The logs are being sent from CISCO ASA directly to SPLUNK without any syslog in the middle.  ... View more

Hi there, I just checked the original logs stored on /opt/syslog/cisco_asa/XXXXXX and the log entries are doubled or even tripled.  Do you think it's a bug of cisco or any wrong configuration on the SPLUNK side ? I'm thinking most probably it's a SPLUNK misconfiguration since i do have 5 different cisco asa FWs and all have the same issue. Thank You    ... View more

Sorry to bother your since i'm new to Splunk and i came from RSA where this "Tricks" did not existed at all. How and where i can check which model of license i do have ? Thank You  ... View more

I'm sharing here the final version of syslog-ng.conf that worked on trimming part of fortigate logs. However i noticed that license is being consumed like before 😞    @version: 3.35 @include "scl.conf"   # Syslog-ng configuration file, compatible with default Debian syslogd # installation.   # First, set some global options. options {      chain_hostnames(off);      flush_lines(0);      use_dns(no);      use_fqdn(no);     owner("root");      group("adm");      perm(0640);      stats_freq(0);     bad_hostname("^gconfd$");     create_dirs(yes);     keep_hostname(yes);     log_fifo_size(2048);     log_msg_size(8192);     time_reopen(10); }; # Adding param to make syslog-ng listen on udp/514 for syslog source s_net {     udp(port(514)); }; # Adding destination for local file to receive FortiGate logs   destination d_fortinet_fortigate {     file("/root/syslog/logs/fortinet/fortigate/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortigate.log" create_dirs(yes)); };   destination d_fortinet_fortiweb {     file("/root/syslog/logs/fortinet/fortiweb/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiweb.log" create_dirs(yes)); };   destination d_fortinet_fortiauthenticator {     file("/root/syslog/logs/fortinet/fortiauthenticator/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiauthenticator.log" create_dirs(yes)); };   destination d_fortinet_fortimail {     file("/root/syslog/logs/fortinet/fortimail/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortimail.log" create_dirs(yes)); }; # Filter to instruct syslog-ng how to identify FortiGate syslog filter f_fortinet_fortigate {     match("devid=\"FG[A-Z0-9]+\"" value("MESSAGE")); };   filter f_fortinet_fortiweb {     match("device_id=FV[A-Z0-9]+" value("MESSAGE")); };   filter f_fortinet_fortiauthenticator {     match("subcategory=\"Authentication\"" value("MESSAGE")); };   filter f_fortinet_fortimail {     match("device_id=FE[A-Z0-9]+" value("MESSAGE")); };   # Trimming different parts together for logging   rewrite rewrite_fortigate {  #subst ( '^time=\d\d:\d\d:\d\d(\.\d{3,6})? ', "", value("MESSAGE") );  subst ( ' devid="[A-Z0-9]{16}" ', " ", value("MESSAGE") );  subst ( ' logid="[0-9]{10}" ', " ", value("MESSAGE") );  subst ( ' srcname="([\w-]+)\.example\.local" ', ' srcname="$1" ', value("MESSAGE") );  subst ( ' srcintfrole="[a-z]+" ', " ", value("MESSAGE") );  subst ( ' dstname="([\w-]+)\.example\.local" ', ' dstname="$1" ', value("MESSAGE") );  subst ( ' dstintfrole="[a-z]+" ', " ", value("MESSAGE") );  subst ( ' poluuid="[a-z0-9-]{36}" ', " ", value("MESSAGE") );  subst ( ' sessionid=[0-9]+ ', " ", value("MESSAGE") );  subst ( ' policytype="policy" ', " ", value("MESSAGE") );  subst ( ' appcat="unscanned"', "", value("MESSAGE") );  subst ( ' crscore=[0-9]+ craction=[0-9]+ crlevel="[a-z]+"', "", value("MESSAGE") );  subst ( ' attackid=[0-9]+ ', " ", value("MESSAGE") );  subst ( ' incidentserialno=[0-9]+ ', " ", value("MESSAGE") );  subst ( ' ref="http:\/\/www\.fortinet\.com\/ids\/VID[0-9]+"', " ", value("MESSAGE") );  subst ( ' msg="application[s]?([0-9]+)?:[^.]+.[^.]+.[^.]+.[^.]+.[^.]+"', " ", value("MESSAGE") ); };   # Creating the different parts together for logging   log {     source(s_net);     filter(f_fortinet_fortigate); rewrite(rewrite_fortigate);     destination(d_fortinet_fortigate); };   log {      source(s_net);     filter(f_fortinet_fortiweb);     destination(d_fortinet_fortiweb); };   log {      source(s_net);     filter(f_fortinet_fortiauthenticator);     destination(d_fortinet_fortiauthenticator); };   log {     source(s_net);     filter(f_fortinet_fortimail);     destination(d_fortinet_fortimail); }; ### # Include all config files in /etc/syslog-ng/conf.d/ ### @include "/etc/syslog-ng/conf.d/*.conf" ... View more

Sharing here for the community the final version of syslog-ng.conf that helped me trimming the logs. However until now during the monitoring i have noticed that unfortunately there is not any difference on the license consume. It is still the same 😞    @version: 3.35 @include "scl.conf"   # Syslog-ng configuration file, compatible with default Debian syslogd # installation.   # First, set some global options. options {      chain_hostnames(off);      flush_lines(0);      use_dns(no);      use_fqdn(no);     owner("root");      group("adm");      perm(0640);      stats_freq(0);     bad_hostname("^gconfd$");     create_dirs(yes);     keep_hostname(yes);     log_fifo_size(2048);     log_msg_size(8192);     time_reopen(10); }; # Adding param to make syslog-ng listen on udp/514 for syslog source s_net {     udp(port(514)); }; # Adding destination for local file to receive FortiGate logs   destination d_fortinet_fortigate {     file("/root/syslog/logs/fortinet/fortigate/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortigate.log" create_dirs(yes)); };   destination d_fortinet_fortiweb {     file("/root/syslog/logs/fortinet/fortiweb/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiweb.log" create_dirs(yes)); };   destination d_fortinet_fortiauthenticator {     file("/root/syslog/logs/fortinet/fortiauthenticator/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiauthenticator.log" create_dirs(yes)); };   destination d_fortinet_fortimail {     file("/root/syslog/logs/fortinet/fortimail/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortimail.log" create_dirs(yes)); }; # Filter to instruct syslog-ng how to identify FortiGate syslog filter f_fortinet_fortigate {     match("devid=\"FG[A-Z0-9]+\"" value("MESSAGE")); };   filter f_fortinet_fortiweb {     match("device_id=FV[A-Z0-9]+" value("MESSAGE")); };   filter f_fortinet_fortiauthenticator {     match("subcategory=\"Authentication\"" value("MESSAGE")); };   filter f_fortinet_fortimail {     match("device_id=FE[A-Z0-9]+" value("MESSAGE")); };   # Trimming different parts together for logging   rewrite rewrite_fortigate {  #subst ( '^time=\d\d:\d\d:\d\d(\.\d{3,6})? ', "", value("MESSAGE") );  subst ( ' devid="[A-Z0-9]{16}" ', " ", value("MESSAGE") );  subst ( ' logid="[0-9]{10}" ', " ", value("MESSAGE") );  subst ( ' srcname="([\w-]+)\.example\.local" ', ' srcname="$1" ', value("MESSAGE") );  subst ( ' srcintfrole="[a-z]+" ', " ", value("MESSAGE") );  subst ( ' dstname="([\w-]+)\.example\.local" ', ' dstname="$1" ', value("MESSAGE") );  subst ( ' dstintfrole="[a-z]+" ', " ", value("MESSAGE") );  subst ( ' poluuid="[a-z0-9-]{36}" ', " ", value("MESSAGE") );  subst ( ' sessionid=[0-9]+ ', " ", value("MESSAGE") );  subst ( ' policytype="policy" ', " ", value("MESSAGE") );  subst ( ' appcat="unscanned"', "", value("MESSAGE") );  subst ( ' crscore=[0-9]+ craction=[0-9]+ crlevel="[a-z]+"', "", value("MESSAGE") );  subst ( ' attackid=[0-9]+ ', " ", value("MESSAGE") );  subst ( ' incidentserialno=[0-9]+ ', " ", value("MESSAGE") );  subst ( ' ref="http:\/\/www\.fortinet\.com\/ids\/VID[0-9]+"', " ", value("MESSAGE") );  subst ( ' msg="application[s]?([0-9]+)?:[^.]+.[^.]+.[^.]+.[^.]+.[^.]+"', " ", value("MESSAGE") ); };   # Creating the different parts together for logging   log {     source(s_net);     filter(f_fortinet_fortigate); rewrite(rewrite_fortigate);     destination(d_fortinet_fortigate); };   log {      source(s_net);     filter(f_fortinet_fortiweb);     destination(d_fortinet_fortiweb); };   log {      source(s_net);     filter(f_fortinet_fortiauthenticator);     destination(d_fortinet_fortiauthenticator); };   log {     source(s_net);     filter(f_fortinet_fortimail);     destination(d_fortinet_fortimail); }; ### # Include all config files in /etc/syslog-ng/conf.d/ ### @include "/etc/syslog-ng/conf.d/*.conf" ... View more

So, if i understood correct: Instead of log { source(s_net); filter(f_fortinet_fortigate); destination(d_fortinet_fortigate); }; should be like this: r rewrite rewrite_fortigate { #subst ( '^time=\d\d:\d\d:\d\d(\.\d{3,6})? ', "", value("MESSAGE") ); subst ( ' devid="[A-Z0-9]{16}" ', " ", value("MESSAGE") ); subst ( ' logid="[0-9]{10}" ', " ", value("MESSAGE") ); subst ( ' srcname="([\w-]+)\.example\.local" ', ' srcname="$1" ', value("MESSAGE") ); subst ( ' srcintfrole="[a-z]+" ', " ", value("MESSAGE") ); subst ( ' dstname="([\w-]+)\.example\.local" ', ' dstname="$1" ', value("MESSAGE") ); subst ( ' dstintfrole="[a-z]+" ', " ", value("MESSAGE") ); subst ( ' poluuid="[a-z0-9-]{36}" ', " ", value("MESSAGE") ); subst ( ' sessionid=[0-9]+ ', " ", value("MESSAGE") ); subst ( ' policytype="policy" ', " ", value("MESSAGE") ); subst ( ' srccountry="[^"]+" ', " ", value("MESSAGE") ); subst ( ' dstcountry="[^"]+" ', " ", value("MESSAGE") ); subst ( ' service="[^"]+" ', " ", value("MESSAGE") ); subst ( ' appcat="unscanned"', "", value("MESSAGE") ); subst ( ' crscore=[0-9]+ craction=[0-9]+ crlevel="[a-z]+"', "", value("MESSAGE") ); }; log { source(s_net); filter(f_fortinet_fortigate); rewrite(rewrite_fortigate); destination(d_fortinet_fortigate); };   ... View more

Hi there,  The syslog-ng configuration that you have described should erase all the other configurations or should be added into some part of the code ?  Currently my syslog-ng configuration looks like:   @version: 3.13 @include "scl.conf" # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); create_dirs(yes); keep_hostname(yes); log_fifo_size(2048); log_msg_size(8192); time_reopen(10); }; # Adding param to make syslog-ng listen on udp/514 for syslog source s_net { udp(port(514)); }; # Adding destination for local file to receive FortiGate logs destination d_fortinet_fortigate { file("/root/syslog/logs/fortinet/fortigate/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortigate.log" create_dirs(yes)); }; destination d_fortinet_fortiweb { file("/root/syslog/logs/fortinet/fortiweb/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiweb.log" create_dirs(yes)); }; destination d_fortinet_fortiauthenticator { file("/root/syslog/logs/fortinet/fortiauthenticator/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiauthenticator.log" create_dirs(yes)); }; destination d_fortinet_fortimail { file("/root/syslog/logs/fortinet/fortimail/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortimail.log" create_dirs(yes)); }; # Filter to instruct syslog-ng how to identify FortiGate syslog filter f_fortinet_fortigate { match("devid=\"FG[A-Z0-9]+\"" value("MESSAGE")); }; filter f_fortinet_fortiweb { match("device_id=FV[A-Z0-9]+" value("MESSAGE")); }; filter f_fortinet_fortiauthenticator { match("subcategory=\"Authentication\"" value("MESSAGE")); }; filter f_fortinet_fortimail { match("device_id=FE[A-Z0-9]+" value("MESSAGE")); }; # Creating the different parts together for logging log { source(s_net); filter(f_fortinet_fortigate); destination(d_fortinet_fortigate); }; log { source(s_net); filter(f_fortinet_fortiweb); destination(d_fortinet_fortiweb); }; log { source(s_net); filter(f_fortinet_fortiauthenticator); destination(d_fortinet_fortiauthenticator); }; log { source(s_net); filter(f_fortinet_fortimail); destination(d_fortinet_fortimail); }; ### # Include all config files in /etc/syslog-ng/conf.d/ ### @include "/etc/syslog-ng/conf.d/*.conf" ... View more

I have used a default syslog-ng.conf that i have found in the internet:   @version: 3.13 @include "scl.conf" # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # First, set some global options. options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); create_dirs(yes); keep_hostname(yes); log_fifo_size(2048); log_msg_size(8192); time_reopen(10); }; # Adding param to make syslog-ng listen on udp/514 for syslog source s_net { udp(port(514)); }; # Adding destination for local file to receive FortiGate logs destination d_fortinet_fortigate { file("/root/syslog/logs/fortinet/fortigate/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortigate.log" create_dirs(yes)); }; destination d_fortinet_fortiweb { file("/root/syslog/logs/fortinet/fortiweb/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiweb.log" create_dirs(yes)); }; destination d_fortinet_fortiauthenticator { file("/root/syslog/logs/fortinet/fortiauthenticator/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortiauthenticator.log" create_dirs(yes)); }; destination d_fortinet_fortimail { file("/root/syslog/logs/fortinet/fortimail/$HOST/$YEAR-$MONTH-$DAY-fortinet-fortimail.log" create_dirs(yes)); }; # Filter to instruct syslog-ng how to identify FortiGate syslog filter f_fortinet_fortigate { match("devid=\"FG[A-Z0-9]+\"" value("MESSAGE")); }; filter f_fortinet_fortiweb { match("device_id=FV[A-Z0-9]+" value("MESSAGE")); }; filter f_fortinet_fortiauthenticator { match("subcategory=\"Authentication\"" value("MESSAGE")); }; filter f_fortinet_fortimail { match("device_id=FE[A-Z0-9]+" value("MESSAGE")); }; # Creating the different parts together for logging log { source(s_net); filter(f_fortinet_fortigate); destination(d_fortinet_fortigate); }; log { source(s_net); filter(f_fortinet_fortiweb); destination(d_fortinet_fortiweb); }; log { source(s_net); filter(f_fortinet_fortiauthenticator); destination(d_fortinet_fortiauthenticator); }; log { source(s_net); filter(f_fortinet_fortimail); destination(d_fortinet_fortimail); }; ### # Include all config files in /etc/syslog-ng/conf.d/ ### @include "/etc/syslog-ng/conf.d/*.conf" ... View more

The output of the first debug is: /opt/splunkforwarder/etc/apps/search/local/inputs.conf                     [monitor:///root/syslog/logs/fortinet/fortigate/*/*.log] /opt/splunkforwarder/etc/apps/search/local/inputs.conf                     index = fortigate The output of the second debug is empty thnx  ... View more

Hi, thank you for your reply.  Sorry but i'm still confused.  In SPLUNK i can see that logs are comming under index="main" sourcetype="fortigate_traffic" ... View more

Thank you @satyenshah for sharing this. In case i cannot deploy syslog, is it possible to do the trimming directly in SPLUNK via transforms.conf ? ... View more

Thank You for your reply. So using this method it can work without adding another syslog in the middle. Is there any guide for the REGEX language that should i use for Fortinet ? Thank You  ... View more

I understand. Thank You  ... View more

In some articles in the community i have seen that they are suggesting trimming firewall logs using syslog-ng to conserve license Solved: Trimming firewall logs using syslog-ng to conserve... - Splunk Community I do not know if this is a good idea and i was looking for something less time consuming using props.conf. But i do not know if anyone had this experience before and can share some information how they have tailored the props.conf  ... View more