Line-breaker not working on FortiGate logs forwarded via syslog

[frapei]

Hi all,

I was hoping someone might be able to help me on how to set this:

I'm consuming logs from Fortigate via Syslog (log sample below for reference), but this pattern doesn't work for event breaks.

601 <189>date=2021-01-29 time=01:13:54 devname="xxxSEC" devid="FG101" logid="0101037141" type="event" subtype="vpn" level="notice" vd="root" eventtime=1611893634651699002 tz="-0300" logdesc="IPsec tunnel statistics" msg="IPsec tunnel statistics" action="tunnel-stats" remip=xxx.xxx.xxx.xxx locip=xxx.xxx.xxx.xxx remport=500 locport=500 outintf="wan1" cookies="xxx" user="N/A" group="N/A" xauthuser="N/A" xauthgroup="N/A" assignip=N/A vpntunnel="VPN-xxx" tunnelip=N/A tunnelid=0 tunneltype="ipsec" duration=4823936 sentbyte=0 rcvdbyte=120 nextstat=600607 <189>date=2021-01-29 time=01:13:54 devname="xxxSEC" devid="FG101" logid="0101037141" type="event" subtype="vpn" level="notice" vd="root" eventtime=1611893634651791593 tz="-0300" logdesc="IPsec tunnel statistics" msg="IPsec tunnel statistics" action="tunnel-stats" remip=xxx.xxx.xxx.xxx locip=xxx.xxx.xxx.xxx remport=500 locport=500 outintf="wan1" cookies="xxx" user="N/A" group="N/A" xauthuser="N/A" xauthgroup="N/A" assignip=N/A vpntunnel="Soc_xxx" tunnelip=N/A tunnelid=0 tunneltype="ipsec" duration=4823936 sentbyte=13785 rcvdbyte=368 nextstat=600597 601 <189>date=2021-01-29 time=01:13:54 devname="xxxSEC" devid="FG101" logid="0101037141" type="event" subtype="vpn" level="notice" vd="root" eventtime=1611893634651699002 tz="-0300" logdesc="IPsec tunnel statistics" msg="IPsec tunnel statistics" action="tunnel-stats" remip=xxx.xxx.xxx.xxx locip=xxx.xxx.xxx.xxx remport=500 locport=500 outintf="wan1" cookies="xxx" user="N/A" group="N/A" xauthuser="N/A" xauthgroup="N/A" assignip=N/A vpntunnel="VPN-xxx" tunnelip=N/A tunnelid=0 tunneltype="ipsec" duration=4823936 sentbyte=0 rcvdbyte=120 nextstat=600607 <189>date=2021-01-29 time=01:13:54 devname="xxxSEC" devid="FG101" logid="0101037141" type="event" subtype="vpn" level="notice" vd="root" eventtime=1611893634651791593 tz="-0300" logdesc="IPsec tunnel statistics" msg="IPsec tunnel statistics" action="tunnel-stats" remip=xxx.xxx.xxx.xxx locip=xxx.xxx.xxx.xxx remport=500 locport=500 outintf="wan1" cookies="xxx" user="N/A" group="N/A" xauthuser="N/A" xauthgroup="N/A" assignip=N/A vpntunnel="Soc_xxx" tunnelip=N/A tunnelid=0 tunneltype="ipsec" duration=4823936 sentbyte=13785 rcvdbyte=368 nextstat=600597

As you can see from the sample above, each event starts with <189> and ends with nextstat=xxx

The rest of the patterns seem to work fine:

285 <190>date=2021-01-29 time=05:01:06 devname="xxxSEC" devid="FG101" logid="0100020027" type="event" subtype="system" level="information" vd="root" eventtime=1611907266834995102 tz="-0300" logdesc="Outdated report files deleted" msg="Delete 1 old report files"279 <190>date=2021-01-29 time=05:01:06 devname="xxxSEC" devid="FG101" logid="0100020027" type="event" subtype="system" level="information" vd="root" eventtime=1611907266835276157 tz="-0300" logdesc="Outdated report files deleted" msg="Delete 4 old report files"

615 <189>date=2021-01-29 time=05:00:45 devname="xxxSEC" devid="FG101" logid="0100040704" type="event" subtype="system" level="notice" vd="root" eventtime=1611907246008443458 tz="-0300" logdesc="System performance statistics" action="perf-stats" cpu=0 mem=39 totalsession=48 disk=1 bandwidth="22/7595" setuprate=1 disklograte=0 fazlograte=0 freediskstorage=426665 sysuptime=4837848 waninfo="name=wan1,bytes=1709688/171114431,packets=7001/1473776;name=wan2,bytes=15678/404589561,packets=167/5559353;" msg="Performance statistics: average CPU: 0, memory: 39, concurrent sessions: 48, setup-rate: 1"

289 <190>date=2021-01-29 time=04:58:45 devname="xxxSEC" devid="FG101" logid="0100026003" type="event" subtype="system" level="information" vd="root" eventtime=1611907125775362364 tz="-0300" logdesc="DHCP statistics" interface="mgmt" total=101 used=0 msg="DHCP statistics"

Any help would be greatly appreciated

[asridhara]

Add LINE_BREAKER in your props.conf at <SPLUNK_HOME>/etc/apps/Splunk_TA_fortinet_fortigate/default under fgt_log

----------------------------------------------------------------------------

LINE_BREAKER = (\d+)?\s*\<\d{2,3}\>date

----------------------------------------------------------------------------

[gbeatty]

@frapei It looks like the issue is that some events start with a 3 digit code and some don't.  If that isn't due to the parser, and the regex from @asridhara does not work, you need to address those cases.