Hello, recently I've added a new firewall as a source to the splunk solution at work but I can't figure why my LINE_BREAKER thing is not working. I've deployed the thing both at the heavy forwarder and the indexers but still can't make it work.
Logs are coming in like this:
Sep 19 16:02:28 host_ip date=2023-09-19 time=16:02:27 devname="fw_name_1" devid="fortigate_id_1" eventtime=1695157347491321753 tz="-0500" logid="0001000014" type="traffic" subtype="local" level="notice" vd="vdom1" srcip=xx.xx.xx.xx srcport=3465 srcintf="wan_1" srcintfrole="undefined" dstip=xx.xx.xx.xx dstport=443 dstintf="client" dstintfrole="undefined" srccountry="Netherlands" dstcountry="Peru" sessionid=1290227282 proto=6 action="close" policyid=0 policytype="local-in-policy" service="HTTPS" trandisp="noop" app="HTTPS" duration=9 sentbyte=1277 rcvdbyte=8294 sentpkt=11 rcvdpkt=12 appcat="unscanned"
Sep 19 16:02:28 host_ip date=2023-09-19 time=16:02:28 devname="fw_name_1" devid="fortigate_id_1" eventtime=1695157347381319603 tz="-0500" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="vdom2" srcip=143.137.146.130 srcport=33550 srcintf="wan_2" srcintfrole="undefined" dstip=xx.xx.xx.xx dstport=443 dstintf="3050" dstintfrole="lan" srccountry="Peru" dstcountry="United States" sessionid=1290232934 proto=6 action="close" policyid=24 policytype="policy" poluuid="12c55036-3d5b-51ee-9360-c36a034ab600" policyname="INTERNET_VDOM" service="HTTPS" trandisp="noop" duration=2 sentbyte=2370 rcvdbyte=5826 sentpkt=12 rcvdpkt=11 appcat="unscanned"
Sep 19 16:02:28 host_ip date=2023-09-19 time=16:02:28 devname="fw_name_1" devid="fortigate_id_1" eventtime=1695157347443046437 tz="-0500" logid="0000000020" type="traffic" subtype="forward" level="notice" vd="vdom2" srcip=xx.xx.xx.xx srcport=52777 srcintf="wan_2" srcintfrole="undefined" dstip=xx.xx.xx.xx dstport=443 dstintf="3050" dstintfrole="lan" srccountry="Peru" dstcountry="Peru" sessionid=1289825875 proto=6 action="accept" policyid=24 policytype="policy" poluuid="12c55036-3d5b-51ee-9360-c36a034ab600" policyname="INTERNET_VDOM" service="HTTPS" trandisp="noop" duration=500 sentbyte=1517 rcvdbyte=1172 sentpkt=8 rcvdpkt=7 appcat="unscanned" sentdelta=1517 rcvddelta=1172
Sep 19 16:02:28 host_ip date=2023-09-19 time=16:02:28 devname="fw_name_1" devid="fortigate_id_1" eventtime=1695157347481317830 tz="-0500" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="vdom2" srcip=xx.xx.xx.xx srcport=18191 srcintf="3050" srcintfrole="lan" dstip=xx.xx.xx.xx dstport=443 dstintf="wan_2" dstintfrole="undefined" srccountry="Peru" dstcountry="Peru" sessionid=1290224387 proto=6 action="timeout" policyid=21 policytype="policy" poluuid="ab285ae0-3d5a-51ee-dce1-3f4aec1e32dc" policyname="PUBLICACION_VDOM" service="HTTPS" trandisp="noop" duration=13 sentbyte=180 rcvdbyte=0 sentpkt=3 rcvdpkt=0 appcat="unscanned"
Sep 19 16:02:28 host_ip date=2023-09-19 time=16:02:27 devname="fw_name_2" devid="fortigate_id_2" eventtime=1695157346792901761 tz="-0500" logid="0000000013" type="traffic" subtype="forward" level="notice" vd="vdom3" srcip=xx.xx.xx.xx srcport=47767 srcintf="3006" srcintfrole="lan" dstip=xx.xx.xx.xx dstport=8580 dstintf="wan_2" dstintfrole="undefined" srccountry="United States" dstcountry="Peru" sessionid=3499129086 proto=6 action="timeout" policyid=18 policytype="policy" poluuid="9cba23b2-3dfa-51ee-847f-49862ff000c0" policyname="PUBLICACION_VDOM" service="tcp/8580" trandisp="noop" duration=10 sentbyte=40 rcvdbyte=0 sentpkt=1 rcvdpkt=0 appcat="unscanned" srchwvendor="Cisco" devtype="Router" mastersrcmac="xxxxxxxxxxxxxxx" srcmac="xxxxxxxxxxxxxxx" srcserver=0
And the configuration I added into props.conf is the following:
[host::host_ip]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\w{3}\s+\d{1,2}\s\d{2}\:\d{2}\:\d{2})
TIME_PREFIX = eventtime=
TIME_FORMAT = %b %d %H:%M:%S
The format is similar to the configuration applied to similar sources so I can't figure out why it isn't working. I'd appreciate any kind of insight you guys could bring.
Thanks in advance!
Hi Berfomet96,
Can you try below line breaker regex:
LINE_BREAKER = ([\r\n]+)\w{3}\s+\d{1,2}\s\d{2}:\d{2}:\d{2}
Also your TIME_PREFIX and TIME_FORMAT do not seem to match as eventtime is an epoch timestamp.