I have a Fortigate firewall that was configured to send UDP logs, lately, I have configured it to send TCP logs instead of UDP, then I have started to see something wrong with the way the logs are received, I have noticed that the logs are being cut in random locations within the single log and continue writing the rest of the log after adding a new line and a time stamp.
The above screenshot is just one example, it happens regularly.
I have tried to modify the syslog-ng.conf configuration file, in the options to be specific:
keep_timestamp (yes); ---> keep_timestamp (no);
log_msg_size(65536); ---> log_msg_size(131072);
But the issue still persists!
Can anybody please help me with this?
Hi @scelikok
I have already set this setting on the HF the minute I converted the firewall to send TCP instead of UDP.
Please check the below screenshot from the props.conf file on the HF:
@PickleRick
Just adding more information here:
When the firewall was sending UPD, the logs were separated by a new line and all worked just fine without modifying anything in the props.conf
Here is a sample of the firewall logs that were sent using UPD:
Ok
1. oops my bad 😂😂
2. thanks for the information 👍
3. I have taken a chunk of the logs from the written file and put it in regex101.com and tested the LINE_BREAKER regex and it worked as expected, please see below:
Also, my setup is as follows:
I have 2 Splunk machines, one acts as HF & DS, and the other acts as IDX & SH, they are all in the same subnet including the firewall.
On the HF/DS machine I have installed syslog-ng on it to receive the syslog logs on it, the syslog-ng writes the logs in a directory on the machine, and I have inputs.conf file that monitors this directory with its contents, and outputs.conf file to forward the logs to the SH/IDX machine.
Also, on both Splunk machines, they have the same "Fortinet FortiGate Add-On for Splunk" with the same configurations.
I have done a tcpdump and put the pcap file in Wireshark and did not see any issues in the logs.
In that same period of time i have done a "tail -f" command to the written .log file and found cuts in the logs same as the picture in my original question.
So in this case I'm almost certain that this issue is caused by the syslog-ng itself.
Moreover, I have worked on Fortinet FortiGate Add-On for Splunk document in the "details" page that recommends to modifies the props.conf as below:
"Note: the UDP port, 514 in this example should be opened in firewall for logs to pass through. If you choose TCP input and on FortiGate use "reliable"(tcp) mode for syslog setting, you will need to add the following in local/props.conf because tcp tranported syslog will have xxx <yyy> header as line indicator.(8514 below is an example of TCP port, you can choose your own. There is no timestamp header like UDP so you can specify the timestamp field in the fortigate log, in our case the precision is in nanoseconds so the time format is %s%9N. If your FOS version has time stamp in different precision"
[source::tcp:8514]
SHOULD_LINEMERGE = false
LINE_BREAKER = (\d{2,3}\s+<\d{2,3}>)
TIME_PREFIX = eventtime=
TIME_FORMAT = %s%9N
The above note can be found here:
https://splunkbase.splunk.com/app/2846
And regarding the receiving of the logs, The firewall is in the same subnet that the syslog-ng machine is in.
Regarding the link you have provided, I will look into it and get back to you thanks.
More on that, when the firewall was sending UPD, the logs were separated by a new line and all worked just fine without modifying anything in the props.conf
Here is a sample of the firewall logs that were sent using UPD:
