I have installed a universal forwarder to read logs from syslog server and forward them to heavy forwarder. I have kiwi syslog server to receive logs from all syslog based data sources and had planned to configure multiple UDP ports for ease of sourcetype categorisation. However, I realised it only supports 1 udp port at a time.
Can anyone please advise what can be done in this case ?
Are you using the free or full version of Kiwi? The full version should be able to take everything on a single UDP port, then use the "AutoSplit" feature, by hostname for example, and have them write out to their own directories. The UF can monitor these individually, so you can sourcetype them properly and use a segment in path to pick up the hostnames, and then send the data on to the HF.
Are you using the free or full version of Kiwi? The full version should be able to take everything on a single UDP port, then use the "AutoSplit" feature, by hostname for example, and have them write out to their own directories. The UF can monitor these individually, so you can sourcetype them properly and use a segment in path to pick up the hostnames, and then send the data on to the HF.
I am using a full version of Kiwi. Thanks for the suggestion. It has helped to deal with the issue of multiple type of logs on one port.
If you absolutely must stick with windows, there are quite a few options. For instance, here's a list of nearly a dozen free syslog servers. I find it interesting that all syslog servers for windows seem to come with some sort of a UI to "display" the data, which isn't a feature you need. Still, any one of those should work - given that you check if they support multiple UDP ports.
If you have more choices, a virtual machine running Ubuntu/CentOS with syslog-ng would also work. I've done decent enough syslog receiving on 1 GB of RAM and 1 CPU though obviously your mileage may vary. For the configuration, I believe you simply add multiple source lines, as per syslog-ng's docs. I've done it before and it seemed relatively straightforward. I DO believe you have to use a fairly current version of syslog-ng, like later in the 3.x series.
If I may ask - why send data from a UF to an HF instead of just right into your indexer?
Happy Splunking,
Rich
Hi @rich7177,
We need the HF for data filtering and dbconnect app.
I checked out each syslog server, however, none of them support multiple UDP ports. Hence, as an alternative to solution to this, I have decided to change the architecture by having all logs sent to the Heavy forwarders instead of syslog server and from there, forward logs to syslog server as well, in addition to the Indexer. That way, I can reduce the risk of data loss.
Please suggest if there could be any drawbacks for this method ?
Thanks.