Syslog input on Splunk HF and forward to Splunk AWS instance.

[b_chris21]

Hello,

I have a Splunk ES instance on AWS. All logs are forwarded there from a Splunk HF (full forwarding - no indexing) which collects Active Directory data. Domain is accessible only via VPN.

I would like to receive inputs from syslog source (FortiGate firewalls) without installing a sysmon-ng server.

How can this happen in order to get the logs to the Cloud?

- Shall I set UDP 514 as data input port and HF will automatically forward data to Splunk ES in the Cloud via 9997? Even though I have a FortiGate addon installed on HF, while setting 514 as UDP input with syslog, there no option to specify the app's correct sourcetype.
- Can I receive on same UDP 514 port a syslog input from another source and have it properly parsed?

Thank you

[PickleRick]

As @VatsalJagani pointed out, there are some limitations to the normal Splunk's tcp or udp input. You lose network-level metadata, you cannot easily distinguish between different source types on a single input. With higher volume of data you can encounter event loss.

For many years the recommended solution was indeed to write into intermediate files and ingest those files but that's unnecessary load on i/o system. That's why it's better to have some solution (sc4s, rsyslog) receiving events from network and send them directly to HEC input.

Oh, and with network input on UF, you can't bind to low port (like the typical 514 syslog port) if you're not running forwarder as root.

[VatsalJagani · Accepted solution]

Here are the points to keep in mind:

Splunk Data Forwarding

• Data forwarding will remain the same and single (9997 port) no matter how many data sources you have.

UDP Input vs Syslog Service and File Monitoring

• To collect the FortiGate firewall you can use any UDP port on Splunk input. Generally, it's not recommended by PS. You should use the Syslog service to write the logs first into files. But if you accept the risk of using direct UDP port input on Splunk then it's okay.
• You cannot use the same UDP port to collect some other logs.

Use of Add-on and assign proper sourcetype

• Because you have to set the sourcetype as you need to use below Add-on and sourcetype="fortigate_log". Please read the details page of the Add-on carefully.
  • https://splunkbase.splunk.com/app/2846/#/details
• The Above Add-on is required to parse the data correctly.

Use of UF for Network Bandwidth

• If you will have lots of logs from the Fortigate Firewall then I would not recommend using HF. I would suggest using UF (Universal Forwarder), as UF uses so much less bandwidth than HF. This is important specifically as you are sending logs to cloud.
• If you will use UF in that case make sure to install the Add-on on the Indexers as well for proper data parsing.

I hope this helps!! Upvote would be appriciated!!!

[b_chris21]

Thanks for your detailed reply!

Can't I use something like this in order to collect inputs from 2 network devices?

[udp://123.456.789:514]
index = networking
sourcetype = cisco

[udp://123.456.890:514]
index = networking
sourcetype = fortinet

514 should listen only to one input source? Will specifying the hostIP help in order to use more than one?

Thanks!

[VatsalJagani]

Yes, you can. But if I'm dealing within the same network which is generally the case as it's UDP. So, I just use a different port which is generally easier to manage and understand for me atleast. 😊

[PickleRick]

You can create multiple inputs on different ports - each receiving a specific sourcetype but - especially in bigger environments - it quickly gets unmanageable. You end up having several dozens of ports open and getting lost in your own configuration. (I'm not sure but it might also add some performance penalty or at least uses up your resources).