ingesting syslog data

[maheshnc]

I want to ingest syslog from different devices like ESXI Hosts, firewalls (fortigate, palo alto), switches can somebody answer how can I achieve this?  I was thinking if I can forward syslog to HF and then to indexers but not aware of the full process. Please let me know if there are any other methods as well.

[PickleRick]

https://help.splunk.com/en/splunk-cloud-platform/splunk-validated-architectures/getting-data-in-forw...

[gcusello]

Hi @maheshnc ,

I usually use rsyslog on a Universal or Heavy Forwarder to receive syslogs also when Splunk is down on this machine.

The best solution is to have two (or more) machines with rsyslog receiver and a Load Balancer in front of them, so you have also HA features in your architecture and you can receive logs even if one of the receivers is down.

Then files written by rsyslog can be read by Splunk Forwarder and sent to Indexers.

In the rsyslog, you can also configure host and technology recognition.

Ciao.

Giuseppe

[richgalloway]

There are a few ways to onboard data into Splunk.

1. Install a universal forwarder on the server to send log files to Splunk
2. Have the server send syslog data to Splunk via a syslog server or Splunk Connect for Syslog
3. Use the server's API to extract data for indexing
4. Use Splunk DB Connect to pull data from the server's SQL database.
5. Have the application send data directly to Splunk using HTTP Event Collector (HEC).

[maheshnc]

Is it possible to configure the syslog device to send data to HF and then forward it to the indexers?

[kml_uvce]

@maheshnc  Yes that is possible , you can configure the syslog device to send data to HF and then forward it to the indexers.

[maheshnc]

Could you please through some light on process and configurations to be done, as I have not onboarded it before.

[gcusello]

Hi @maheshnc ,

the process is linear:

• choose a protocol (UDP or TCP) and a port to use (default 514),
• check the firewall routes between the source devices and the receiver for the defined protocol and port,
• configure rsyslog to receive syslogs and write logs in a folder using the defined protocol and port,
• configure the Splunk Forwarder (Universal or Heavy) to send logs to the Indexers (outputs.conf),
• configure the Splunk Forwarder (Universal or Heavy) to read the files containing syslogs (inputs.conf),
• define and install the add-ons to use to parse your data.

to have more information about rsyslog configuration, you can read at https://www.rsyslog.com/doc/index.html

Ciao.

Giuseppe

[PickleRick]

With a fairly modern rsyslog you can send directly to HEC input bypassing the need to create local files.

[PickleRick]

You can enable a tcp or udp source on a Splunk component. But. The other ways of handling syslogs are usually better - more robust, easier to maintain, less downtime needed for maintenance and so on.