Getting Data In

How to set up a universal forwarder for AS400 (iSeries) to send data to a Splunk AWS instance, or is it possible to send data via syslog?

patricktownsend
New Member

We offer a third party solution (Alliance LogAgent) that sends IBM i security events in syslog format to Splunk in real time. It works great for in-house deployments, but we have prospective customers who would like to use Splunk in the AWS cloud. I checked and I don't see a Universal Forwarder for the IBM i server platform. So a couple of questions come to mind:

1) Is it possible to send data to a Splunk AWS instance using standard syslog communications?

2) If we deployed a Windows or Linux instance of the Universal Forwarder, could we send security events from the IBM i server to the in-house instance of the Universal Forwarder, and then have it go to Splunk in AWS?

3) Is there an open source version of the Universal Forwarder?

Thanks,
Patrick

0 Karma
1 Solution

Jeremiah
Motivator

1) You can send syslog data directly to Splunk in AWS if it is a self-managed Splunk install. If your prospective customers are using Splunk Cloud (the SaaS version of Splunk) then you'll need to send the data through a forwarder first.

2) Yes you can do this, and that would be the best option (see below).

3) No there isn't an open source version, but you can extend the forwarder's capability with custom apps.

Even in our on-premise Splunk deployments, we use forwarders to collect our syslog data. A very typical deployment is to run syslog-ng on a host (or set of hosts) as remote syslog collectors. Configure all of your syslog devices to send data to these collectors. You then configure syslog-ng to write the data out to files and have the forwarder process read the files and send them to the Splunk indexer(s). This way, if communication is interrupted between your network and AWS, you will not lose any data-- the forwarder will keep a checkpoint of the last event sent and will resend from that checkpoint once connectivity is returned. You also have the ability to encrypt the connection between the forwarder and the indexer if that is required.

View solution in original post

Jeremiah
Motivator

1) You can send syslog data directly to Splunk in AWS if it is a self-managed Splunk install. If your prospective customers are using Splunk Cloud (the SaaS version of Splunk) then you'll need to send the data through a forwarder first.

2) Yes you can do this, and that would be the best option (see below).

3) No there isn't an open source version, but you can extend the forwarder's capability with custom apps.

Even in our on-premise Splunk deployments, we use forwarders to collect our syslog data. A very typical deployment is to run syslog-ng on a host (or set of hosts) as remote syslog collectors. Configure all of your syslog devices to send data to these collectors. You then configure syslog-ng to write the data out to files and have the forwarder process read the files and send them to the Splunk indexer(s). This way, if communication is interrupted between your network and AWS, you will not lose any data-- the forwarder will keep a checkpoint of the last event sent and will resend from that checkpoint once connectivity is returned. You also have the ability to encrypt the connection between the forwarder and the indexer if that is required.

patricktownsend
New Member

Thank you Jeremiah, very helpful!
Patrick

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...