Getting Data In

How to forward data from an indexer to a 3rd party server

anton085
Path Finder

Hi,

I have the following setup:

3rd Party Server <---- Splunk Enterprise (Indexer):9997 <---- [Splunk Enterprise (Heavy Forwarder)] OR [Universal Forwarder]

If the forwarder is monitoring a file, for example: /var/log/syslog, how can I forward the events from only that file it from the Indexer to the 3rd party server? My conf files in the Indexer are given below, and this settings don't work:

props.conf:
[source::/var/log/syslog]
TRANSFORMS-routing=send_to_syslog

transforms.conf:
[send_to_syslog]
DEST_KEY=_SYSLOG_ROUTING
FORMAT=syslog_abc
REGEX=.

outputs.conf:
[syslog:syslog_abc]
disabled=false
server=x.x.x.x:514
timestampformat=%b %e %H:%M:%S
type=tcp

Thanks

traxxasbreaker
Communicator

I hope you found something for the actual routing in the time since you asked this, or would request to see any relevant events in your splunk.d log related to that config, but I also wanted to put a word of warning out there on TCP syslog forwarding from your indexers.

If your syslog destination is down, what will happen? Is the IP you put in there actually a VIP that will always point to an active syslog destination?

If not what I've seen happen in scenarios when a TCP syslog destination is down, Splunk continues to hold the data destined for it in it's internal queues. Over time, which is relatively short for a high volume of data the queues all fill up and eventually result in the indexer being blocked and unable to return search results. As the forwarders redirect to other indexers, they start taking the rest down too.

While I hear there's also some settings that could change the behavior of whether splunk continues to hold the data in queue while waiting on the TCP response, we've also switched to UDP syslog forwarding to prevent a problem on the destination from taking out our indexing cluster again.

0 Karma
Get Updates on the Splunk Community!

SOC4Kafka - New Kafka Connector Powered by OpenTelemetry

The new SOC4Kafka connector, built on OpenTelemetry, enables the collection of Kafka messages and forwards ...

Your Voice Matters! Help Us Shape the New Splunk Lantern Experience

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Building Momentum: Splunk Developer Program at .conf25

At Splunk, developers are at the heart of innovation. That’s why this year at .conf25, we officially launched ...