Getting Data In

How to route a subset of data to syslog?


I was hoping to get some help with routing a subset of data to a syslog server.

The goal is to send a handful of windows eventlog id's to a third party syslog server so they can use them for correlation. We want them indexed in our cluster as well. We know we have to use a hf to route to syslog which isn't a problem. But we also don't want to send those events through the hf to get to our indexer cluster, because it isn't as robust.

So the plan is to send the all Win Security events to both our indexer cluster and the heavy forwarder (that does work). Then on the heavy forwarder, send all events except the ones we want to the nullQueue. And finally route the desired events to the syslog server. We would also like for our heavy forwarder itself to log its data to the indexer cluster.

I've tried many different configs, but there seems to be a general theme. If I try to send any of the security events to the null queue, they all go to the null queue and nothing goes to syslog. If I try to just route the specific events to syslog, they do go as expected. But the other events still get forwarded to our indexer cluster - so we have duplicate events there - one from the uf and one from the hf.

Any idea what I'm doing wrong? Here are conf files on the heavy forwarder.


# Ensure indexer received data
# Use 7MB buffer
maxQueueSize = 7MB
useACK = true

# Switch indexers every 30s
forceTimebasedAutoLB = true

# Deploy org_all_forwarder_outputs to search heads as well, to dictate
# where the data should be sent.
index = false

disabled = false
server = <bunch of servers here>

server = <the syslog server here>:514
timestampformat = %b %e %H:%M:%S


#TRANSFORMS-2_null_routing = route_to_null_queue
#TRANSFORMS-1_syslog_routing = route_to_palo_alto
TRANSFORMS-syslog_routing = route_to_null_queue,route_to_palo_alto

TRANSFORMS-indexer_routing = route_to_indexers


REGEX = EventCode=(4624|5447|4957)
FORMAT = palo_alto

DEST_KEY = queue
FORMAT = nullQueue

FORMAT = indexers

If I get rid of the tcpout stuff and don't try to send anything to the null queue, it works well enough. Just the events I want go to syslog, and the others are just lost I guess. But then I lose the local logs for the heavy forwarder as well.

This is for a POC right now, so not a huge deal, but just really would like to know what I'm missing.

0 Karma


Transforms are performed in order listed. So, you have null queue first. So everything goes to null queue. Try this:

 TRANSFORMS-syslog_routing = route_to_palo_alto,route_to_null_queue

Once you try that, and if it doesn't work, comment below, and we will dive deeper.

0 Karma


That was one of the configs I tried, but I can give it another shot. My understanding, though, is that we should set everything to go to the null queue, then right after that change those few events to go to syslog instead. If we do the null queue transformation last and are using a "." regex, then everything would end up going.

Meaning, all of the transformations get applied in order and then whatever the final state of a message is gets processed.

0 Karma


You can also find me on IRC (#splunk on for additional help without all the emails.

0 Karma
Get Updates on the Splunk Community!

Observability Highlights | November 2022 Newsletter

 November 2022Observability CloudEnd Of Support Extension for SignalFx Smart AgentSplunk is extending the End ...

Avoid Certificate Expiry Issues in Splunk Enterprise with Certificate Assist

This blog post is part 2 of 4 of a series on Splunk Assist. Click the links below to see the other ...

Using Machine Learning for Hunting Security Threats

REGISTER NOW Seeing the exponential hike in global cyber threat spectrum, organizations are now striving more ...