I am trying to configure a heavy forwarder to route all of data to SyslogNG while route some data to null queue.
I need my data flow as below:
For Data Archive: HF -> all of data -> SyslogNG
For Daily Search: HF -> NullQueue -> Indexer
My issue is, when logs go to nullqueue, they do not go to syslogNG at all.
Is there any way to send to syslog while not indexing?
Also, I can not use these in inputs.conf:
SYSLOGROUTING = primarySyslogs
_TCPROUTING = somethingThatDoesntExistInOutputsConf
The reason is I use checkpoint lea app to get data, this app do not need to config regular inputs.conf.
Here is what I configured in props.conf, transforms.conf:
TRANSFORMS-route = RouteToNG, RouteToNullQueue
REGEX = .
DESTKEY = _SYSLOGROUTING
FORMAT = syslogNG
REGEX = action=accept
DEST_KEY = queue
FORMAT = nullQueue
defaultGroup = default-autolb-group
disabled = false
server = indexer:9997
server = x.x.x.x:514
I'm curious as to why you have your HF in front of syslog-ng. The more common configuration is to write to syslog first (letting it filter as it needs to) and let a forwarder to pick up the events from there.
It is not possible to route events anywhere after nullQueue. nullQueue is a dead end - any events sent there are discarded.
I know that this isn't a direct solution to your problem but a workaround: did you explored the choice to send all data to SyslogNG and Indexer and then, on Indexer, filter them?