I am having some issues getting my heavy forwarder to forward events. The configuration I'm trying to achieve is as follows.
Server running universal forwarder, forward all security logs to Heavy Forwarder | Heavy Forwarder receives the events, filters them on event id and forwards them to the indexer | Logs get indexed on the indexer (in a index called test_index)
I can get the logs to go from the UF to the HF, and if I set the HF to store a local copy of forwarded events I can see the events logging on the forwarders local index. But my filtered logs from the heavy forwarder are not getting sent to the indexer. Basic example of the configuration I'm trying to get working is for a server running the UF to send event logs to the HF and filter the logon and logoff event ids 4624 and 4634 and forward these to the indexer. What am I doing wrong? 🙂
[WinEventLog:Security] index = test_index disabled = 0 start_from = oldest current_only = 1 checkpointInterval = 5
[tcpout] defaultGroup = heavyforwarder_9997 [tcpout:heavyforwarder_9997] server = heavyforwarder:9997 [tcpout-server://heavyforwarder:9997]
[WinEventLog:Security] disabled = 0 start_from = oldest current_only = 1 checkpointInterval = 5 #index = test_index
[tcpout] defaultGroup = splunkindexer_9997 indexAndForward = 1 [tcpout:splunkindexer_9997] autoLB = true server = splunkindexer:9997
[WinEventLog:Security] TRANSFORMS-security= events-null, events-filter
[events-filter] REGEX=(?msi)^EventCode=(4624|4634)\D DEST_KEY = queue FORMAT = indexQueue [events-null] REGEX = . DEST_KEY = queue FORMAT = nullQueue
Having a quick skim I think you're missing;
_TCP_ROUTING = splunkindexer_9997
From your inputs statements;
[WinEventLog:Security] disabled = 0 start_from = oldest current_only = 1 checkpointInterval = 5 _TCP_ROUTING = splunkindexer_9997 #index = test_index
Logs are still logging into the index on the intermediate forwarder but not indexing on the indexer.
I can see 40,000bytes/sec inbound from the universal forwarder and 7000bytes/sec outbound to the indexer.
I've made changes after a bit of reading to inputs.conf. Same outcome though. For props.conf and transforms.conf, is [WinEventLog:Security] correct on an intermediate forwarder? Or should it be something else like [WinEventLog:ForwardedEvents]?
[splunktcp://9997] disabled = 0 _TCP_ROUTING = splunkindexer_9997