I've been reading into how to filter incoming events between the Parsing and Indexing stages of our data pipeline and found a Splunk doc (https://docs.splunk.com/Documentation/SplunkCloud/8.0.2001/Forwarding/Routeandfilterdatad) which suggests it should be possible to do this by just adding a props.conf and transforms.conf at the indexer:
I have been through and added in what I thought would be valid entries in the conf files, as follows:
props.conf
[source::IIS_Exchange]
TRANSFORMS-set= setnull,ExchangeParsing
transforms.conf
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue
[ExchangeParsing]
REGEX = /(-\s401\s1\s1[39][02][69])/
DEST_KEY = queue
FORMAT = indexQueue
The regex should be monitoring for HTTP 401 errors with a sub-status of 1 and a pair of event error codes we're monitoring for (1326 & 1909), for example:
401 1 1909 9
401 1 1326 67
I've tested the regex in both Notepad++ and on RegExr.com and it looks like it should work.
We use a Universal Forwarder to send the data into the indexer, which is just trying to monitor standard IIS logs on a load balanced server pair; the inputs.conf for the relevant source is as follows:
[monitor://G:\inetpub\logs\LogFiles\W3SVC1*.log]
index = mar
source_type = iis
disabled = false
recursive = true
source = IIS_Exchange
I've restarted Splunk on the indexer for this to take effect but nothing appears to be happening. I had expected that if the 2nd stanza in the transforms.conf was incorrect we'd get no data at all from the source, as it would all initially be filtered to the nullQueue. This suggests to me that the transform isn't applying to the incoming data at all but I cannot fathom why.
Can anybody please tell me what I'm doing wrong?
... View more
