Why aren't transforms working when attempting to not index keepalive data from source or source type?

[wgawhh5hbnht]

Source type is being set in inputs.conf via
/deployment-apps/Splunk_TA_microsoft-iis/local/inputs.conf
contents of inputs.conf:

    [monitor://C:\inetpub\logs\AdvancedLogs\*.log]
    index = winiislog 
    disabled = false
    #sourcetype = ms:iis:default
    sourcetype = iis

On the indexers, I have two different entries, 1 by source type & 1 by source, in props
contents of props.conf:

[iis]
TRANSFORMS-null = setnull
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N

[source::C:\\inetpub\\...]
TRANSFORMS-null = setnull

contents of transforms.conf:

[ise_setnull]
REGEX = (^Compiled\s|^Copyright\s|^Technical\sSupport:)
DEST_KEY = queue
FORMAT = nullQueue

[setnull]
REGEX = keepalive
DEST_KEY = queue
FORMAT = nullQueue

I've pushed the cluster-bundle and did a rolling restart on the indexers, but I'm still getting keepalive data.

Any ideas?

[gcusello]

Hi wgawhh5hbnht,
some additional information:
I don't understand your regex, what do you want to exclude?
parenthesis and beginning of string should be used in different way, try with

^((Compiled\s)|(Copyright\s)|(Technical\sSupport:))

at the same time the source [source::C:\inetpub\...] should be used without double backslash

[source::C:\inetpub\...]

Bye.
Giuseppe

[martin_mueller · Accepted solution]

Assuming you've got a default-configured Windows Universal Forwarder it will already cook the data on the UF for sourcetype iis. Run splunk btool --debug props list iis on the UF to confirm, look for something like INDEXED_EXTRACTIONS = w3c. The indexers won't re-cook already cooked data, so your transforms aren't being touched.

Your best option is to copy the nullqueueing to the UFs. I'd keep a copy on the indexers in case you get uncooked data in the future for some reason.

See https://wiki.splunk.com/Community:HowIndexingWorks for some background on what happens where.

[wgawhh5hbnht]

Thank you martin_mueller, that worked!

For those that want more information as to why you would need the props & transforms on the UF instead of the indexers:
https://answers.splunk.com/answers/268335/why-is-the-sourcetype-specified-in-inputsconf-on-t.html

[wgawhh5hbnht]

not shown, but I've tried both entries in the transforms.conf of [ise_setnull] & [setnull], neither work.