I am currently passing all logs through a Heavy Forwarder so I can filter out "noisy" logs before they are indexed. I am successfully filtering 4 other items including Windows Logs, SYSLOG, and Windows WMI data. Now I am trying to filter IIS logs, and am running into trouble.
I believe my props.conf and transforms.conf are setup correctly. Can someone help me figure out where this filter is failing?
Props.conf: [iis] TRANSFORMS-null2=IIS_Remove_F5_Health_Checks
transforms.conf: [IIS_Remove_F5_Health_Checks] REGEX = . DEST_KEY = queue FORMAT = nullQueu
I realize my REGEX is going to filter EVERYTHING. I had a more specific REGEX in there previously, but I replaced it with the " REGEX = ." as part of my troubleshooting process. The original REGEX was supposed to filter out any logs with a 10.1.100.8 or 10.1.100.9 IP address, that REGEX was:
The sourcetype for my ISS logs is "iis" (all lower case). In my mind it looks like all iis logs should be passing processed by the transform, but it looks like that is not the case. How can I troubleshoot this?
Thanks for the help!
Are your IIS logs at the forwarder using INDEXED_EXTRACTIONS? If so, data arrives at the heavy "cooked" already. Configure the forwarder to nullQueue it. Seems wrong, I know. But..
The host that is forwarding the iis logs is using the default props.conf, which includes:
INDEXED_EXTRACTIONS = w3c
I do want INDEXED_EXTRACTIONS TO be active as that helps the logs be "Useful". If I disabled INDEXED_EXTRACTIONS on each of my host would the Heavy Forwarder be able to provide that function?