Deployment Architecture

Filtering Out Load Balancer Health Checks in IIS Logs

dlofstrom
Path Finder

I'm in the process of evaluating the Splunk for Exchange app and am having a bit of trouble with the TA for IIS (via the vanilla Universal Forwarder install). We have a lot of noise in our logs related to load balancer health checks that I'd like to filter out before it hits the indexers. I set up my props.conf and transforms.conf files as follows, but it still seems to be capturing those health checks. I've triple checked the regex using an online tool, so I'm pretty sure it's correct, but I'm wondering if anyone can pinpoint what might be going wrong, or where I might troubleshoot it further.

props.conf:
[MSWindows:2008R2:IIS]
TZ = GMT
SHOULD_LINEMERGE = false
CHECK_FOR_HEADER = false
TRANSFORMS-comments = ignore_comments
TRANSFORMS-balancer = ignore_lbchecks

transforms.conf
[ignore_comments]
REGEX = ^#.*
DEST_KEY = queue
FORMAT = nullQueue

[ignore_lbchecks]
REGEX = ^.\sKEMP\S\s.*
DEST_KEY = queue
FORMAT = nullQueue

Sample log record:
2013-03-02 00:00:00 192.168.40.224 HEAD /owa - 443 - 192.168.40.222 KEMP+1.0 - - 301 0 0 245 81 0

Tags (1)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

There is not space after KEMP in your sample.
You also can try with a smaller regex, no need to try to match the full line.

[ignore_lbchecks]
REGEX =\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\sKEMP
DEST_KEY = queue
FORMAT = nullQueue

To verify, try this search in splunk

sourcetype=iis KEMP | regex _raw="\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\sKEMP"

also you can use a single transforms line in props.conf

TRANSFORMS-filternullQueue = ignore_comments, ignore_lbchecks

View solution in original post

0 Karma

yannK
Splunk Employee
Splunk Employee

There is not space after KEMP in your sample.
You also can try with a smaller regex, no need to try to match the full line.

[ignore_lbchecks]
REGEX =\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\sKEMP
DEST_KEY = queue
FORMAT = nullQueue

To verify, try this search in splunk

sourcetype=iis KEMP | regex _raw="\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\sKEMP"

also you can use a single transforms line in props.conf

TRANSFORMS-filternullQueue = ignore_comments, ignore_lbchecks

View solution in original post

0 Karma

dlofstrom
Path Finder

That was it! Thanks for clarifying.

0 Karma

yannK
Splunk Employee
Splunk Employee

That's why

To clarify the setup, this config is running on the Universal Forwarder on one of our Exchange servers, under the TA folder's 'local' directory. The indexer as far as I know shouldn't need these configs as it's my understanding the parsing is done at the forwarder, correct?

The parsing /filtering occurs on the indexers (or heavy forwarder) only.
please move your props/transforms to the indexer.

0 Karma

dlofstrom
Path Finder

I must still be doing something wrong because it still captures the load balancer lines.

I updated the regex after confirming it via search, then updated TRANSFORMS to a single comma separated line as suggested above.

To clarify the setup, this config is running on the Universal Forwarder on one of our Exchange servers, under the TA folder's 'local' directory. The indexer as far as I know shouldn't need these configs as it's my understanding the parsing is done at the forwarder, correct?

0 Karma

dlofstrom
Path Finder

Here's the corrected props.conf/transforms.conf code. The style I chose stripped out the backslashes and for some reason captcha keeps failing whenever I try to edit it...

props.conf

[MSWindows:2008R2:IIS]
TZ = GMT
SHOULD_LINEMERGE = false
CHECK_FOR_HEADER = false
TRANSFORMS-comments = ignore_comments
TRANSFORMS-balancer = ignore_lbchecks

transforms.conf
[ignore_comments]
REGEX = ^#.*
DEST_KEY = queue
FORMAT = nullQueue

[ignore_lbchecks]
REGEX = ^.*\sKEMP\S*\s.*
DEST_KEY = queue
FORMAT = nullQueue
0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.