IIS Log Files parsing and Removing Load Balance Health Check

felipemn — Tue, 29 Sep 2020 15:59:49 GMT

I,m using the new 7.0.0 version of Splunk at my distributed installation (Indexer,Search Head) and i´m trying to parse iis logs from a Windows Server 2016.
The parsing is working but i´ve tried to avoid some noise (Probe validation from Load Balancer) using "nullqueue" but somehow, that it´s not working.
The noisy probe logs still is coming...

Here we go:

Part of of the IIS log file:

Software: Microsoft Internet Information Services 10.0

Version: 1.0

Date: 2017-09-30 18:22:33

Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) cs-host sc-status sc-substatus sc-win32-status time-taken

2017-09-30 18:22:33 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 718
2017-09-30 18:22:38 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 15
2017-09-30 18:22:43 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 15
2017-09-30 18:22:48 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 15
2017-09-30 18:22:53 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 15
2017-09-30 18:22:58 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 15
2017-09-30 18:23:03 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 0
2017-09-30 18:23:08 W3SVC6 10.50.10.40 GET / - 25002 - 168.63.129.16 Load+Balancer+Agent - 100.76.216.215:25002 200 0 0 15*

*inputs.conf (at C:\Program Files\SplunkUniversalForwarder\etc\system\local) Universal Forwarder *
[monitor://C:\Logs\IIS\W3SV**.log]
index = private_backend
sourcetype = iis
disabled = false
ignoreOlderThan = 0d

*/opt/splunk/etc/system/local/props.conf (at the Indexer server) *
[iis]
TRANSFORMS-null=remove_log_probe

*/opt/splunk/etc/system/local/transforms.conf (at the Indexer server) *
[remove_log_probe]
REGEX=Load\SBalancer\SAgent
DEST_KEY=queue
FORMAT=nullQueue

I´m definetily missing something (maybe silly rsrsr). Can, please, somebody help?

Re: IIS Log Files parsing and Removing Load Balance Health Check

gcusello — Sun, 01 Oct 2017 15:45:34 GMT

Hi felipemn,
I'm not sure to have understood your need: do you want to discard events where there is Load+Balancer+Agent ?
If this is your need your regex is correct, also if I'd use Load\+Balancer\+Agent
Anyway, as you can see in http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad , I think that you have to modify:
props.conf

[iis]
TRANSFORMS-null=set_index,remove_log_probe

transforms.conf

[remove_log_probe]
REGEX = Load\+Balancer\+Agent
DEST_KEY = queue
FORMAT = nullQueue

[set_index]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

Bye.
Giuseppe

Re: IIS Log Files parsing and Removing Load Balance Health Check

felipemn — Tue, 03 Oct 2017 14:22:54 GMT

Hi Giuseppe

Thanks for the help. Unfortunatelly it didn´t work yet.
Is there any way to debug the process of parsing and check whats going on?

topic Re: IIS Log Files parsing and Removing Load Balance Health Check in Getting Data In

IIS Log Files parsing and Removing Load Balance Health Check

Version: 1.0

Date: 2017-09-30 18:22:33

Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Referer) cs-host sc-status sc-substatus sc-win32-status time-taken

Re: IIS Log Files parsing and Removing Load Balance Health Check

Re: IIS Log Files parsing and Removing Load Balance Health Check