Getting Data In

How to configure a heavy forwarder to filter events based on the hostname found in IIS logs?

Path Finder

I have not yet started ingesting IIS logs from my systems. The systems have roughly 2 years of logs stored on them, that I need to ingest for IA compliance reasons. I'm going to utilize ignoreOlderThan = 0 setting to ingest all of the logs that are stored on the system.

My situation is complicated by the fact that all systems in our environment are clones, so some of the logs that are present on the systems are valid, and some of the logs that are present are remnants from the system that it was cloned from. I became aware of this when I was testing and a co-worker pointed out that Splunk had logs for my test system going back 4 months, but the system was just cloned 2 weeks ago.

For example, the following 2 logs came from the same system, which is currently named ZA70027.

2015-03-10 13:24:08 W3SVC2 ZA70043 10.1.101.116 GET /XenApp/loading.htm - 443 - 204.223.38.5 Mozilla/5.0+(compatible;+MSIE+10.0;+Windows+NT+6.1;+WOW64;+Trident/6.0) <Address Removed> 200 0 0 1957 345 1872

2015-06-15 18:32:22 W3SVC2 ZA70027 10.1.101.124 GET /XenApp/media/LoginPaneFooterMidBorderGlow.png - 443 <Account Removed>  192.156.37.34 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/7.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+InfoPath.3;+.NET4.0C;+.NET4.0E) <Address Removed>  200 0 0 549 600 46

I am already running all my logs through heavy forwarder. I would like to utilize a Regex search string in my transforms to drop unwanted logs.

Is there a way to verify that the current system's hostname exist in the IIS log before sending them to the indexer?

0 Karma
1 Solution

SplunkTrust
SplunkTrust

As far as I'm aware, there's no dynamic-placeholder-kind-of-thing you can use in transforms.conf regular expressions... however, you could build a script that frequently inserts the current hostname into your transforms.conf nullqueue filter and restarts the forwarder if the hostname changed.

View solution in original post

0 Karma

SplunkTrust
SplunkTrust

As far as I'm aware, there's no dynamic-placeholder-kind-of-thing you can use in transforms.conf regular expressions... however, you could build a script that frequently inserts the current hostname into your transforms.conf nullqueue filter and restarts the forwarder if the hostname changed.

View solution in original post

0 Karma

Path Finder

I was afraid of that.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!