Getting Data In

How to configure a heavy forwarder to filter events based on the hostname found in IIS logs?

k2skaterii
Path Finder

I have not yet started ingesting IIS logs from my systems. The systems have roughly 2 years of logs stored on them, that I need to ingest for IA compliance reasons. I'm going to utilize ignoreOlderThan = 0 setting to ingest all of the logs that are stored on the system.

My situation is complicated by the fact that all systems in our environment are clones, so some of the logs that are present on the systems are valid, and some of the logs that are present are remnants from the system that it was cloned from. I became aware of this when I was testing and a co-worker pointed out that Splunk had logs for my test system going back 4 months, but the system was just cloned 2 weeks ago.

For example, the following 2 logs came from the same system, which is currently named ZA70027.

2015-03-10 13:24:08 W3SVC2 ZA70043 10.1.101.116 GET /XenApp/loading.htm - 443 - 204.223.38.5 Mozilla/5.0+(compatible;+MSIE+10.0;+Windows+NT+6.1;+WOW64;+Trident/6.0) <Address Removed> 200 0 0 1957 345 1872

2015-06-15 18:32:22 W3SVC2 ZA70027 10.1.101.124 GET /XenApp/media/LoginPaneFooterMidBorderGlow.png - 443 <Account Removed>  192.156.37.34 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.1;+WOW64;+Trident/7.0;+SLCC2;+.NET+CLR+2.0.50727;+.NET+CLR+3.5.30729;+.NET+CLR+3.0.30729;+Media+Center+PC+6.0;+InfoPath.3;+.NET4.0C;+.NET4.0E) <Address Removed>  200 0 0 549 600 46

I am already running all my logs through heavy forwarder. I would like to utilize a Regex search string in my transforms to drop unwanted logs.

Is there a way to verify that the current system's hostname exist in the IIS log before sending them to the indexer?

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

As far as I'm aware, there's no dynamic-placeholder-kind-of-thing you can use in transforms.conf regular expressions... however, you could build a script that frequently inserts the current hostname into your transforms.conf nullqueue filter and restarts the forwarder if the hostname changed.

View solution in original post

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

As far as I'm aware, there's no dynamic-placeholder-kind-of-thing you can use in transforms.conf regular expressions... however, you could build a script that frequently inserts the current hostname into your transforms.conf nullqueue filter and restarts the forwarder if the hostname changed.

0 Karma

k2skaterii
Path Finder

I was afraid of that.

0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...