All Apps and Add-ons

Splunk Add-on for MS IIS and INDEXED_EXTRACTIONS

dschmidt_cfi
Path Finder

I am having an issue ever since we deployed Splunk Add-on for Microsoft IIS (Splunk_TA_microsoft-iis) and tried to configure it to work in INDEXED_EXTRACTIONS mode. In addition to getting duplicated data—it is apparently not parsing everything correctly, as I end up with many garbage fields.

Our environment is v.6.6.2 across the board. I am using Universal Forwarders (UF) on the IIS servers themselves forwarding via tcpout:syslog_group to the Heavy Forwarder (HF) where I would like to do the INDEX_EXTRACTIONS and then be forwarded to the clustered indexers via tcpout:indexer_cluster_01.

I have been through all of the inputs.conf, props.conf, transforms.conf and outputs.conf for the various configurations I have tried. The sourcetype=ms:iis:auto / index=web across the board. I have configured it with just the inputs / outputs on the IIS server’s UF and currently added the props.conf to the mix with the INDEXED_EXTRACTIONS = w3c line removed. All variations have an EVAL in the props like EVAL-extraction_time = “UF / HF / SH” hoping to see where the extractions are taking place.

The HF has just the props and transforms plus the outputs to the indexer cluster. The full TA has been added to Search Heads where the searches will take place. As soon as I did that—it is showing the extractions are taking place on SH which could explain the duplicate data if I am getting both indexed and raw data sent; however, after days of investigation I cannot find that possibility in any of my configurations.

The configuration I thought would work is:
UF: inputs / outputs to HF
HF: props / transforms / outputs to indexer cluster
SH: TA on those where searches will be held.

Any help would be appreciated.

0 Karma
1 Solution

micahkemp
Champion

The IIS Addon has to be installed on the UF to make use of indexed extractions. From the documentation:

If you use a universal forwarder for data collection, install the add-on on both your universal forwarder and indexer.
The forwarder needs to be installed directly on the Microsoft IIS server for directory monitoring. As an alternative, the Microsoft IIS log files can be copied or shared to the machine where the forwarder is installed

You are not alone in your confusion. I inquired about this to my .conf lab assistant and he asked the docs team to clarify it to the above statement.

View solution in original post

micahkemp
Champion

The IIS Addon has to be installed on the UF to make use of indexed extractions. From the documentation:

If you use a universal forwarder for data collection, install the add-on on both your universal forwarder and indexer.
The forwarder needs to be installed directly on the Microsoft IIS server for directory monitoring. As an alternative, the Microsoft IIS log files can be copied or shared to the machine where the forwarder is installed

You are not alone in your confusion. I inquired about this to my .conf lab assistant and he asked the docs team to clarify it to the above statement.

dschmidt_cfi
Path Finder

I guess one of my options (which I do not like) is to do the indexed extractions on the UF (losing out on the transforms?) and pass through the HF onto the indexers where I would have to install the TA as well? Not the option I want but if need be--I can do and have probably have done over the past couple of months.

I do not believe the UF uses the transforms.conf which I use to filter out things like "health checks" from our F5s'

0 Karma

dschmidt_cfi
Path Finder

Today I have accepted this as the proper way to go and will "adjust" my system to reflect that. Now off to help my Windows Administrator work out a PowerShell script to make all of the headers for the IIS servers identical across all environments.

0 Karma

micahkemp
Champion

If you use the indexed extraction method (which is what requires the TA be installed on the UF) you don't have to have headers consistent headers. ms:iis:auto, which is the indexed extractions version, handles the logs based on the headers present in the file. ms: iis:default requires you to define the headers in your props.

References:

Selecting sourcetype:
http://docs.splunk.com/Documentation/AddOns/released/MSIIS/Setupaddon

Additional steps for ms:iis:default:
http://docs.splunk.com/Documentation/AddOns/released/MSIIS/Setupaddon#Perform_additional_steps_for_s...

0 Karma

dschmidt_cfi
Path Finder

I understand that but being a stickler for detail and uniformity we have made the script to alter all of them already and will implement it through the different environments. Then the documentation will be clean and straightforward going forward.

Even new ones we clone will be setup like that and no worries going forward if something changes in the future which always seems to happen. I do the same in my *nix environment. Guess I am a little retentive.

0 Karma

dschmidt_cfi
Path Finder

On driving home--I think I may have the answer I was looking for. I read and re-read that paragraph several times before asking my original question; nevertheless, I believe if I just put a standard monitor on those log files and forward them to the actual TA on the HF it might work. My version of "sharing" to the machine where the forwarder (HF) is installed. Will know shortly.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...