W3C Fields With Light Forwarder - Still don't have it

[dveith]

Please advise.

Linux Splunk Server 4.1.5 Light forwarder is installed on Windows IIS web Servers Trying to get W3C Extended fields available for searching on the Splunk Server. the data is forwarded, just not with fields defined.

Windows IIS Servers have this inputs.conf

[default]
host = servername

[monitor://C:\WINNT\system32\LogFiles\W*\ex*.log]
SOURCETYPE = iis

Records also display with source types "IIS" "IIS-1" IIS-5" on the Splunk server.

What the best way to configure this to the IIS logs have their W#C Extended fields available for searching?

thanks.

[gkanapathy]

Yeah. So, the default settings in props.conf don't work well if you're using a forwarder for IIS log files, which admittedly a lot of people do (and should do).

Here's what I would do. First, SOURCETYPE should be sourcetype, i.e., lower-case. Next, on the forwarder (where the input phase occurs, reference) add this to a props.conf next to your inputs.conf:

 [iis]
 CHECK_FOR_HEADER = false

Then, on the search head, configure your fields manually in a props.conf:

 [iis]
 REPORT-iisfields = iisfields

and transforms.conf

[iisfields]
DELIMS = " "
FIELDS = date,time,csWhatever,csWhatever2,csNextField,scMoreInfo

If you have multiple different sets of fields (e.g., different servers/instances/sites log different fields), then specify a different sourcetype for them in inputs, and define different fields for it on the search head.

[dveith]

Hi, I will submit an enhancement request. And before I saw you note I got it working sending to a null queue. Thanks for your help!!

[gkanapathy]

I would also encourage you to file an enhancement request (aka P4 support ticket) on this. This is something that Splunk should fix, and if the "Getting Data In" tasks for the next version do anything at all, it should deal with this issue.

[gkanapathy]

You can mostly just ignore them, or construct your search queries to ignored them (e.g., NOT user=csUser or whatever will exclude those items) Otherwise you can do a TRANSFORMS at index time and filter out (nullQueue) anything that matches ^#

[dveith]

gkanapathy, this worked well. Except the header records make it through now too. How can I eliminate them?

[dveith]

Thank you for your excellent response. We do have different sets of fields for different web sites on the same IIS servers so we will need to specify multiple sourcetypes and fields. Thanks for that tip too.

It's things like this that still make me feel that Windows is still a second-class citizen to Splunk.

[araitz]

Why is this still painful? Amazing....