How to configure inputs.conf to parse key value pairs in IIS 8.5 logs?

[dlofstrom]

I'm trying to parse IIS logs in Windows 2012 R2 based on the blog article:
http://blogs.splunk.com/2013/10/18/iis-logs-and-splunk-6/

From what I understand, as long as I set the sourcetype to "iis", the KV pairs should automatically be parsed as indexed extractions, but I'm not seeing them appear in my search results, nor can I specify them at search time (ie: "sourcetype=iis s-port=443" produces 0 results). Our setup is as follows:

Universal Forwarder (6.4.0)
inputs.conf:

[monitor://C:\inetpub\logs\LogFiles\W3SVC*\*.log]
disabled = 0
sourcetype = iis
queue = parsingQueue
ignoreOlderThan = 1d

Our indexing infrastructure uses an indexer cluster with a separate search server. All three systems (2 indexers, 1 search) are at Splunk 6.4.1.

[woodcock]

Do no use ignoreOlderThan = 1d for this case. Those logs should already be rotated automatically so there won't be days and weeks and months of logs in it. When using this setting, if the thing that writes to the log EVER stops writing for 24 hours or more, you will have PERMANENTLY blacklisted the log file from ever being forwarded, even when the events start flowing into the log file again. This is a VERY dangerous setting and most people do not realize how it works.

Also, make sure that you are not running your search in fast mode because this prevents search-time field extractions from being executed.

[dlofstrom]

I had never thought of that! Good to know. I was using it here to reduce indexing volume since I was constantly deleting/cleaning things while I was trying to get this to work, but we certainly use it in other areas really just because we wanted to reduce the index volume at the initial ingestion only. Going forward, it's not really necessary for us.

[dwaddle · Accepted solution]

First guess, remove queue=. You should not have to tell Splunk a specific queue for data at the input-side, like 99.99% of the time. Also, for INDEXED_EXTRACTIONS to work, data needs to go into the structuredParsing queue ... http://wiki.splunk.com/Community:HowIndexingWorks

Also, upgrade that forwarder! There's been plenty of good patches on 6.4 to remove bugs, some of which may impact your situation.

[dlofstrom]

Removed the queue=parsingQueue line and it worked like a charm! I copied that from a similar input, and assumed since it was the default anyway, that it wouldn't have made much of a difference. I guess not...

[somesoni2]

In what mode you're running your searches? Could you provide a sample raw event? If the log format is standard, they should get extracted.

[dlofstrom]

Searches are running in Smart Mode. Sample raw event:

2017-02-23 14:06:46 172.17.46.192 GET /userfiles/image/admissions/bg-content.png - 443 - 172.17.46.100 Mozilla/5.0+(iPad;+CPU+OS+9_2_1+like+Mac+OS+X)+AppleWebKit/601.1.46+(KHTML,+like+Gecko)+Mobile/13D15 - 200 0 0 93

This is a standard IIS log.

[dlofstrom]

Could this have anything to do with it?

curl -k -s https://localhost:8089/services/server/info | grep kvStore: kvStoreStatus reports "failed"