IIS log fields (How to configure)

[josephrehling]

I have been trying to figure out how to make IIS logs searchable in Splunk by IIS fields. We installed the latest version of the splunk agent and selected the IIS logs directory during the install. I went in and modified the inputs.conf in the MSICreated\Local folder as follows:

[monitor://C:\inetpub\logs\logfiles\W3SVC1]
Disabled = false
Sourcetype=iis
ignoreOlderThan = 14d
host = servername.domain.com

I can now see the IIS logs in the Spunk server, but I don't see that the fields are being properly identified. I have downloaded a couple of years work of logs and I would expect to be able to search by fields from the iis logs. For example: Shouldn't I be able to search by s-IP, since that field exists in the log?

I have already checked props.conf on the Spunk server and it appears to be right given the following entries:

[iis]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

We are running Splunk 5.0.2 on UNIX. Logs are being forwarded with the current forwarder from a Windows 2008 box.

[kmattern]

Put this in you retc\system\local\transforms.conf

[iis_fields]
DELIMS=" "
FIELDS="date","time","s-sitename","s-ip","cs-method","cs-uri-stem","cs-uri-query","s-port","cs-username","c-ip","cs-version","cs(User-Agent)","cs(Cookie)","cs(Referer)","cs-host","sc-status","sc-substatus","sc-win32-status","sc-bytes","cs-bytes","time-taken"

[kmattern]

Unless your iis logs are all different you should need only one transforms entry. I've been indexing iis for a couple of years with just this entry in my transforms.conf

[josephrehling]

So I am guessing that the FIELDS section needs to match exactly the order from the sending file in order to work? So for each ISS log, I need a transforms.conf entry that tells splunk what each field and delimiter is? If that is the answer, this doesn't seem worth the trouble.

[mikelanghorst · Accepted solution]

Looks like I was wrong above. This is likely what you'll need, I'm not finding any default extractions setup.

http://splunk-base.splunk.com/answers/24986/iis-log-fields-not-parsing

[kmattern]

in that case put this in your props.conf and you'll be good to go.

[iis-2]
rename=iis

[josephrehling]

So the final answer on this is a bit more complicated. I needed to set IIS first, then open a log and fine the header entry. I then used the header information in the IIS log to create the fields value. I also needed to change the CHECK_FOR_HEADER to false. One mistake I made is that I started forwarding logs before I understood how this works. I ingested 2GB of logs that don't match my final solution, so I would need to do a seperate field convertion for those logs if I wanted them field searchable.

[josephrehling]

I think this is close. I made these changes to my props.conf and transforms.conf... I have noticed though that no matter what I put in the client inputs.conf, the server sees the sourcetype as iis-2...

[mikelanghorst]

I'm pretty sure that true/false is case sensitive.

[mikelanghorst]

Hmm, I'd had a similar issue, but I've not indexed any IIS logs as of yet. There's a similar question that someone setup their own extractions I'll link in seperate answer

[josephrehling]

I see other examples where the case is exactly the same as what I had initially. I have changed it and restarted splunk. It appears to have no impact either way. Can you tell me what I am missing?

[josephrehling]

So are you saying it should be:

[iis]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = false
CHECK_FOR_HEADER = true

I didn't put in those lines (they were already there). Will it take a restart of splunk for those settings to take effect?