IIS logs and v4.2.4

[appmandan]

I have set up a universal forwarder to forward IIS logs from C:\inetpub\logs\LogFiles\W3SVC7 to my splunk server on port 9997. I'm getting the logs fine, but I'm not getting the fields in the header...

I've read a lot about changing the props.conf file to tell splunk to pull the fields out of the header, but I'm assuming props.conf is no longer used in v4.2 because I do not have a props.conf file in ..\etc\apps\search\local. The lastest information I can find pertains to Splunk v4.1.1

What do I need to do to configure Splunk to pull the headers from the log file?

Thanks!

[tgow · Accepted solution]

If you want you can just specify the directory once in the inputs.conf and not have to worry about specifying each directory, ie:

[monitor://C:\inetpub\logs\logfiles]
sourcetype=iis
ignoreOlderThan = 14d
host = compdev2

Sometimes Splunk recognizes the header and sometimes it doesn't. I would keep your props.conf file the same but you will need to add the field definitions in the transforms.conf. Also, if the fields in the iis logs are delimited by a space then you can make this even easier.

props.conf

[iis*] 
pulldown_type=true
MAXTIMESTAMPLOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER
REPORT - iis2 = iis2

transforms.conf

[iis2]
DELIMS = " "
FIELDS = date, time,  s-ip, cs-method, cs-uri-stem, cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus, sc-win32-status, time-taken

Your field names might be different but I think this gives you an idea.

[appmandan]

Thanks everyone for the help. The transforms.conf file was the missing piece.

THANKS!

[appmandan]

Here's my code.
inputs.conf on universal forwarder:

[default]
[monitor://C:\inetpub\logs\LogFiles\W3SVC7]
sourcetype= iis
ignoreOlderThan = 14d
host = compdev2
#
[monitor://C:\inetpub\logs\LogFiles\W3SVC8]
sourcetype= iis
ignoreOlderThan = 14d
host = compdev2

props.conf on indexer:

[iis*] 
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32 
SHOULD_LINEMERGE = False 
CHECK_FOR_HEADER = True

I've created the props.conf file as instructed above. I am getting some IIS logs, however it's quite a bit fewer than what I was getting before the props.conf file creation. I initially pulled in over 17,000, but after enabling the props.conf file, I have only pulled 107. I now have less fields than I had before I configured the props file. I'm wanting to be able to pull top IP talkers in the search, but even though I'm getting IP addresses in the logs, I do not have a field for the IP addresses. I have tried creating the field, but that was identifying both the server and client IPs under the same field which I don't want to do. The iis header has a field for c-ip and s-ip. Any other suggestions as to what I'm doing wrong here?

[tgow]

The props.conf file does not get automatically created in the $SPLUNK_HOME/etc/system/local directory. The $SPLUNK_HOME/etc/system/README/props.conf.example and props.conf.spec have the examples of what is required for the props.conf file. You will need to create the $SPLUNK_HOME/etc/system/local/props.conf file manually.

[appmandan]

See below.

[Lamar]

Are you sourcetyping your log files as iis?

If so, you may want to ensure you have CHECK_FOR_HEADER enabled for that sourcetype.

This is what is being defined for that sourcetype by default:

[iis]
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

[appmandan]

One last question. My install doesn't have a props.conf file in the etc/system/local folder. I'm going to create one statically, but should an install include a props.conf file?

[Lamar]

Yeah, you have to be careful about iis log files as they tend to append the -n to the end for abstract reasons.

You could simple create your props.conf as such

[iis*]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

That should cover any iis sourcetypes that get created.

[appmandan]

I do have a custom sourcetype. What conf file is this in and could I add the same attributes for my custom sourcetype?

I did change the sourcetype to iis in the inputs.conf file on the universal forwarder. The sourcetype when it shows up in the indexer is iis-2