Splunk Search

IIS logs and v4.2.4

Path Finder

I have set up a universal forwarder to forward IIS logs from C:\inetpub\logs\LogFiles\W3SVC7 to my splunk server on port 9997. I'm getting the logs fine, but I'm not getting the fields in the header...

I've read a lot about changing the props.conf file to tell splunk to pull the fields out of the header, but I'm assuming props.conf is no longer used in v4.2 because I do not have a props.conf file in ..\etc\apps\search\local. The lastest information I can find pertains to Splunk v4.1.1

What do I need to do to configure Splunk to pull the headers from the log file?

Thanks!

Tags (3)
0 Karma
1 Solution

Splunk Employee
Splunk Employee

If you want you can just specify the directory once in the inputs.conf and not have to worry about specifying each directory, ie:

[monitor://C:\inetpub\logs\logfiles]
sourcetype=iis
ignoreOlderThan = 14d
host = compdev2

Sometimes Splunk recognizes the header and sometimes it doesn't. I would keep your props.conf file the same but you will need to add the field definitions in the transforms.conf. Also, if the fields in the iis logs are delimited by a space then you can make this even easier.

props.conf

[iis*] 
pulldown_type=true
MAXTIMESTAMPLOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER
REPORT - iis2 = iis2

transforms.conf

[iis2]
DELIMS = " "
FIELDS = date, time,  s-ip, cs-method, cs-uri-stem, cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus, sc-win32-status, time-taken

Your field names might be different but I think this gives you an idea.

View solution in original post

Splunk Employee
Splunk Employee

If you want you can just specify the directory once in the inputs.conf and not have to worry about specifying each directory, ie:

[monitor://C:\inetpub\logs\logfiles]
sourcetype=iis
ignoreOlderThan = 14d
host = compdev2

Sometimes Splunk recognizes the header and sometimes it doesn't. I would keep your props.conf file the same but you will need to add the field definitions in the transforms.conf. Also, if the fields in the iis logs are delimited by a space then you can make this even easier.

props.conf

[iis*] 
pulldown_type=true
MAXTIMESTAMPLOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER
REPORT - iis2 = iis2

transforms.conf

[iis2]
DELIMS = " "
FIELDS = date, time,  s-ip, cs-method, cs-uri-stem, cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus, sc-win32-status, time-taken

Your field names might be different but I think this gives you an idea.

View solution in original post

Path Finder

Thanks everyone for the help. The transforms.conf file was the missing piece.

THANKS!

0 Karma

Path Finder

Here's my code.
inputs.conf on universal forwarder:

[default]
[monitor://C:\inetpub\logs\LogFiles\W3SVC7]
sourcetype= iis
ignoreOlderThan = 14d
host = compdev2
#
[monitor://C:\inetpub\logs\LogFiles\W3SVC8]
sourcetype= iis
ignoreOlderThan = 14d
host = compdev2

props.conf on indexer:

[iis*] 
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32 
SHOULD_LINEMERGE = False 
CHECK_FOR_HEADER = True

I've created the props.conf file as instructed above. I am getting some IIS logs, however it's quite a bit fewer than what I was getting before the props.conf file creation. I initially pulled in over 17,000, but after enabling the props.conf file, I have only pulled 107. I now have less fields than I had before I configured the props file. I'm wanting to be able to pull top IP talkers in the search, but even though I'm getting IP addresses in the logs, I do not have a field for the IP addresses. I have tried creating the field, but that was identifying both the server and client IPs under the same field which I don't want to do. The iis header has a field for c-ip and s-ip. Any other suggestions as to what I'm doing wrong here?

0 Karma

Splunk Employee
Splunk Employee

The props.conf file does not get automatically created in the $SPLUNK_HOME/etc/system/local directory. The $SPLUNK_HOME/etc/system/README/props.conf.example and props.conf.spec have the examples of what is required for the props.conf file. You will need to create the $SPLUNK_HOME/etc/system/local/props.conf file manually.

0 Karma

Path Finder

See below.

0 Karma

Splunk Employee
Splunk Employee

Are you sourcetyping your log files as iis?

If so, you may want to ensure you have CHECK_FOR_HEADER enabled for that sourcetype.

This is what is being defined for that sourcetype by default:

[iis]
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

Path Finder

One last question. My install doesn't have a props.conf file in the etc/system/local folder. I'm going to create one statically, but should an install include a props.conf file?

0 Karma

Splunk Employee
Splunk Employee

Yeah, you have to be careful about iis log files as they tend to append the -n to the end for abstract reasons.

You could simple create your props.conf as such

[iis*]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

That should cover any iis sourcetypes that get created.

Path Finder

I do have a custom sourcetype. What conf file is this in and could I add the same attributes for my custom sourcetype?

I did change the sourcetype to iis in the inputs.conf file on the universal forwarder. The sourcetype when it shows up in the indexer is iis-2

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!