Splunk Search

IIS logs and v4.2.4

appmandan
Path Finder

I have set up a universal forwarder to forward IIS logs from C:\inetpub\logs\LogFiles\W3SVC7 to my splunk server on port 9997. I'm getting the logs fine, but I'm not getting the fields in the header...

I've read a lot about changing the props.conf file to tell splunk to pull the fields out of the header, but I'm assuming props.conf is no longer used in v4.2 because I do not have a props.conf file in ..\etc\apps\search\local. The lastest information I can find pertains to Splunk v4.1.1

What do I need to do to configure Splunk to pull the headers from the log file?

Thanks!

Tags (3)
0 Karma
1 Solution

tgow
Splunk Employee
Splunk Employee

If you want you can just specify the directory once in the inputs.conf and not have to worry about specifying each directory, ie:

[monitor://C:\inetpub\logs\logfiles]
sourcetype=iis
ignoreOlderThan = 14d
host = compdev2

Sometimes Splunk recognizes the header and sometimes it doesn't. I would keep your props.conf file the same but you will need to add the field definitions in the transforms.conf. Also, if the fields in the iis logs are delimited by a space then you can make this even easier.

props.conf

[iis*] 
pulldown_type=true
MAXTIMESTAMPLOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER
REPORT - iis2 = iis2

transforms.conf

[iis2]
DELIMS = " "
FIELDS = date, time,  s-ip, cs-method, cs-uri-stem, cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus, sc-win32-status, time-taken

Your field names might be different but I think this gives you an idea.

View solution in original post

tgow
Splunk Employee
Splunk Employee

If you want you can just specify the directory once in the inputs.conf and not have to worry about specifying each directory, ie:

[monitor://C:\inetpub\logs\logfiles]
sourcetype=iis
ignoreOlderThan = 14d
host = compdev2

Sometimes Splunk recognizes the header and sometimes it doesn't. I would keep your props.conf file the same but you will need to add the field definitions in the transforms.conf. Also, if the fields in the iis logs are delimited by a space then you can make this even easier.

props.conf

[iis*] 
pulldown_type=true
MAXTIMESTAMPLOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER
REPORT - iis2 = iis2

transforms.conf

[iis2]
DELIMS = " "
FIELDS = date, time,  s-ip, cs-method, cs-uri-stem, cs-uri-query, s-port, cs-username, c-ip, cs(User-Agent), sc-status, sc-substatus, sc-win32-status, time-taken

Your field names might be different but I think this gives you an idea.

appmandan
Path Finder

Thanks everyone for the help. The transforms.conf file was the missing piece.

THANKS!

0 Karma

appmandan
Path Finder

Here's my code.
inputs.conf on universal forwarder:

[default]
[monitor://C:\inetpub\logs\LogFiles\W3SVC7]
sourcetype= iis
ignoreOlderThan = 14d
host = compdev2
#
[monitor://C:\inetpub\logs\LogFiles\W3SVC8]
sourcetype= iis
ignoreOlderThan = 14d
host = compdev2

props.conf on indexer:

[iis*] 
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32 
SHOULD_LINEMERGE = False 
CHECK_FOR_HEADER = True

I've created the props.conf file as instructed above. I am getting some IIS logs, however it's quite a bit fewer than what I was getting before the props.conf file creation. I initially pulled in over 17,000, but after enabling the props.conf file, I have only pulled 107. I now have less fields than I had before I configured the props file. I'm wanting to be able to pull top IP talkers in the search, but even though I'm getting IP addresses in the logs, I do not have a field for the IP addresses. I have tried creating the field, but that was identifying both the server and client IPs under the same field which I don't want to do. The iis header has a field for c-ip and s-ip. Any other suggestions as to what I'm doing wrong here?

0 Karma

tgow
Splunk Employee
Splunk Employee

The props.conf file does not get automatically created in the $SPLUNK_HOME/etc/system/local directory. The $SPLUNK_HOME/etc/system/README/props.conf.example and props.conf.spec have the examples of what is required for the props.conf file. You will need to create the $SPLUNK_HOME/etc/system/local/props.conf file manually.

0 Karma

appmandan
Path Finder

See below.

0 Karma

Lamar
Splunk Employee
Splunk Employee

Are you sourcetyping your log files as iis?

If so, you may want to ensure you have CHECK_FOR_HEADER enabled for that sourcetype.

This is what is being defined for that sourcetype by default:

[iis]
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

appmandan
Path Finder

One last question. My install doesn't have a props.conf file in the etc/system/local folder. I'm going to create one statically, but should an install include a props.conf file?

0 Karma

Lamar
Splunk Employee
Splunk Employee

Yeah, you have to be careful about iis log files as they tend to append the -n to the end for abstract reasons.

You could simple create your props.conf as such

[iis*]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = True

That should cover any iis sourcetypes that get created.

appmandan
Path Finder

I do have a custom sourcetype. What conf file is this in and could I add the same attributes for my custom sourcetype?

I did change the sourcetype to iis in the inputs.conf file on the universal forwarder. The sourcetype when it shows up in the indexer is iis-2

0 Karma
Get Updates on the Splunk Community!

Exporting Splunk Apps

Join us on Monday, October 21 at 11 am PT | 2 pm ET!With the app export functionality, app developers and ...

Cisco Use Cases, ITSI Best Practices, and More New Articles from Splunk Lantern

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Build Your First SPL2 App!

Watch the recording now!.Do you want to SPL™, too? SPL2, Splunk's next-generation data search and preparation ...