topic Help configuring Universal Forwarder with IIS logs in Getting Data In

Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 17:36:19 GMT

I tried going through the documentation, but haven't been able to get much working with the exception of syslog messages from my firewall.

On the web server where I installed the Universal Forwarder, I created 2 files in etc\system\local

I used bits and pieces of information that I found in various answers here.

inputs.conf

[monitor://D:\Logs\inetpub\*\*.log]
disabled = false
followtail = 0
sourcetype=iis
ignoreOlderThan = 14d

outputs.conf

[tcpout-server://myserverip:9000]

On my server I added a TCP data input on port 9000

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 17:50:18 GMT

Did you restart the forwarder?
Are there any errors in the splunkd.log found in splunk\var\log\splunk\

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 17:54:16 GMT

Yes, I restarted it.

I see the following:
LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.

So I guess my outputs.conf is wrong? Again I put that together from another post/article.

If that is incorrect, what should it look like?

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 18:09:09 GMT

The outputs.conf should have been created when you installed the forwarder, and it will be located in splunk\etc\system\local\outputs.conf It should contain a tcpout:default and/or a tcpout-server://ip:port.

Try adding a _TCP_ROUTING attribute to your inputs.conf monitor stanza, the value should be your indexing server name:

Inputs.conf

  [monitor://D:\Logs\inetpub\*\*.log]
    _TCP_ROUTING = indexingservername
    disabled = false
    followtail = 0
    sourcetype=iis
    ignoreOlderThan = 14d

Outputs.conf

[tcpout]
defaultGroup=indexingservername

[tcpout:indexingservername]
server=ipaddress:port of indexing server

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 18:12:08 GMT

I had to create the input.conf and output.conf files manually. They were not there. I do not understand what you mean by indexing server name.

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 18:18:45 GMT

You can use the name of the indexer, or it's IP address.
The indexing server is the server with the main Splunk instance - the forwarder sends logs to the indexing server.
You can also use the IP address of the indexing server.

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 18:22:10 GMT

Now you are confusing me even more than I already am hehe. I thought configuration settings for where the data goes was done in outputs.conf, yet your recommendation was to add a setting in inputs.conf for the server?

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 18:25:02 GMT

It happens, I updated the answer to make it more clear.
Both inputs and outputs need to be on the forwarder.

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 18:28:51 GMT

I reinstalled it and now it created the inputs.conf and output.confs. The inputs.conf has the following

[default]
host = MYSERVER

[script://$SPLUNK_HOME\bin\scripts\splunk-wmi.path]
disabled = 0

I added the log monitor lines from before.

The output.conf now looks like this:

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = myserverip:9000

[tcpout-server://myserverip:9000]

I don't really understand the configuration as it looks redundant.

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 18:28:57 GMT

Anyways, after restarting the forwarder, it seems to have sent 4 messages, but it looks like garbage.

--splunk-cooked-mode-v3--\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00

and it's definitely not the data from my IIS logs.

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 18:34:15 GMT

Also, I see the following WARN in the logs:
TcpOutputProc - Cooked connection to ip=myserverip:9000 timed out

In the Splunk server, I have an TCP input configured on port 9000 set to Source type = iis

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 18:55:31 GMT

In your splunk\etc\system\local\inputs.conf what is the [splunktcp:port] port number?

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 19:37:19 GMT

There is no line [splunktcp:port] in my inputs.conf

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 20:03:54 GMT

What is the default tcp port for inputs on the indexer?
You should not have to create a special input port for the iis logs.
Also, iis logs are typically not configured with the default iis sourcetype because it does not handle headers well. You will probably want to create a custom sourcetype and configuration in your props.conf and transforms.conf. There are many examples in this forum, but if you want specific help then post the header line that shows the fields.
Lastly, universal forwarders should not be sending cooked data, are you sure you installed a universal forwarder?

Re: Help configuring Universal Forwarder with IIS logs

adrianp — Wed, 13 Nov 2013 20:28:48 GMT

Sigh, apparently I missed the part in the documentation where it went over the configuration steps on the receiver. As soon as I did that I started receiving the events. I followed your recommendation and removed the TCP input on port 9000. I have the forwarder and receiver configured to use the default port 9997. Thank you so much! Next I get to work with IIS advanced logging hehe, but this is a good start!

Re: Help configuring Universal Forwarder with IIS logs

lukejadamec — Wed, 13 Nov 2013 20:32:03 GMT

Roger that. First step is to get the header line from the top of the iis log file. You'll need that to tell splunk what the field names are and what to use as a delimiter.

Re: Help configuring Universal Forwarder with IIS logs

mccurity — Tue, 10 Sep 2019 20:55:36 GMT

Check your monitor path. I believe it should be D:\inetpub\Logs......