Splunk Search

manual inputs.conf issues

ulikabbq
Path Finder

I am having trouble with manual inputs.conf.

I have been able to successfully setup a windows universal forwarder, and if I define the monitor folder during the setup, everything works fine. However, when I try to manually configure the inputs.conf in etc\system\local I am not getting data on my indexer. The splunkd.log shows me that it is adding a watch path on the manually configured path and it shows that it is connected to the indexer.

08-03-2014 14:25:27.851 -0500 INFO  TailingProcessor - Adding watch on path: D:\Splunk\ForwardTest\Test1.
08-03-2014 14:25:27.851 -0500 INFO  TailingProcessor - Adding watch on path: D:\Splunk\ForwardTest\Test2.
08-03-2014 14:25:28.139 -0500 INFO  TcpOutputProc - Connected to idx=x.x.x.x:9997

inputs.conf

[monitor://D:\Splunk\ForwardTest\Test2]
disabled = false
sourcetype = userdata

props.conf

[userdata]
SHOULD_LINEMERGE = False
pulldown_type = 1
REPORT-getfields = userdata_fields

transforms.conf

[userdata_fields]
DELIMS="/t"
"key","date","url",
Tags (2)
0 Karma

kristian_kolb
Ultra Champion

Just to be crystal clear - the inputs.conf you describe in your question should go on the Forwarder, and there should also be an outputs.conf pointing to the Indexer (you seem to have this already, no?).

The props.conf and transforms.conf should be on the Indexer. Of course, there should be an inputs.conf on the Indexer as well, stating that it should listen for incoming traffic on port (usually) 9997.

Btw, is the DELIMS supposed to be \t or /t?

Finally - how do you know that you are not getting data? It may seem like a silly question, but if the timestamp information (date) is parsed wrong, your events may end up in a different time than you think (and you will not see them when searching for something like 'last hour').

When adding new log sources, it is a good practice to use a test index. This allows you to quickly search the whole index, and also avoids polluting your production data with logs that are not correctly parsed. When you got your sourcetype, host, timestamps and linebreaking done correctly, you re-configure to send data to the production index.

/K

0 Karma

ulikabbq
Path Finder

yes my inputs.conf is on the forwarder. Also I have one server with an even easier inputs.conf for monitoring iis logs and I am not getting that data either. I have one install where I set the iis log directory up when I setup the forwarder and that works fine, but the manual config does not work.

here is the manual inputs.conf
[default]
host=x.x.x.x

[monitor://C:\inetpub\logs\LogFiles\W3SVC1] (the forward slashes are in the right place just not showing)
disabled = false
sourcetype = iis
index = main

I know I don't have data because I don't see my files in the data summary in the search.

0 Karma

lguinn2
Legend

First, the file is transforms.conf not Transform.Conf

Also, your transforms.conf has at least one problem. I would change it to match the following

transforms.conf

[userdata_fields]
DELIMS="/t"
FIELDS=key,date,url

I assume that there is a directory named D:\\Splunk\ForwardTest\Test2 - although the double slash \\ does concern me. Is there anything in Test2 directory? Again, remember that Splunk is case-sensitive.

0 Karma

ulikabbq
Path Finder

also when I run $SPLUNK_HOME/bin/splunk list monitor on the forwarder I see the files I want.

When I run 'sudo netstat -taupen' on my indexer I see my forwarder.

What am I missing here? I have this problem on 2 servers that I have manually setup.

0 Karma

ulikabbq
Path Finder

I also installed S.O.S and it sees the forwarder so I feel like there is some small detail that I am missing.

0 Karma

ulikabbq
Path Finder

I have updated the errors in my post. It is D:\Splunk\ForwardTest\Test2 and I have it set to transforms.conf. (update: it seems these comment boxes don't always see my forward slashes)

I took out the " " from the transforms.conf and restarted the forwarder, but I still don't have any data in my indexer.

0 Karma