I am new Splunk user. I configured the index server and set it up as a receiver. I then installed the light forwarder on another Windows box and configured it to forward to the index server. It appears to be connecting to the Splunk index, according to the splunkd logs on the index.
However, Splunk web does not seem to be indexing the forwarded server data. Under Apps--> Windows, only the original index server shows up under hosts. Shouldn't that show 2 now and have the forwarder listed under there as well? The manual doesn't really explain what to expect in these screens once forwarding is complete, but it doesn't show any content for the forwarded server. Here is the relevant info from the log files on splunk.
I see entries saying "Connecting in cooked mode from (server)." I also see entries saying "Connection accepted from (server)." The other entry I see that might be relevant is "Hostname=(server) closes connection.. ended without a done-key."
There could be a couple of things going on here:
first, the trivial - have you actually created any monitoring stanzas on the forwarder? i.e. are you actually monitoring anything at all?
then, when you go to the SEARCH app summary dashboard, under the list of hosts, do you see the forwarder there?
Lastly, if you do a search like:
index=_internal do you just see logs from your indexer or your forwarder as well?
Sorry for the delayed response. I forget to check the notify box, so I had no idea someone had answered me.
I actually don't know what monitoring stanzas are, so I will look into that. I do not see the forwarder under hosts in the Search app. Just the indexer. I see no logs at all from the forwarder.
again, you need to be sure that you are indeed monitoring data within the light weight forwarder. monitor stanzas are what you tell splunk to actually monitor, check your inputs.conf for example.
The idea is, you cannot forward if you have nothing to forward. Since when you do index=_internal you do see data, then the forwarding is working correctly. it is just that you are not monitoring anything. Check this link for more info: http://www.splunk.com/base/Documentation/4.1.5/admin/Inputsconf