I have deployed the Splunk Add-on for Microsoft Windows and the send to indexer app successfully to my forwarders.
I have a server called activedirectory which I can see has downloaded the apps, however, no data is being indexed. Actually, I can't see data from any of the servers that have the app deployed.
I can see on the splunk server that activedirectory has made a connection on 9997.
if I search for host=activedirectory there are no events
Any ideas where I should start troubleshooting?
Host Count Last Update
SERVER1 135 3/24/15 2:30:30.000 PM
splunk 1 3/24/15 12:45:33.000 PM
[WinEventLog://Application] disabled = 0 start_from = oldest current_only = 0 checkpointInterval = 5 index = wineventlog renderXml=false [WinEventLog://Security] disabled = 0 start_from = oldest current_only = 0 evt_resolve_ad_obj = 1 checkpointInterval = 5 blacklist1 = EventCode="4662" Message="Object Type:\s+(?!groupPolicyContainer)" blacklist2 = EventCode="566" Message="Object Type:\s+(?!groupPolicyContainer)" index = wineventlog renderXml=false [WinEventLog://System] disabled = 1 start_from = oldest current_only = 0 checkpointInterval = 5 index = wineventlog renderXml=false
[tcpout] defaultGroup = default-autolb-group [tcpout:default-autolb-group] server = splunk:9997 [tcpout-server://splunk:9997]
I have a few troubleshooting questions for you:
a) What do your splunkd logs say for both forwarder and indexer? Are there any errors at the forwarder? Maybe the user running Splunk doesn't have winevent log read access?
b) on indexer side, you're writing to index wineventlog. How do you know that it isn't indexing? Have you tried specified the index as part of your query?
c) how do you know that your host is activedirectory and not activedirectory?
Always start with the splunkd logs for any basic errors.
a) My splunkd log on the deployment client says:
ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.
Does this mean it isnt referencing the outputs.conf file in the send to indexer app? This has the configuration telling it to tcp out to the indexers.
b) I know it isnt indexing because i cant see any files populating in the wineventlog folder on the indexers.
c) my host doesnt appear,
Interestingly if i install the SplunkTAwindows app directly on the indexers it starts getting logs straight away and storing them locally. From there i can search them from the search head. It must be a problem with the send to indexer app.
How did you see the deployment client made a connection on 9997? Did you get a fix for this? I have the same problem.
Hope it helps.