Cant get data using the Windows App for Windows Infrastructure.

[uktechnologyser]

Cant get any Data from this, setting it up with a 1 deployment app, 1 search head and 2 clustered indexers. Worked through the 'Get Windows Data' and noticed a comment on the document - http://docs.splunk.com/Documentation/MSApp/latest/MSInfra/Confirmandtroubleshootdatacollection

"After running thru the install, I wasn't receiving events in my indexes. It appears that the input.conf from the Splunk Add-on for Windows and the indexes.conf from the Splunk App for Windows Infrastructure don't match up.

Inputs.conf from the Splunk Add-on for Windows, which I deployed to my Universal Forwarder sends the events to an index called wineventlog, but the indexes.conf file that is copied to C:\Program Files\Splunk\etc\system\local\ during the indexer setup step doesn't create that index. It only creates MSAD, PERFMON, & WINEVENTS.

I also have events on the input.conf from the Splunk Add-on for Windows that are trying to hit an index called WINDOWS.

I guess I may need to tweak either the indexes.conf file or the inputs.conf file so that the events can be correctly indexed? If I change one will it break something else?(dashboards)

BWWB
August 28, 2015"

Can anyone verify if this is the problem? I have continued onto the 'Get Active Directory Data' section and the input.conf file for this app does reference the MSAD, PERFMON, & WINEVENTS. indexes but still no data appears. Will the fact i dont have a licence installed have anything to do with this?

Thanks in advance,

Jay

[uktechnologyser]

Update on this, i have spoken to a support representative and they think the issue is because i have Splunk Free and NOT Splunk Enterprise installed Slams head against desk. Hopefully when i get an enterprise licence installed the clients will start sending info to the indexers.

[alemarzu]

Hi uktechnologyservices,

Try like this,

[tcpout]
defaultGroup = My_Cluster_1

[tcpout:My_Cluster_1]
server = 10.1.4.32:9997,10.1.4.33:9997

[uktechnologyser]

Hi alemarzu,

I have made that change and my deployment client has picked up the configuration but i still cant see anything coming from the client on my search head. Wish the send to indexer app has its own log so we could figure out what's going on. splunkd just seems to show the client calling home to the deployment server and making any changes if it sees them. Is there a debug log for send to indexer or the windows infrastructure app?

[alemarzu]

In $SPLUNKFORWARDER_HOME/var/log/splunk you will find splunkd.log you may find a hint there.

Did you read this ?

https://answers.splunk.com/answers/223604/why-is-no-data-being-indexed-after-deploying-the-s.html#an...

[uktechnologyser]

Yeah i've looked at the splunkd.log, cant find any errors other than the ones i have mentioned in the thread you have provided the link for.

ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.

I get the same error on the other DC i just installed the Universal forwarder on. Maybe i should try putting the config in the C:\Program Files\SplunkUniversalForwarder\etc\system\local inputs.conf file as it doesn't seem to pick it up from the inputs.conf on the send to indexer app?

[uktechnologyser]

Looking at the troubleshooting document i thought i would check the indexers input.conf file to see if its the same as they show. In $SPLUNK_HOME/etc/system/local on each indexer It should be:

[splunktcp://9997]
disabled = 0

However all i have is

[default]
host = IndexerHostName

I set up receiving in Splunk Web for each Indexer and restarted them. I have used a port query program and i can confirm both are listening on 9997 as i have defined. Strange how it isnt showing in the inputs.conf file as it should. Maybe something to do with it?

[alemarzu]

It's okey, it wont be in /system/local. Check under $SPLUNK_HOME/etc/apps/launcher/local it's going to be there.

You could also run this from the CLI for troubleshooting that particular .conf file

$SPLUNK_HOME\bin> splunk cmd btool inputs list

[uktechnologyser]

Ah yes i can see it now, Although it doesn't have disabled=0 but connection_host = ip under the first line. Going to try and install the forwarder on a server that isnt a DC to see if that makes a difference. I dont think it will work, im sure there is a problem with the app configuration.

If i run the same tool for troubleshooting but replace inputs with outputs will that give me all of the outputs.conf files on the server i run it on?

[alemarzu]

Yes, you can troubleshoot any .conf file u need.

[Richfez]

This sounds very familiar - I believe I had this issue about 6 months ago when I installed the same set of apps. I'm pretty sure I just created that missing index (adding a stanza to the same indexes.conf where all the other indexes for it were defined) and everything was fine afterwards.

You shouldn't hurt anything by adding that index - the events aren't going anywhere at the moment so making them go somewhere is better than nowhere.

And no, this has nothing to do with having no license installed.

[uktechnologyser]

Hi Rich,

Thanks for your reply, i have added the missing indexes 'wineventlog' and 'windows' but i still cant see any data on my search head. I can see the new indexes have appeared on my deployment server and my indexers. It think maybe that my deployment client isn't sending the information? I cant see anything that looks like logs in the folders on the indexers (what would they look like?), is there a way to tell if my deployment client is trying to send the logs to the indexers? I have looked at the splunkd.log but not really sure what i am looking for to be honest. The only error i can see says:

"ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf."

My outputs.conf file locate din C:\Program Files\SplunkUniversalForwarder\etc\apps\Send_To_Indexer looks like this:

[tcpout]
defaultGroup = My_Cluster_1

[tcpout:My_Cluster_1]
server = 10.1.4.32:9997,10.1.4.33:9997

[tcpout-server://10.1.4.32:9997]

[tcpout-server://10.1.4.33:9997]

This should send the data to both indexers.

Any help appreciated.

Cheers,

Jay

[uktechnologyser]

This still doesn't seem to be working. in the splunkd log i get the same error that says ERROR TcpOutputProc - LightWeightForwarder/UniversalForwarder not configured. Please configure outputs.conf.It looks like the deployment client it isn't referencing the outputs.conf file in the send to indexer app like it should. Any ideas why it would do this?