Getting Data In
Highlighted

What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

I followed the tutorial very carefully on setting up the forwarder on my two Tomcat servers. Now I am trying to verify that I can actually receive data from my catalina logs to my sandbox. When I go to 'Add Data', and click on 'forward' it gives me the notice: "There are currently no forwarders configured as deployment clients to this instance." But at the top of my screen I get another notice stating that: "Forwarding to indexer group default-autolb-group blocked for 1200 seconds.", which 'default-autolb-group' is the defaultGroup in my /opt/splunkforwarder/etc/system/local/output.conf file. I think that I am close on getting a connection but I am missing some step to complete it. Can someone help me on what I missing to verify a successful connection?

Also, my inputs.conf file only has the ip address of my server; do I need to put information about my catalina log file and if so what is the format, thanks!

Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

Please post your inputs.conf and outputs.conf files. In a simple setup on your forwarder you should have your sandbox set up as a forward server and your inputs should be defined.

For tomcat, you would want monitor stanza(s) specifying the files you want to start indexing. I just answered another question (here: http://answers.splunk.com/answers/207373/why-am-getting-error-there-are-currently-no-forwar.html ) with regards to the "deployment clients" error. It seems that some information about setting up deployment clients has been left out here for the way sandbox "wizards" are designed. I am thinking that you are pretty close and perhaps seeing the conf files will help get it straightened out.

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

I followed your last comment and my outputs.conf is:

[tcpout-server://input-prd-p-c325dgfktbm7.cloud.splunk.com:9997]

[tcpout:splunkcloud]
disabled = false
server = input-prd-p-c325dgfktbm7.cloud.splunk.com:9997

[tcpout]
defaultGroup = splunkcloud

and my inputs.conf is:

[default]
host = ip-172-31-35-141

I have only made changes to my outputs.conf and I am not sure on what to change for inputs

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

You will need to have appropriate monitor stanzas on the forwarder for the tomcat logs you want to start indexing, ideally these will also need to be assigned an appropriate sourcetype.

Have a look at this:
http://docs.splunk.com/Documentation/Splunk/latest/Data/Monitorfilesanddirectories

Here is another answer which should get you in the right direction on inputs. This person appears to have set up different sourcetypes for the different logs:
http://answers.splunk.com/answers/135355/proper-input-conf-setup-apache-tomcat.html

My procedure is to load an example file on a splunk instance through add data and use the "data preview" functionality it to make sure timestamps and event breaks are getting parsed and what sourcetype settings are needed to make this happen for each sourcetype.

BTW, I removed tcpout-server stanza from my outputs.conf before my remote forwarder actually connected to the sandbox and forwarded events.

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

I also realized that I am changing my files from /opt/splunkforwarder/etc/system/local/outputs.conf but should it be from /opt/splunkforwarder/etc/apps/search/local?

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

In my opinion, no. The configs under SPLUNKHOME/etc/apps/search for the search app, which is not relevant on a Universal Forwarder system.

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

thanks for your help, quick question about the monitor, i cant simply just do

/opt/splunkforwarder/bin/splunk add monitor /var/lib/tomcat7/logs

to add a monitor?

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

and if my inputs.conf file isn't correctly setup with a monitor, would that be the reason why I am still not picking up the forwarder?

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Path Finder

I just changed my inputs.conf to:

[default]
host = ip-172-31-35-141

[monitor:/var/lib/tomcat7/logs/catalina.*]

disabled = false
index = test
sourcetype = catalina

0 Karma
Highlighted

Re: What am I missing to get a successful connection between my Universal Forwarder and the sandbox?

Builder

At this point I would check the splunkd.logs on your fowarder and run the following search on your sandbox:
index=_internal xx.xx.xx.xx

where xx.xx.xx.xx is your forwarder's outside IP address.

This might provide some clues about connection status.

0 Karma