topic Re: Active forwards don't forward data to Splunk Cloud instance in Getting Data In

Active forwards don't forward data to Splunk Cloud instance

johnwl — Fri, 19 Jun 2015 19:13:16 GMT

I use username: admin and password: changeme to log in to my Splunk universal forwarder. I am trying to forward logs from my Ubuntu server that's running on Vagrant VM. I know that the forwarder is active because:

root@vagrant-ubuntu-trusty-64:/opt/splunkforwarder/bin# ./splunk list forward-server
Your session is invalid.  Please login.
Splunk username: admin
Password: 
Active forwards:
    192.168.33.10:9997
Configured but inactive forwards:
    None

[monitor://$SPLUNK_HOME/var/log/splunk]
index = _internal

[monitor://$SPLUNK_HOME/etc/splunk.version]
_TCP_ROUTING = *
index = _internal
sourcetype=splunk_version

[monitor:///var/log/upstart/docker.log]

[monitor:///var/log/upstart/]

[monitor:///var/log/]

[tcpout]
defaultGroup = default-autolb-group

[tcpout:default-autolb-group]
server = 192.168.33.10:9997

[tcpout-server://192.168.33.10:9997]

I configured the guest and host ports on the Vagrant VM as 9997. But nothing at all is being sent to my Splunk Cloud. Any help?!!! Thanks

Re: Active forwards don't forward data to Splunk Cloud instance

ChrisG — Fri, 19 Jun 2015 21:43:37 GMT

Are you using the Splunk Cloud universal forwarder app for the certificate and credentials?

Re: Active forwards don't forward data to Splunk Cloud instance

johnwl — Fri, 19 Jun 2015 22:10:19 GMT

Yes, from the page where all of this is listed:

Use this app to set up the universal forwarder. After you download the universal forwarder, follow the steps to install and start it, then specify the data you want to send to Splunk Cloud.

To set up the Universal Forwarder:
Download the universal forwarder from splunk.com to the /opt directory on the machine that will send data to Splunk Cloud.

Download the universal forwarder credentials to the /opt directory of the machine that will send data to Splunk Cloud.

Install the universal forwarder on your operating system by following the Splunk Enterprise installation instructions.
Install the universal forwarder credentials by entering the following command:

/opt/splunkforwarder/bin/splunk install app /opt/splunkclouduf.spl -auth admin:changeme
Add data to Splunk Cloud using the command line interface (CLI).
For example, add application logs to Splunk Cloud using the following command:

/opt/splunkforwarder/bin/splunk add monitor -auth admin:changeme /path/to/app/logs/
Where /path/to/app/logs/ is the path to application logs that you want to add to Splunk Cloud.

I downloaded the 64-bit linux distribution .deb and saw the following when I executed it yesterday:

vagrant@vagrant-ubuntu-trusty-64:/vagrant$ sudo dpkg -i splunkforwarder-6.2.3-264376-linux-2.6-amd64.deb
Selecting previously unselected package splunkforwarder.
(Reading database ... 97090 files and directories currently installed.)
Preparing to unpack splunkforwarder-6.2.3-264376-linux-2.6-amd64.deb ...
Unpacking splunkforwarder (6.2.3) ...
Setting up splunkforwarder (6.2.3) ...
complete

Re: Active forwards don't forward data to Splunk Cloud instance

johnwl — Fri, 19 Jun 2015 22:10:33 GMT

I just installed the security file splunkclouduf.spl and I have some progress: The spunk cloud instance is finally listed as a forward input-prd-p-m56vqljf2w93.cloud.splunk.com:9997 (ssl) .

Active forwards:
192.168.33.10:9997
input-prd-p-m56vqljf2w93.cloud.splunk.com:9997 (ssl)
Configured but inactive forwards:
None

The data still doesn't forward though.

Re: Active forwards don't forward data to Splunk Cloud instance

esix_splunk — Sat, 20 Jun 2015 05:04:42 GMT

If you are seeing the cloud indexers in your forwarder, its most likely working. On your cloud SH, run a search on the _internal index and see if you can see your forwarder.

Aside from that, what does your outputs look like? What sourcetype and index are you sending to?

Re: Active forwards don't forward data to Splunk Cloud instance

johnwl — Mon, 22 Jun 2015 18:39:40 GMT

Thanks for helping me. I have not specified any specific index for indexing. I am getting hundreds of thousands of events, but they are all from the spunk log, and not from the path that I wanted them to come from (/var/log/mylogs/). This is an example of what an entry looks like in my Splunk Cloud interface:

6/22/15 6:04:21.485 PM

2015-06-22 18:04:21,485 WARNING Generator Queue Full, looping
host = ip-192-168-106-249 source = /opt/splunk/var/log/splunk/eventgen.log sourcetype = eventgen

Any idea on what I can do to get them to come from the path I specified?

Re: Active forwards don't forward data to Splunk Cloud instance

johnwl — Mon, 22 Jun 2015 20:58:13 GMT

I solved the problem. I had to put the following into the outputs.conf file:

[tcpout]
defaultGroup = default-autolb-group,splunkcloud

Re: Active forwards don't forward data to Splunk Cloud instance

esix_splunk — Tue, 23 Jun 2015 01:17:52 GMT

One thing to note about this configuration, is that if one of the group defined is down or blocked, both tcpoutputs will halt. If you need to dual stream to your cloud instance, and your local instance, its recommended to install another UF on your host and route data to your cloud instance from a different UF.