topic Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial? in Getting Data In

How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Thu, 07 Apr 2016 22:23:07 GMT

Hi,

I recently started using the Splunk Cloud free trial. I installed a universal forwarder locally and authorized it with the credential downloaded from Splunk Cloud.
I don't see any option in the Splunk Cloud UI to configure a receiving port. How do I make the forwarder send data to Splunk Cloud?

Thanks,
Saravana

Update with additional information:

These are the steps I have done...

Universal Forwarder
-I got my Splunk cloud free trial login
-Downloaded the universal forwarder app
-Installed the app by using the credential downloaded as spl file.
-I added a particular directory to monitor.

Using Splunk Enterprise Forwarder
-Configured the Splunk Cloud instance and port in forwarder section of my Splunk Enterprise.
-Not able to see receiving port section in Splunk Cloud instance

When I do list monitor, I get the directory in list of monitored directories. but data is not available in search of Splunk Cloud

Please let me know as to where the problem might be.

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

gearmesh — Thu, 07 Apr 2016 22:55:14 GMT

I have the same issue.
On my client I ran:
SPLUNK.exe install app splunkclouduf.spl -auth

I get: Login Failed

Do I use a different name and password than what I use to login into my Splunk Trial/Console on the web?

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

ChrisG — Thu, 07 Apr 2016 23:04:25 GMT

You don't need to configure a receiving port. Did you define inputs? See How to forward data to Splunk Cloud in the Forwarder Manual.

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

gearmesh — Fri, 08 Apr 2016 17:36:57 GMT

I found my answer.

The default pre-populated url below would not accept the default username and password
http://computername:8000/en-US/account/login
change it to this:
http://localhost:8000
and the defaults username and password work and allow you to change the password.

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Fri, 08 Apr 2016 17:52:15 GMT

Where did you change this url ? Is it part of universal forwarder configuration somewhere?

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Mon, 11 Apr 2016 18:16:32 GMT

I downvoted this post because when i do list monitor i get the directory in list of monitored directories. but data is not available in search of splunk cloud. i have installed the universal forwarder with the spl file downloaded from my splunk cloud instance.

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

ChrisG — Mon, 11 Apr 2016 19:30:46 GMT

Can you give an example of your inputs.conf file? Did you add the necessary stanzas as described in Monitor files and directories with inputs.conf in the Getting Data In manual?

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Mon, 11 Apr 2016 20:50:05 GMT

This is the input.conf file in C:\Program Files\Splunk\etc\system\local

[default]
host = SAKARUNA-WS

[monitor://$SPLUNK_HOME\etc\splunk.version]
disabled = false

[monitor://C:\SplunkDir]
disabled = false

c:\SplunkDir is the directory i want to monitor

Thanks,
Saravana

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

gearmesh — Tue, 12 Apr 2016 00:08:35 GMT

That is the url that opens after completing the forwarder (6.1) installation.
It also can be entered in a browser once the forwarder is installed.

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Wed, 13 Apr 2016 17:48:57 GMT

Any thoughts on this Chris.?

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

yannK — Wed, 13 Apr 2016 18:04:35 GMT

first of all, there is no UI in an universal forwarder, so if you see an UI, this is a full instance, or a heavy forwarder.
you do not need to open ports or inputs on the cloud instsances, they are already listening, just setup your forwarder

1 - When you install the forwarder package
download from splunk.com or from the splunkcloud UI, Usually the Universal Forwarder is fine, in some special cases, you may need the full splunk install (to use it as an Heavy forwarder)

on linux it's simple. untar, rpm, deb ....
on windows, there is a wizard, please do not use the wizards pages to setup the forwarding to cloud

2 - once the forwarder is installed, the user for the CLI is "admin" password "changeme"
Then you need to install the cloud app package (download from your splunkcloud instance, un the app UF)
the package is a *.spl

you can install it on the command line with

#on linux
   cd /opt/splunkforwarder/bin
   ./splunk install app /path/to/my/<mycloudforwarderpackage.spl>



 #on windows, 
    cd C:\Program files\splunkforwarder\bin
    splunk.exe install app path\to\my\<mycloudforwarderpackage.spl>

If it fails, or if you want to install the app manually (or tune , or prepare for a deployment server)

rename the .spl to a .tar.gz
untar the file, to a folder
copy the app folder to your /opt/splunkforwarder/etc/apps/ or C:\Program files\splunkforwarder\etc\apps (or on your deployment server and push)
restart the forwarder to apply

3 - To validate, read your forwarder /opt/splunkforwarder/var/log/splunk/splunkd.log
and test from the cloud instance that you can see the internal logs

   index=_internal host=<myforwarder> *

4- next step, setup your inputs, you can read the classic splunk inputs manuals, or use apps.
http://docs.splunk.com/Documentation/Splunk/latest/Data/WhatSplunkcanmonitor

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Wed, 13 Apr 2016 18:34:50 GMT

Thanks a ton for a quick and elaborate reply. Really helps.

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Wed, 13 Apr 2016 19:04:39 GMT

Hi @Yannk,

Had a quick question.

Below is my splunkd.log

04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - TailWatcher initializing...
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: batch://$SPLUNK_HOME/var/spool/splunk.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: batch://$SPLUNK_HOME/var/spool/splunk/...stash_new.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: monitor://$SPLUNK_HOME/etc/splunk.version.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: monitor://$SPLUNK_HOME/var/log/splunk.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: monitor://$SPLUNK_HOME/var/log/splunk/metrics.log.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: monitor://$SPLUNK_HOME/var/log/splunk/splunkd.log.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Parsing configuration stanza: monitor:///root/data.
04-13-2016 11:54:45.568 -0700 INFO  TailReader - State transitioning from 1 to 0 (initOrResume).
04-13-2016 11:54:45.568 -0700 INFO  TailReader - State transitioning from 1 to 0 (initOrResume).
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Adding watch on path: /opt/splunkforwarder/etc/splunk.version.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Adding watch on path: /opt/splunkforwarder/var/log/splunk.
04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Adding watch on path: /opt/splunkforwarder/var/spool/splunk.
**04-13-2016 11:54:45.568 -0700 INFO  TailingProcessor - Adding watch on path: /root/data**.
04-13-2016 11:54:45.568 -0700 INFO  TailReader - Registering metrics callback for: tailreader0
04-13-2016 11:54:45.568 -0700 INFO  TailReader - Starting tailreader0 thread
04-13-2016 11:54:45.569 -0700 INFO  TailReader - Registering metrics callback for: batchreader0
04-13-2016 11:54:45.570 -0700 INFO  TailReader - Starting batchreader0 thread
04-13-2016 11:54:45.571 -0700 INFO  loader - Limiting REST HTTP server to 1365 sockets
04-13-2016 11:54:45.571 -0700 INFO  loader - Limiting REST HTTP server to 1365 threads
04-13-2016 11:54:45.571 -0700 WARN  X509Verify - X509 certificate (O=SplunkUser,CN=SplunkServerDefaultCert) should not be used, as it is issued by Splunk's own default Certificate Authority (CA). This puts your Splunk instance at very high-risk of the MITM attack. Either commercial-CA-signed or self-CA-signed certificates must be used; see: 
04-13-2016 11:54:45.597 -0700 INFO  WatchedFile - File too small to check seekcrc, probably truncated.  Will re-read entire file='/opt/splunkforwarder/var/log/splunk/searchhistory.log'.
04-13-2016 11:54:45.659 -0700 ERROR TcpOutputFd - Read error. Connection reset by peer
04-13-2016 11:54:45.661 -0700 INFO  WatchedFile - Will begin reading at offset=2558565 for file='/opt/splunkforwarder/var/log/splunk/metrics.log'.
**04-13-2016 11:54:50.665 -0700 INFO  TailReader - Could not send data to output queue (parsingQueue), retrying...
04-13-2016 11:55:15.392 -0700 WARN  UserManagerPro - Can't find [distributedSearch] stanza in distsearch.conf, using default authtoken HTTP timeouts
04-13-2016 11:55:15.528 -0700 ERROR TcpOutputFd - Read error. Connection reset by peer
04-13-2016 11:55:45.527 -0700 ERROR TcpOutputFd - Read error. Connection reset by peer**

I see that the folder is monitored, but connection is getting reset. I checked out certain other answers and set sendCookedData = true. Even that didn't work. Is there something else I am missing?

Thanks,
Saravana

Re: How do I configure a universal forwarder to send data to the Splunk Cloud free trial?

sakarunanitk — Wed, 13 Apr 2016 20:39:49 GMT

I am able to telnet to the splunk host and port 9997.