All Apps and Add-ons

Splunk App for Stream Installation: Missing directories, wire data input and Splunk is running as root

Path Finder

Hi -
I have tried installing the Splunk App for Stream on 2 different Splunk servers(ubuntu 14.04 x86_64) and the experience is the same. I follow install directions: http://docs.splunk.com/Documentation/StreamApp/latest/DeployStreamApp/InstallSplunkAppforStream

  • Splunk is running as root.
  • Directories $SPLUNK_HOME/etc/apps/Splunk_TA_stream and $SPLUNK_HOME/etc/deployment-apps/Splunk_TA_stream are not created.
  • The Wire Data input is not listed.

Any help is appreciated.

Thanks,
Joe

Tags (1)

Splunk Employee
Splunk Employee

Splunk App for Stream 6.0.1 has been released! This build fixes several problems regarding the initial configuration of the wire data input. You can download it here:

http://apps.splunk.com/app/1809/

Release notes here:

http://docs.splunk.com/Documentation/StreamApp/6.0.1/ReleaseNotes/FixedProblems

Please let me know if you experience any problems with the wire data input using version 6.0.1.

Thanks,

-Mike

Path Finder

stream installer log

[root@splunk splunk]# cat stream_installer.log
2015-02-12 16:20:53,667 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-12 16:24:11,975 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-16 10:31:14,928 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-16 14:46:10,484 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-17 10:25:51,415 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-17 14:50:41,790 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-17 15:02:53,880 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-17 15:05:35,691 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-17 15:25:11,921 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-18 15:41:06,928 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-19 09:23:44,605 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-19 15:19:53,318 [INFO] Splunk App for Stream Dependency Manager: Starting...
2015-02-19 15:23:15,402 [INFO] Splunk App for Stream Dependency Manager: Starting...

[root@splunk splunk]#

0 Karma

Path Finder

while adding streamfwd from data inputs wire data

Encountered the following error while trying to save: In handler 'streamfwd': The script returned with exit status 2.

0 Karma

Splunk Employee
Splunk Employee

I encountered the same error ( Encountered the following error while trying to save: In handler 'streamfwd': The script returned with exit status 2.) when trying to edit the modular input streamfwd (more settings) to set it to another index.
splunkd showed this:
08-18-2015 23:57:46.463 -0700 ERROR ModularInputs - Argument validation for scheme=streamfwd: killing process, because executing it took too long (over 30000 msecs).
08-18-2015 23:57:46.465 -0700 INFO ModularInputs - Argument validation for scheme=streamfwd: script running failed (killed by signal 9: Killed: 9).

I initially untarred the splunk_app_stream.tar file and copied it into etc/apps/ and restarted splunk for the first install which led me to the error.

To fix the issue I removed the Splunk_TA_stream and the splunk_app_stream , restarted splunk then installed from the web UI under Apps>Find More Apps . I then enabled the modular input through the web UI (Settings>Data Inputs> Wire Data > streamfwd - enable.

To see http data I went to the Splunk App for Stream from the app menu and enabled the http protocol. Then did a search for index=* source=stream* and see data now.

My system was a standalone server so fwdr/SH/IDX all in one.

0 Karma

Path Finder

[root@splunk Splunk_TA_stream]# more streamfwd.log
2015-02-19 15:37:48 INFO 140253906425664 stream.CaptureServer - Found DataDirecto
ry: /opt/splunk/etc/apps/Splunk_TA_stream/data
2015-02-19 15:37:48 INFO 140253906425664 stream.CaptureServer - Found UIDirectory
: /opt/splunk/etc/apps/Splunk_TA_stream/ui
2015-02-19 15:37:48 INFO 140608789518144 stream.CaptureServer - Found DataDirecto
ry: /opt/splunk/etc/apps/Splunk_TA_stream/data
2015-02-19 15:37:48 INFO 140608789518144 stream.CaptureServer - Found UIDirectory
: /opt/splunk/etc/apps/Splunk_TA_stream/ui
2015-02-19 15:37:48 INFO 140608789518144 stream.CaptureServer - Loaded configurat
ion file: /opt/splunk/etc/apps/Splunk_TA_stream/local/streamfwd.xml
2015-02-19 15:37:48 ERROR 140608789518144 stream.CaptureServer - Unable to ping s
erver (b37e1dc2-6377-4308-8556-02d2da6543ca): Unable to establish connection to localhost: Connection refuse
d

0 Karma

Builder

Had the same problem too. Referred to Before You Deploy > Deployment Requirements section of DeployStreamApp documentation and found that Splunk instances on Windows are not supported. Also noted that non-enterprise versions of Splunk are also not supported.

0 Karma

Splunk Employee
Splunk Employee

Hi Joe,

Did you install using Splunk's web interface or by just uncompressing the file into $SPLUNK_HOME/etc/apps? You do need to restart your splunk server for it to create the Splunk_TA_stream directory, an add-on that provides the Wire data input. The web UI prompts to do this for you, but installing via command line requires a manual restart using "$SPLUNK_HOME/bin/splunk restart". The script that does this should also create a log file $SPLUNK_HOME/var/log/splunk/stream_installer.log; if there is a problem it may indicate what is wrong.

If all else fails, you can also just manually copy the contents from $SPLUNK_HOME/etc/apps/splunk_app_stream/install/Splunk_TA_stream into $SPLUNK_HOME/etc/apps/Splunk_TA_stream and restart splunk. It should definitely pick things up after that.

Take care,

-Mike

Path Finder

ty@mdickey_splunk

Yes, I had the App for Unix installed. I had tried going through the documentation in the link you refer to "before" finding this thread. I ended up wrestling it with a few more hours.

Copied files per instructions didn't work.
Removing / deleting Unix app didn't work.
Reinstalled stream app. Copied directory manually. / worked

Thanks again.

Splunk Employee
Splunk Employee

@shandman I'm sorry you are experiencing problems with this. Do you have App for Unix installed and see similar errors in your splunkd.log file about dependency_manager.py? If so then the step-by-step instructions documented in the troubleshooting guide (http://docs.splunk.com/Documentation/StreamApp/6.0/DeployStreamApp/Troubleshooting#Splunk_TA_stream_...) should fix the problem. We've also updated the installation documentation to refer to this troubleshooting article, and plan to have a new release including the fix soon.

Path Finder

i didn't have the app for unix installed and still got the same problems

0 Karma

Path Finder

Having the same issue. Have spent hours on this. Would sure be nice to get a simple documented process to fix this.

0 Karma

Path Finder

@mdickey_splunk

thanks for the help. i finally (almost) got it to work. i'm now seeing the streamfwd logs. and i saw this message: "No capture devices found (must be root/Administrator)"
better contact our sysadmins to give streamfwd root access 🙂

0 Karma

Splunk Employee
Splunk Employee

@rizzo75, thanks for that post. I just reproduced and can confirm we seem to have a bug in 6.0.0 where the depedency_manager.py script (which deploys Splunk_TA_stream) conflicts with a similarly-named script provided by the App for Unix.

You can work-around this by manually copying Splunk_TA_stream from $SPLUNK_HOME/etc/apps/splunk_app_stream/install/Splunk_TA_stream into $SPLUNK_HOME/etc/apps/.

Please note that this script also creates the default "streamfwd" Wire data input, so when you manually copy the TA directory, you will also need to create a new Wire data input using the Splunk UI. Normally, you should be able to just cut and paste the example URL into the single-field input box.

Path Finder

I also manually copied the Splunk_TA_stream directory to the apps directory and did not see a streamfwd input.

Joe

0 Karma

Path Finder

me too. do not see a streamfwd input

0 Karma

Path Finder

P.S. I still don't see "streamfwd" in the Wire Data settings page but I just ran the query -> sourcetype="stream:http"

and i'm now seeing HTTP data stream! awesome! 🙂

0 Karma

Path Finder

Thanks for the response.

I installed the app via the web interface.

$SPLUNK_HOME/var/log/splunk/stream_installer.log does not exist.

I just tried installing from the command line with the same results.

I do notice this in the splunkd.log: http://pastebin.com/MDuHXWqK

Thanks,
Joe

0 Karma

Path Finder

Mike,

I manually copied Splunk_TA_stream directory into $SPLUNK_HOME/etc/apps and i'm now able to see the "Wire data" option in the Data Inputs. Nice!

However, when I click "Wire Data" I don't see "streamfwd" in the list. In fact, there are no items displayed on the Data Inputs > Wire Data page.

I also checked $SPLUNK_HOME/var/log/splunk/stream_installer.log but this file does not exist in the log directory.

Any ideas what I'm missing? Thanks!

0 Karma

Path Finder

I've encountered the same issue. It's also interesting to note that Splunk_TA_stream was installed under $SPLUNK_HOME/etc/apps/splunk_app_stream/install/ directory.

0 Karma