Getting Data In

slave-apps doesn't like my app

vanvan
Path Finder

Hi,

I built a simple app with the add-on builder (using the python script inputs) and then copied it in the "master-apps" of my cluster master.

After successful bundle replication I see the app is on the indexers as well, but I noticed some strange things immediately:

  1. The icons used in the app were missing from the GUI and were replaced by the default "app" icon.
  2. When I tried to access the app from the menu on the indexers, I got only a blank screen with only "Splunk>" logo in it...
  3. The _internal logs were showing that my custom app is missing some python scripts...

I already checked app sizes on Master and Peers and it is the same everywhere. No files are missing from the app...

Any ideas why this might happen? Some strange limitations on "slave-apps" content maybe?

0 Karma

vanvan
Path Finder

Ok, I have a workaround that seems to solve the issue...but it's dirty.

The root cause is that splunkd does not search for any scripts,icons,css,whatever resources when apps are under "slave-apps". This is how it works currently, I don't know why. If you create a symbolic link with the same name under "apps" directory the problem will disappear.

E.g. you have an app, called "TA-myapp" and you deploy it through the Cluster Master to "slave-apps" on the Cluster Peers. The app path would be something like "/opt/splunk/etc/slave-apps/TA-myapp". In order to solve the issue with python scripts not found, simply run the command:
ln -s /opt/splunk/etc/slave-apps/TA-myapp /opt/splunk/etc/apps/TA-myapp

This will create a symbolic link under "/opt/splunk/etc/apps" that points to the app under "slave-apps" and then Splunkd would be able to find the missing resource. This will fix also issues with missing icons (e.g. like in the case of some other add-ons like TA for PaloAlto).

atownson
Explorer

I can confirm the same behavior with such apps as config_explorer. Thanks for the workaround.

0 Karma

woodcock
Esteemed Legend

Are you using Deployment Server and is it running WIndows OS?

0 Karma

vanvan
Path Finder

No,

I am using the Cluster Master (with its master-apps directory) and it is running on SuSE Linux.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Don't deploy scripted inputs to indexer clusters. Each indexer will run the script(s) and you'll end up with duplicated data.
Instead, deploy the app to a heavy forwarder.

---
If this reply helps you, Karma would be appreciated.

richgalloway
SplunkTrust
SplunkTrust

Yes, it is related to the issue you are having. If you weren't trying to do something that shouldn't be done, you wouldn't have the issue.

---
If this reply helps you, Karma would be appreciated.
0 Karma

vanvan
Path Finder

Ok 🙂

I am trying to run scripts on all the indexers in order to do some housekeeping locally on the machine. The resulting output is collected as events, which are relatable only to the local machine, e.g. they are not "duplicate" in any sense whatsoever. Kinda like the stuff in "_internal", you know?

If the peers weren't in a cluster I would've used the Deployment Server and probably it would've worked, but since the peers are part of an indexer cluster...guess I'm stuck with the master-apps method.
Which gives me the issue...

What you are talking about is perfectly fine, but doesn't really explain why it doesn't/shouldn't work in my case.

0 Karma

vanvan
Path Finder

Ok, thanks for the feedback, although it isn't related to the issue, I'm afraid.

I think I found something else though... The permissions look very different when doing "ls -al" on master-apps compared to the slave-apps... Maybe that's the reason.

0 Karma

vanvan
Path Finder

...but it isn't...Still getting the same results, even when the permissions are 100% same...

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...