Can a Raspberry Pi that acts a universal forwarder be controlled from a splunk server?

[teodor1746]

Hello

I have a RPi 4 at home running Raspbian and I have the universal forwarder installed on it and logging data to be sent to the Splunk server on my VM.
My question is would it be possible to control said raspberry pi from the server itself? By control I mean send a command or a script to the RPi that would change the current directory or something similar. I have a few apps running on the RPi and I would like to shutdown/restart etc. them from the Splunk server, without needing to manually log into the Pi itself.

Thank you taking the time read this and I apologise if I didn't include enough details, it is my first question.

[PavelP]

By control I mean send a command or a script to the RPi that would change the current directory or something similar. I have a few apps running on the RPi and I would like to shutdown/restart etc. them from the Splunk server, without needing to manually log into the Pi itself.

For a small deployment is is better to follow KISS principle and do not introduce unnecessary complexety (unless you want to learn). If you already logged on your splunk server, why not just use ssh/scp to control your RPi? You can configure ssh keys and aliases and perform desired tasks without typing password.

[teodor1746]

Thank you for the advice, I probably will as this more for me to learn and kind of be a proof of concept if it works. I have one working RPi now, but I plan to add 2 more and would like to control them from the splunk server if possible.

[darrenfuller]

Hi Teodor1746,

You could use your main Splunk server (or another Splunk installation) as a deployment server, then you can send new apps or updates to apps to your Raspberry Pi. To do this... you need to do this:

1) on your raspberry pi, create a deploymentclient.conf file either in an app (in $SPLUNK_HOME/etc/apps/[appname]/local) or in $SPLUNK_HOME/etc/system/local. the file should look like this:

[deploymentclient]
disabled = false
phoneHomeIntervalInSecs = 60 

[target-broker:deploymentServer]
targetUri = hostname.or.ip.of.splunk.server:8089

Once that is in place, restart the forwarder on the raspberry Pi. To see that it is completed successfully, Login to your Splunk UI, and open Settings -> Forwarder Management. You should see your pi box on the list of clients.

Now, to send new configs to the pi, you create apps on the splunk server under the $SPLUNK_HOME/etc/deployment-apps directory with the configs that you want to send.

- Scripts should be located in $SPLUNKHOME/etc/deployment-apps/appname/bin directory
- Inputs to run those scripts are contained in $SPLUNKHOME/etc/deployment-apps/appname/local/inputs.conf

inputs.conf example for a script called foo.py:

[script://./bin/foo.py]
disabled = false 
interval = 300
index = main 

When the app is ready to deploy to the pi...
- go back to settings -> forwarder management
- click on the server classes tab
- Click add a new server class
- on the clients side, add your raspberry pi host
- on the apps side, add the app or apps that you want to send to the pi
- CLick save

On the next poll from the pi, it will notice there is a new app to deploy to it, pull it from the deployment server and then install it on the pi.

Oh, and make sure when you are creating the server class you click "Restart splunkd", otherwise scripts will be copied to the box and the splunk process won't restart, so it won't start running anything until the next time it is restarted..which defeats the purpose.

More information on deployment servers can be found in the docs at https://docs.splunk.com/Documentation/Splunk/latest/Updating/Deploymentserverarchitecture

Hope this helps...
./D

[teodor1746]

This worked great, thank you for the quick and detailed answer. I managed to set up the app and the RPi downloaded the script that was in it (a simple script that creates a directory).

But I cannot get it to actually execute as I am getting a "Permission Denied" error message, regardless of which user I try to run it from (root, splunk, pi etc). I went to $SPLUNKHOME/etc/apps/app-name/bin and tried to execute it from there.

I also used "chown -R splunk /opt/splunkforwarder" to change the ownership to the splunk user but to no avail.