My apps running in docker containers currently use 8.2.9 splunk univeral forwarder that works fine. My image are based on linux alpine image.
I have for some time been trying to get 9.x.x UF working instead but I cannot get it to work. When it boots, it prints the following error:
"/opt/splunkforwarder/bin/splunk" start --accept-license --answer-yes --no-prompt
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk:splunk /opt/splunkforwarder"
This appears to be your first time running this version of Splunk.
Creating unit file...
Error calling execve(): No such file or directory
Error launching command: No such file or directory
Failed to create the unit file. Please do it manually later.
Splunk> The Notorious B.I.G. D.A.T.A.
Checking prerequisites...
Checking mgmt port [8089]: open
Creating: /opt/splunkforwarder/var/lib/splunk
Creating: /opt/splunkforwarder/var/run/splunk
Creating: /opt/splunkforwarder/var/run/splunk/appserver/i18n
Creating: /opt/splunkforwarder/var/run/splunk/appserver/modules/static/css
Creating: /opt/splunkforwarder/var/run/splunk/upload
Creating: /opt/splunkforwarder/var/run/splunk/search_telemetry
Creating: /opt/splunkforwarder/var/run/splunk/search_log
Creating: /opt/splunkforwarder/var/spool/splunk
Creating: /opt/splunkforwarder/var/spool/dirmoncache
Creating: /opt/splunkforwarder/var/lib/splunk/authDb
Creating: /opt/splunkforwarder/var/lib/splunk/hashDb
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from '/opt/splunkforwarder/splunkforwarder-9.1.2-b6b9c8185839-linux-2.6-x86_64-manifest'
All installed files intact.
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
PYTHONHTTPSVERIFY is set to 0 in splunk-launch.conf disabling certificate validation for the httplib and urllib libraries shipped with the embedded Python interpreter; must be set to "1" for increased security
However it seems to start a background process but I dont see the logs in splunk. Using the status command kills the background process:
"/opt/splunkforwarder/bin/splunk" status
Warning: Attempting to revert the SPLUNK_HOME ownership
Warning: Executing "chown -R splunk:splunk /opt/splunkforwarder"
splunkd 165 was not running.
Stopping splunk helpers...
I have tried disabling boot start but that gives me a similar error::
"/opt/splunkforwarder/bin/splunk" disable boot-start
Error calling execve(): No such file or directory
Error launching command: No such file or directory
execve: No such file or directory
while running command /sbin/chkconfig
After researching this, I think it could be related to systemd perhaps? I dont think Alpine includes it but it uses OpenRC instead. However, I dont really have any use for this autostart feature anyway, is there a way to ignore/skip it somehow?
