Universal forwarder fails to install credentials package on Linux (RPM / tgz)

[nilseupadilha]

I followed the procedures in Get Data In tutorial for an Amazon Linux.

Tried both the RPM and tar ball and got he same errors when enabling the credentials file at the same point:

[ec2-user@ip-10-10-29-187 bin]$ ./splunk start

[ec2-user@ip-10-10-29-187 bin]$ ./splunk edit user admin -password PWDPWD -auth admin:changeme
User admin edited.

[ec2-user@ip-10-10-29-187 bin]$ ./splunk install app /home/ec2-user/splunkforwarder/splunkclouduf.spl -auth admin:PWDPWD
Error during app install: failed to extract app from /home/ec2-user/splunkforwarder/splunkclouduf.spl to /home/ec2-user/splunkforwarder/splunkforwarder/var/run/splunk/bundle_tmp/eb859b5b37e018d6: No such file or directory

[ec2-user@ip-10-10-29-187 bin]$ sudo ./splunk install app /home/ec2-user/splunkforwarder/splunkclouduf.spl -auth admin:PWDPWD
Error during app install: failed to extract app from /home/ec2-user/splunkforwarder/splunkclouduf.spl to /home/ec2-user/splunkforwarder/splunkforwarder/var/run/splunk/bundle_tmp/c09d0c1b2a0339b3: No such file or directory

Is there another universal forwarder for linux version for download?

[randomusername1]

This is the answer:
You cannot use wget to fetch the file inside ur linux as it will indeed download an error html page!!!
Bets practice is to download the splunkclouduf.spl with your browser and transfer it to ur nix box using ssh or scp or whatever u use...

[randomusername1]

This is the answer:
You cannot use wget to fetch the file inside ur linux as it will indeed download an error html page!!!
Bets practice is to download the splunkclouduf.spl with your browser and transfer it to ur nix box using ssh or scp or whatever u use...

[ernestmueller]

After fighting with this a while getting this exact same error message, I found out trying to wget that credential file was the problem, I had to download it and scp it and make sure it stayed binary and check md5sums, but once I did it went from

"Error during app install: failed to extract app from /opt/splunkforwarder/splunkclouduf.spl to /opt/splunkforwarder/var/run/splunk/bundle_tmp/d655161de6f2af02: No such file or directory"

to

"App '/opt/splunkforwarder/splunkclouduf.spl' installed "

[jethompson_splu]

Hello nilseupadiha,

I would first like to know which Amazon AMI you used? The reason for asking this is I can see that the user listed from the errors provided is the EC2 User which suggests the use of the Amazon Linux AMI. Depending on which Linux AMI was used will depend on how you will need to move forward with this process. This is most likely a Permissions issue with the Splunk installation directory and the EC-2 User Account.

One thing that I would like to point out is with the Amazon Linux AMI this is not really CentOS, its actually based off of Fedora and this can be seen by running the following command sets:

cat /etc/*-release
or
lsb_release -a

The following are examples of the printouts from a CentOS AMI from Amazon:

[root@jthompsonlin03 ~]# cat /etc/*-release
CentOS Linux release 7.2.1511 (Core)
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

CentOS Linux release 7.2.1511 (Core)
CentOS Linux release 7.2.1511 (Core)

[root@jthompsonlin03 ~]# lsb_release -a
LSB Version:    :core-4.1-amd64:core-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.2.1511 (Core)
Release:    7.2.1511
Codename:   Core

Outside of the permissions issue that you appear to be experiencing there may be other issues experienced when using the Amazon Linux AMI as again that is not actually based on CentOS, its based off of Fedora and there are enough differences between the 2 that there could be further issues experienced.

One thing that you could do for testing to verify, would be to try starting Splunk as Root and re-attempt the process that you are attempting. If you are able to complete the process at that time I would recommend double checking the Permissions of the Directories listed in the Error printed to the screen and correct them based on the User Account that Splunk is running as. For my systems I run Splunk as the Splunk User Account.

[nilseupadilha]

Running as root I obtained the same results:

[root@ip-10-10-29-1 bin]# ./splunk install app /home/ec2-user/splunkforwarder/splunkclouduf.spl -auth admin:clarivate
Error during app install: failed to extract app from /home/ec2-user/splunkforwarder/splunkclouduf.spl to /home/ec2-user/splunkforwarder/splunkforwarder/var/run/splunk/bundle_tmp/3d2b194105ac894d: No such file or directory
[root@ip-10-10-29-187 bin]#

[jethompson_splu]

Hello nilseupadiha,

Thank you for the information in regards to which AMI was used. Please understand that as I advised in my previous post the Amazon Linux AMI is NOT CentOS/RHEL, it is based on Fedora and as such not directly supported by Splunk. I would recommend using the Amazon CentOS AMI for your Splunk Environment.

Now with that being said in an attempt to further assist you, have you looked at the User Account that the Splunk Service is running as? If this is anything other than the EC-2 User Account that you were using to start the service then you will need to make sure that you have supplied that User with the needed permissions on the EC-2 Users Home Directory.

Alternatively you could also attempt to stop Splunk, then Start Splunk while logged in as Root, and re-attempt the process that you are. What I am suspecting from the information available is that the Splunk Service is running as a different User Account than that which you used to start the Splunk Service.

[oozzal]

@jethompson Looks like this is a known issue in the latest version of splunk binary. I'm able to replicate it on Ubuntu 14.04 even though /opt/splunkforwarder has root permissions and I tried to install as a root user.

[nilseupadilha]

Hey Thompson,

tks for your diligent support.

Unfortunately, we wont be able to change to CentOS. We have a few thousands of instances already running on Amazon Linux that supports dozens of microservices.

I'll double check the alternatives.

[nilseupadilha]

Tks Thompson.

the last output i obtained by extracting the plain tar ball and running directly from logged user home folder. The directories where created under logged user.

Previously i tried the RPM version, that created the user splunk for me, which i logged in and performed the installation procedures. The directories created by splunk forwarder start (/opt/splunkforwarder/var) was created under splunk ownership user too.

I stopped at the same point.

Here is the output from my current AMI:

[ec2-user@ip-10-10-29-1 ~]$ cat /etc/*-release
NAME="Amazon Linux AMI"
VERSION="2017.03"
ID="amzn"
ID_LIKE="rhel fedora"
VERSION_ID="2017.03"
PRETTY_NAME="Amazon Linux AMI 2017.03"
ANSI_COLOR="0;33"
CPE_NAME="cpe:/o:amazon:linux:2017.03:ga"
HOME_URL="http://aws.amazon.com/amazon-linux-ami/"
Amazon Linux AMI release 2017.03