https://community.splunk.com/t5/Getting-Data-In/forwarder-fails-to-start-with-permission-errors/m-p/196841#M39134 <P>For starters, being on CentOS why you chose not to install the x86_64 RPM version. The install script will create the right user/and group for you. Installing with the tarball needs manual intervention to get things right, I fail to see any benefit from side-stepping, and you may your problem rooted in this very fact. As a permissions issue this looks more like an underlying system/file-system-level problem.</P> <P>First of all, your one example prompt says "root&gt;", but does than mean you genuinely are running su? Or is it just a static prompt? Second, what is the state of the files? You say you changed permissions to 777, but that's a really awful thing to do. Show us the result of</P> <PRE><CODE>id ls -ld /opt/splunkforwarder/etc/users/users.ini </CODE></PRE> <P>Can you</P> <PRE><CODE>cat /opt/splunkforwarder/etc/users/users.ini </CODE></PRE> <P>Are you running SELinux enabled? Is it enforcing? If so (and I suspect this is probably your problem) your security context is going to be out of whack. It may be that that is disallowing the file open.</P> Thu, 12 Jun 2014 19:42:52 GMT grijhwani 2014-06-12T19:42:52Z https://community.splunk.com/t5/Getting-Data-In/forwarder-fails-to-start-with-permission-errors/m-p/196840#M39133 <P>Brand new system, new colo. <BR /> <PRE><BR /> cat /etc/redhat-release <BR /> CentOS release 6.5 (Final)<BR /> arch<BR /> x86_64</PRE></P> <P>Installing this:<BR /> <PRE><BR /> splunkforwarder-6.1.1-207789-Linux-x86_64.tgz</PRE></P> <P>After the unzip, attempting to start the forwarder:<BR /> <PRE><BR /> root&gt; /opt/splunkforwarder/bin/splunk start --accept-license --answer-yes --auto-ports --no-prompt<BR /> </PRE><BR /> Gives all these "Permission Denied" errors.<BR /> Even when changing the file:<BR /> /opt/splunkforwarder/etc/users/users.ini<BR /> to 777 it still gives this error.</P> <P>Help!</P> <PRE> ./splunk start Splunk&gt; Winning the War on Error Checking prerequisites... Checking mgmt port [8089]: open Cannot create username mapping file: /opt/splunkforwarder/etc/users/users.ini: Permission denied Cannot open file=/opt/splunkforwarder/etc/users/users.ini for parsing: Permission denied Error opening username mapping file: /opt/splunkforwarder/etc/users/users.ini New certs have been generated in '/opt/splunkforwarder/etc/auth'. ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied ERROR - Error opening "/opt/splunkforwarder/var/log/splunk/splunkd-utility.log": Permission denied Checking conf files for problems... ERROR UsernameMapper - Cannot create username mapping file: /opt/splunkforwarder/etc/users/users.ini: Permission denied ERROR IniFile - Cannot open file=/opt/splunkforwarder/etc/users/users.ini for parsing: Permission denied ERROR UsernameMapper - Error opening username mapping file: /opt/splunkforwarder/etc/users/users.ini ERROR UsernameMapper - Cannot create username mapping file: /opt/splunkforwarder/etc/users/users.ini: Permission denied ERROR IniFile - Cannot open file=/opt/splunkforwarder/etc/users/users.ini for parsing: Permission denied ERROR UsernameMapper - Error opening username mapping file: /opt/splunkforwarder/etc/users/users.ini Cannot open file to check: /opt/splunkforwarder/etc/system/local/inputs.conf Your indexes and inputs configurations are not internally consistent. For more information, run 'splunk btool check --debug' ERROR UsernameMapper - Cannot create username mapping file: /opt/splunkforwarder/etc/users/users.ini: Permission denied ERROR IniFile - Cannot open file=/opt/splunkforwarder/etc/users/users.ini for parsing: Permission denied ERROR UsernameMapper - Error opening username mapping file: /opt/splunkforwarder/etc/users/users.ini ERROR UsernameMapper - Cannot create username mapping file: /opt/splunkforwarder/etc/users/users.ini: Permission denied ERROR IniFile - Cannot open file=/opt/splunkforwarder/etc/users/users.ini for parsing: Permission denied ERROR UsernameMapper - Error opening username mapping file: /opt/splunkforwarder/etc/users/users.ini </PRE> <P>FYI: </P> <PRE> ./bin/splunk btool check --debug Checking: /opt/splunkforwarder/etc/apps/search/local/inputs.conf No spec file for: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/app.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/default-mode.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/inputs.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/limits.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/outputs.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/props.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/server.conf Checking: /opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/default/web.conf No spec file for: /opt/splunkforwarder/etc/apps/introspection_generator_addon/default/app.conf Checking: /opt/splunkforwarder/etc/apps/introspection_generator_addon/default/inputs.conf Checking: /opt/splunkforwarder/etc/apps/introspection_generator_addon/default/server.conf No spec file for: /opt/splunkforwarder/etc/apps/search/default/app.conf Checking: /opt/splunkforwarder/etc/apps/search/default/props.conf Checking: /opt/splunkforwarder/etc/apps/search/default/restmap.conf Checking: /opt/splunkforwarder/etc/system/default/alert_actions.conf No spec file for: /opt/splunkforwarder/etc/system/default/app.conf Checking: /opt/splunkforwarder/etc/system/default/audit.conf Checking: /opt/splunkforwarder/etc/system/default/authentication.conf Checking: /opt/splunkforwarder/etc/system/default/authorize.conf No spec file for: /opt/splunkforwarder/etc/system/default/conf.conf Checking: /opt/splunkforwarder/etc/system/default/default-mode.conf Checking: /opt/splunkforwarder/etc/system/default/inputs.conf Checking: /opt/splunkforwarder/etc/system/default/limits.conf Checking: /opt/splunkforwarder/etc/system/default/outputs.conf Checking: /opt/splunkforwarder/etc/system/default/procmon-filters.conf Checking: /opt/splunkforwarder/etc/system/default/props.conf Checking: /opt/splunkforwarder/etc/system/default/restmap.conf Checking: /opt/splunkforwarder/etc/system/default/server.conf Checking: /opt/splunkforwarder/etc/system/default/source-classifier.conf Checking: /opt/splunkforwarder/etc/system/default/web.conf Checking: /opt/splunkforwarder/etc/system/local/inputs.conf Checking: /opt/splunkforwarder/etc/system/local/server.conf </PRE> Thu, 12 Jun 2014 03:34:08 GMT https://community.splunk.com/t5/Getting-Data-In/forwarder-fails-to-start-with-permission-errors/m-p/196840#M39133 swissarmychains 2014-06-12T03:34:08Z https://community.splunk.com/t5/Getting-Data-In/forwarder-fails-to-start-with-permission-errors/m-p/196841#M39134 <P>For starters, being on CentOS why you chose not to install the x86_64 RPM version. The install script will create the right user/and group for you. Installing with the tarball needs manual intervention to get things right, I fail to see any benefit from side-stepping, and you may your problem rooted in this very fact. As a permissions issue this looks more like an underlying system/file-system-level problem.</P> <P>First of all, your one example prompt says "root&gt;", but does than mean you genuinely are running su? Or is it just a static prompt? Second, what is the state of the files? You say you changed permissions to 777, but that's a really awful thing to do. Show us the result of</P> <PRE><CODE>id ls -ld /opt/splunkforwarder/etc/users/users.ini </CODE></PRE> <P>Can you</P> <PRE><CODE>cat /opt/splunkforwarder/etc/users/users.ini </CODE></PRE> <P>Are you running SELinux enabled? Is it enforcing? If so (and I suspect this is probably your problem) your security context is going to be out of whack. It may be that that is disallowing the file open.</P> Thu, 12 Jun 2014 19:42:52 GMT https://community.splunk.com/t5/Getting-Data-In/forwarder-fails-to-start-with-permission-errors/m-p/196841#M39134 grijhwani 2014-06-12T19:42:52Z