Permission Issues when dealing with Splunk_TA_nix scripts

[Jarohnimo]

Hello All,

I have a solid understanding of the files/ how to deploy this application but my issue is with permissions.
We have 4 brand New Linux Splunk Instances, each instance is running splunk as splunk per best practice. sh, 2 indexers, 1 uf

I could use some pointers on how to properly deal with Linux TA nix with respects to permissions. The source of my problem could be how I'm executing/ copying, moving files around while logged into the Linux machines. So I'm interested in know how you guys are doing things.

This is what I'm doing:
1) I log in to the machine as bob, bob has sudo permissions
2) anytime I need to move a file/ directory onto the Linux box i Filezilla it over from my windows machine to my home directory first /home/bob

From there I'll copy the file into the splunk instance: /opt/splunk/etc/apps

HOWEVER.... in order for me to copy the files into the splunk directories, I have to sudo cp -R the files there as my normal user account didn't have permissions over the splunk directories (as they are owned by splunk). I wasn't able to copy the files via splunk account as it doesn't have permissions in /home/bob. So what's the right way to do things?

Should i grant splunk access to /home/bob so it can grab the files and move it to the proper destination? Where i don't have to chown and chmod? Should i not be moving files into /home/bob? OR something else?

As a workaround Id' then chown the files back to Splunk and things would work properly.

I noticed this is what the permission looked like on Splunk_TA_nix drwx------.
however on all other applications, I've installed they have drwxr-xr-x.

When I searched _internal logs to investigate, it stated none of the servers had permissions to execute any *.sh files

These issues that were consistent with both local Splunk_TA_nix and when the app is deployed via deployment server so obviously I'm doing something wrong

As a workaround, I chmod 775 -R Splunk_TA_nix after the files were deployed. I would also sudo chmod 775 -R Splunk_TA_nix locally on the search head and this would also work. However, I'd like to know the proper way to deploy Splunk_TA_nix where there isn't any permission issue running the scripts.

Please consider how I'm transferring, login in, sudo'n, copying the files etc. Thanks for your direction!

[Roy_9]

yes @shocko  i have manually set the executable permissions for the ones that are missing and it worked.

[woodcock · Accepted solution]

File permissions are all over the place on Splunk apps. You need to decide what you think is best (within reason, obviously executables need some +x) and you should adjust to your standard every time you download any app. Having said that, try to avoid ANY use of windows except for UFs. Also, seriously consider using a Deployment Server to make this all waaaaaay easier. Or you can use any other orchestration/distribution tool like puppet or chef (there is a ton of stuff on github for this). I would definitely not continue to do this distribution by hand, even if you have a tool that makes it easier like tanium.

[Jarohnimo]

Hi woodcock thanks for responding. I wrote a lot so you may have missed I'm using a deployment server to distribute (also need to install the app locally /etc/apps) for one server.

The issue is the permissions on the directory is this drwx------

However I noted the permissions need to
Be drwxr-xr-x.

I chmod after the deployment distributes the apps to the client in order to fix however I know this isn't the ideal way to manage. It's not ideal to login and mod all your clients

What have you done in your environment to get this to work? Mod the permissions prior to deploying does that work? Just trying to figure out what's the best route..

Also I'm not really using Windows other than downloading the files and filezilla them to the Linux machine. If I'm doing something in touching windows at all let me know

[woodcock]

The only reason that the permissions would be messed up is if they are messed up on the DS (if so, fix them there) or if you are using a Windows-based DS which is a known limitation and a terrible idea for many reasons. If your Splunk architecture is on Windows, then trash it and start over.

[Roy_9]

@woodcock  Hi, i am getting the permissions errors where it the execute bit is nor getting replicated to shc even though they are set to executable on deployers. do you have any idea?

[woodcock]

You are using Windows OS for your DS and deploying to non-Windows servers.  This cannot be made to work.  Start over and use a Linxus OS for your DS.

[Jarohnimo]

It's all Linux, I feel splunk should specify in their instructions the need to chmod apps. For this one /Splunk_TA_nix/bin needed to be modded 644 on apps and my deployment apps. Them the scripts would execute correctly when deployed. I know plenty who actually wrote scripts to modify the bin permissions of the deployed app. (Talk about alot of mess).
Im good to go now but I'm hoping this information will help someone else who maybe struggling with this.

I didn't see any verbage about modifying permissions anywhere in he instructions so it was a hidden surprise when it didn't work.

[shocko]

Same issue here. Bottom line is Linux will not create a file with the execute bit set for obvious security reasons. That said, I would have expected the forwarder to have some mechanism to do this for apps coming from deployment server. We have the same issue and are simply running  cron job to set the permission. I really don't like this though as it's not centrally controlled and we don't have chef/puppet or ansible. 

[michael_mcgrail]

I'm having this same issue - chmod a deployment app to 775 on the Linux DS, but when a Linux UF pulls the app it defaults to 700. Did I miss something in configuration where Linux to Linux doesn't retain permissions?

Thanks,
Mike

[Jarohnimo]

I have a lot more knowledge of splunk now since when I first posted this question. It's very important to run and do things as splunk... The last few times I deployed splunk TA Nix I had no issues with permissions nor had the need to modify. Ensure your doing everything humanly possible as splunk, not as root, not as your user but as splunk. Things generally work a lot better.

Chmod 644 was almost a work around as I don't know what caused the permissions to be out of wack in the first place...

[shocko]

In my experience if DS is restarted the app might not need any permissions changed it it has not changed but otherwise it will. 

[Roy_9]

Is there any script or playbook that can help in adding executible permissions for particular scripts for nix app?

[shocko]

In our case we have some CRON scripts and subsequently Ansible playbooks to do this but essentialy it is just doing +X on the permission or FACLS of the script files. 

[shocko]

Same here but we use Ansbile.