Getting Data In
Highlighted

Scripted input permission denied

Builder

Hello. I have a HF and I want it to download a .csv file from another internal server. Right now, I can download it as the splunk user using wget on CLI so I know connectivity and permissions are no issue. I looked at scripted inputs but I don't think that's the right way about it, as I can't get it to work. Not sure how to go about this?

I just want to download a csv file and then send it to my indexer tier.

/opt/splunk/etc/apps/my_app/bin/script.sh

/usr/bin/wget -O file.csv 'https://myserver.com/feeds/list?v=csv&f=indicator&tr=1'
exit 0

/opt/splunk/etc/apps/my_app/local/inputs.conf

[script://./bin/script.sh]
index = main
sourcetype = test
interval = 600.0
disabled = 0

splunkd.log

07-10-2019 00:08:18.082 +0000 ERROR ExecProcessor - message from "/opt/splunk/etc/apps/my_app/bin/script.sh" file.csv: Permission denied

I checked and the app is owned by splunk:splunk. The script is 755. I ran the ad-hoc command below as the splunk user and it downloaded the file just fine

/opt/splunk/bin/splunk cmd ../etc/apps/my_app/bin/script.sh

I tried adding the input through the HF's gui (Settings > Data Inputs > Scripts > Add new) but my app and script are not showing up in the dropdown...

alt text

0 Karma
Highlighted

Re: Scripted input permission denied

Builder

can you check the level of permission on "script.sh". does it have execute permissions?

0 Karma
Highlighted

Re: Scripted input permission denied

Builder
-rwxr-xr-x. 1 splunk splunk  217 Jul  9 23:35 script.sh
0 Karma
Highlighted

Re: Scripted input permission denied

SplunkTrust
SplunkTrust

Hi DEAD_BEEF,

it's not the script that is the issue here, it's the output file file.csv AND it's location the script is trying to create it. Set the output file to use a full path that you are sure the user splunk can write into.

Hope this helps ...

cheers, MuS

View solution in original post

0 Karma
Highlighted

Re: Scripted input permission denied

Builder

If I just want splunk to read the stream and then send it to indexers rather than download a log file and then send it, how would I adjust the script?

0 Karma
Highlighted

Re: Scripted input permission denied

SplunkTrust
SplunkTrust

No -O, everything that goes to stdout will be indexed by Splunk when running scripted/modular inputs.

cheers, MuS

0 Karma
Highlighted

Re: Scripted input permission denied

Builder

So I put a full path /var/log/file.txt and now the file is saved on the HF, but nothing was indexed...

0 Karma
Highlighted

Re: Scripted input permission denied

Motivator

Splunk can index whatever is returned to standard out as a scripted input.

wget -O - 'https://myserver.com/feeds/list?v=csv&f=indicator&tr=1' 2> /dev/null

or, if you want to redirect standard error output also:

wget -O - 'https://myserver.com/feeds/list?v=csv&f=indicator&tr=1'  2>&1
0 Karma
Highlighted

Re: Scripted input permission denied

Builder

@rob_jordan I changed the script to just be the 1-liner that you put (as I didn't know how to send data to stdout). Now it shows this and no data in index=main

07-10-2019 02:07:56.729 +0000 ERROR ExecProcessor - message from "/opt/splunk/etc/apps/my_app/bin/script.sh" 2019-07-10 02:07:56 (8.61 MB/s) - written to stdout [4011976]
0 Karma
Highlighted

Re: Scripted input permission denied

Motivator

Good point -O file.csv possibly should have a specific path before the file name (where the file should be written) unless preceded with a cd some dir command

0 Karma