topic Re: Why my indexation doesn't work ? in Dashboards & Visualizations

Why my indexation doesn't work ?

mah — Wed, 26 Aug 2020 07:45:17 GMT

Hi,

I can't see where I am wrong in my configuration file :

inputs.conf : /opt/splunk/etc/apps/my_app_poller/local

[script://./bin/my_python_script.py]
interval = 27 7 * * *
index = my_index
sourcetype = script:python
source = script://./bin/my_python_script.py
disabled = 0

[batch:///opt/splunk/etc/apps/my_app_poller/bin/*my_python_script.json]
move_policy = sinkhole
index = my_index
sourcetype = script:python
crcSalt = <SOURCE>
disabled = 0

props.conf : /opt/splunk/etc/apps/my_app_2/default
[script:python]
INDEXED_EXTRACTIONS = json
DATETIME_CONFIG = CURRENT
TRUNCATE = 999999
JSON_TRIM_BRACES_IN_ARRAY_NAMES = true

Re: Why my indexation doesn't work ?

gcusello — Wed, 26 Aug 2020 08:26:03 GMT

Hi @mah,

the first dubt I usually have using scripts is the grants on the my_python_script.py file, did you already checked them?

you could try to run the script outside Splunk.

Then (but it's a different thing), why did you used batch?

batch is used for large archives of historic data. If you want to continuously monitor a directory or index small archives, use "monitor".

If these files are the output of the script of the previous stanza, you don't need this stanza, if instead they are different files, use monitor.

Then, from your script stanza I see that you want to run this script once a day (at 07:27:00), is it correct?

An additional test: run the script outside splunk as Linux command, verify if it's runned and redirect the output on a text file, then ingest it via web GUI, so you can test the props.conf.

Ciao.

Giuseppe

Re: Why my indexation doesn't work ?

mah — Wed, 26 Aug 2020 09:03:50 GMT

Hi,

Yes I tried to run the script in the file my_app_poller_/bin : the json files went back to splunk with correct parsing.

I use this batch parameter because uselly I apply this input configuration with .sh script ans it works well.

The only difference I changed is to put a .py script instead of .sh, and it does not work anymore.

Last, I search in _internal and I find out this error :

ERROR ExecProcessor - Ignoring: "/opt/splunk/etc/my_app_poller/bin/my_script.py"

ERROR FrameworkUtils - Incorrect path to script: /opt/splunk/etc/my_app_poller/bin/my_script.py. Script must be located inside $SPLUNK_HOME/bin/scripts.

Can you tell me if I have to put my script in a path like : /opt/splunk/etc/my_app_poller/bin/scripts ?

Re: Why my indexation doesn't work ?

gcusello — Wed, 26 Aug 2020 09:18:02 GMT

Hi @mah,

scripts must be located in the main bin folder (as indicated in the error message) or in the bin folder of an app, in your path you forgot the apps folder:

/opt/splunk/etc/apps/my_app_poller/bin/my_script.py

Ciao.

Giuseppe

Re: Why my indexation doesn't work ?

mah — Wed, 26 Aug 2020 09:35:24 GMT

Hi,

sorry I did a bad copy paste, the script is already located in /opt/splunk/etc/apps/my_app_poller/bin/my_script.py

After a new test I get the error below :

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/my_app_poller/bin/my_script.py" IOError: [Errno 13] Permission denied:

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/my_app_poller/bin/my_script.py" Traceback (most recent call last😞

ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/my_app_poller/bin/my_script.py" File "/opt/splunk/etc/apps/my_app_poller/bin/my_script.py", line 13, in <module>

Re: Why my indexation doesn't work ?

gcusello — Wed, 26 Aug 2020 10:07:25 GMT

Hi @mah,

"Permission denied", it seems to be a grant problem, what are the grants of the python script?

check the grants with the user that you use to run Splunk.

Ciao.

Giuseppe

Re: Why my indexation doesn't work ?

mah — Wed, 26 Aug 2020 12:08:22 GMT

hi,

here are the rights applied to the script :

And the fact that it works when I run the script in the folder with command : python my_script.py

The json files are returned in splunk successfully parsed.

So I really don't understand.

Re: Why my indexation doesn't work ?

gcusello — Wed, 26 Aug 2020 13:53:24 GMT

Hi @mah,

and about the grants of the objects read by script?

Anyway, have you seen this documentation: https://docs.splunk.com/Documentation/Splunk/latest/AdvancedDev/ScriptedInputsIntro ?

Ciao.

Giuseppe