topic Re: Where to place common python libraries in Splunk Search

Where to place common python libraries

eddieparra — Sun, 11 Mar 2018 09:06:57 GMT

I have multiple alert actions in Python. I am trying to have the modalert helper for each action to load a common library, which I would like to place in this package. I do not know where is the appropriate location within the package directory structure to place such a library.

Re: Where to place common python libraries

tiagofbmm — Tue, 29 Sep 2020 18:22:02 GMT

Any python library you want to add should be in:

$SPLUNK_HOME/etc/apps/your_app_name/bin/lib

Remember that in the lib folder, the sub-folders containing the py library must contain file init.py

Let me know if solved your question and if so, please accept the answer for future queries

Re: Where to place common python libraries

eddieparra — Sun, 11 Mar 2018 17:58:25 GMT

How can I make it part of the AR app itself? This would be more of a helper function vs. a proper library. When using the app builder, the bin/ directory has many mainstream libs. I would like to embed it in a similar fashion, for all my actions. I just don't know what is the proper location to pass validation. Thanks.

Re: Where to place common python libraries

tiagofbmm — Sun, 11 Mar 2018 18:11:25 GMT

Well if you put it in the app folder, under /bin/lib/ (not under /bin) then it is part of the app.
That's it!

If you are planning to create an custom command in Splunk with the python library, you also need to configure commands.conf in that same app you put the python library. The python script that the command would call should be in the app folder, under /bin

Re: Where to place common python libraries

eddieparra — Sun, 11 Mar 2018 18:38:46 GMT

Have you seen this documented anywhere? As I look at other AR apps, I never see this. I trust you are correct, so let me give this a try.

Re: Where to place common python libraries

tiagofbmm — Sun, 11 Mar 2018 18:56:08 GMT

What do you mean AR app?

Re: Where to place common python libraries

eddieparra — Sun, 11 Mar 2018 18:57:13 GMT

Adaptive Response

Re: Where to place common python libraries

tiagofbmm — Sun, 11 Mar 2018 19:27:54 GMT

Well I believe they work the same way though.

Let me know if they don't 🙂

Re: Where to place common python libraries

eddieparra — Tue, 29 Sep 2020 18:24:29 GMT

I created the directory you stated:

$SPLUNK_HOME/etc/apps/your_app_name/bin/lib

I then created:

$SPLUNK_HOME/etc/apps/your_app_name/bin/commands.conf.

Inside I added:

# type of script: 'python', 'perl'
 TYPE = python

# is command streamable?
streaming = false

# maximum data that can be passed to command (0 = no limit)
maxinputs = 50000

[test_sdk]
filename = test_sdk.py

In bin/lib I have an init.py file and test_sdk.py

[root@splunk lib]# ls -l
total 0
-rw-r--r--. 1 root root 0 Mar 11 12:01 __init__.py
-rw-r--r--. 1 root root 0 Mar 11 12:00 test_sdk.py

In $SPLUNK_HOME/etc/apps/your_app_name/bin/ta__adaptive_response/, I have "modalert_blacklist_block_helper.py". This file has "import test_sdk".

When I run a test, it still can't import test_sdk. Any ideas?

Re: Where to place common python libraries

eddieparra — Sun, 11 Mar 2018 19:29:30 GMT

I just posted a more details msg here. It is pending moderator approval.

Re: Where to place common python libraries

tiagofbmm — Tue, 29 Sep 2020 18:22:04 GMT

Two things I can see right away:

1 - The commands.conf is a conf file and must be placed under $SPLUNK_HOME/etc/apps/your_app_name/local or $SPLUNK_HOME/etc/apps/your_app_name/default, so move it to either of the directories.

2 - The test you are performing, "import test_sdk", I suggest you first try it by using this:

Go to the $SPLUNK_HOME/etc/apps/your_app_name/bin/ (where test_sdk.py must be)

Then try executing this for test:
$SPLUNK_HOME/bin/splunk cmd python test_sdk.py

If it still doesn't work, you need to append new directories for which python will search for modules to import, adding this to your modalert_blacklist_block_helper.py (before the import test_sdk)

import os, stat
import sys

dir = os.path.join(os.path.join(os.environ.get('SPLUNK_HOME')), 'etc', 'apps', 'your_app_name', 'bin','lib')
if not dir in sys.path:
    sys.path.append(dir)

And finally, I am not aware of the structure of having a TA inside an a Splunk App, I've never seen that before anywhere honestly, I can't judge that

Re: Where to place common python libraries

tiagofbmm — Wed, 21 Mar 2018 16:55:02 GMT

Please let me know if the answer was useful for you. If it was, accept it and upvote. If not, give us more input so we can help you with that