topic Re: How to import Python module from subdirectory in appserver/controllers? in Splunk Dev

How to import Python module from subdirectory in appserver/controllers?

twesthead — Tue, 06 Dec 2016 12:01:09 GMT

Dear Splunk,

I have quite a few custom endpoints in my web.conf file.
This leads to many Python scripts in my appserver/controllers directory and many helper scripts for functions used by several endpoints.
I would like to tidy up this directory by putting some of the scripts in local packages or subdirectories.
As an example, here's my file structure:

appserver
--->controllers
------->my_script.py
------->mylib
---------->mymod.py

My code:

#mymod.py
def scream():
    return 'YEAH!'

#my_script.py
import splunk.appserver.mrsparkle.controllers as controllers
from splunk.appserver.mrsparkle.lib.decorators import expose_page

import mylib.mymod as mm

class Controller(controllers.BaseController):
    @expose_page(must_login=False, methods=['GET']) 
    def scream(self, **kwargs) :
        return mm.scream()

Now, if I add a empty __init__.py file to /mylib, mylib behaves like a package, and if I test the script locally with splunk's python interpreter in CLI: splunk cmd python my_script.py , I am able to import to import my module and use the scream() function from my_script.py.

However, the import doesn't work one I try to run the script through the web service: http://localhost:8000/en-US/custom/my_app/my_script/scream throws a 404 Error.

What's the Splunk-approved way of organizing code in my controllers directory ?

Re: How to import Python module from subdirectory in appserver/controllers?

alacercogitatus — Tue, 06 Dec 2016 15:03:02 GMT

I have a controller in one of my apps. The only real difference I can see is that you did not call the class the same as the filename. Change it, and let me know.

 class my_script(controllers.BaseController):

This should present it self and work. Additionally, you may want to try putting this in web.conf

 [endpoint:my_script]

Re: How to import Python module from subdirectory in appserver/controllers?

twesthead — Tue, 06 Dec 2016 15:31:41 GMT

Hi, thank you for your answer.
My controllers already work fine with :
- [endpoint:my_script] in web.conf
- my_script.py as a file name in controllers
- class random_name(controllers.BaseController): in my_script.py

In this Splunk Wiki page, someone shows an example where the class name doesn't match the file name. https://wiki.splunk.com/Community:40GUIDevelopment

What I am trying to achieve is moving some code from controllers to subdirectories.

Re: How to import Python module from subdirectory in appserver/controllers?

dominiquevocat — Sun, 16 Apr 2017 15:44:07 GMT

did you ever manager to get it to run? I need a ready made class and i can not load it inside my controller 😕

Re: How to import Python module from subdirectory in appserver/controllers?

twesthead — Tue, 18 Apr 2017 07:46:42 GMT

I still haven't found a solution to my problem which was: keeping the code inside the app structure.
If you don't have this requirement, have you tried putting your ready made class inside a module and inside the following folder?

$SPLUNK_HOME/lib/python2.7/site-packages

Re: How to import Python module from subdirectory in appserver/controllers?

dominiquevocat — Tue, 18 Apr 2017 08:34:12 GMT

actually i managed... will post it as a answer but it is hackish, you can exted the python path... 😕

Re: How to import Python module from subdirectory in appserver/controllers?

dominiquevocat — Tue, 18 Apr 2017 08:35:02 GMT

You need to extend the python path 😞

        import sys
        sys.path.append(os.path.join(os.environ['SPLUNK_HOME'],'etc','apps','YourAppNameHere','appserver','controllers')) #build local path and add it to the python path so we can load modules, hack!

Re: How to import Python module from subdirectory in appserver/controllers?

twesthead — Tue, 18 Apr 2017 09:02:57 GMT

This works, indeed. Thank you for your answer.

Re: How to import Python module from subdirectory in appserver/controllers?

dominiquevocat — Tue, 02 May 2017 15:11:27 GMT

Glad it works for you. When its checked by Splunk you can look at what i used it for in https://splunkbase.splunk.com/app/3573/ 🙂