Splunk Phantom

Why is compile_app.py failing to install my phantom app?

Communicator

I'm working my way through the phantom appdev tutorial and can't get past an annoying issue when attempting to compile and install the test app.

I've used pip to install a bunch of missing dependencies including "django<2", psycopg2, python-magic, simple-json, kombu, passwords... and validators (0.12.2) - referenced in the ImportError final line below.

I've walked through the code in pudb and can see the issue occuring as django.setup() walks through modules it needs to import - and fails when modelsmodulename is set to phantom_ui.ui.models.

I have no idea about jenkins. I also have no idea why "wishes to do different things depending on the Python version." is showing up in the output, given that's just one line in one of the user.py files.

cd'ing into ./
Compiling: ./__init__.py
Compiling: ./samplewhois_consts.py
Compiling: ./samplewhois_connector.py
Validating: ./samplewhois.json
Installing app...
Creating tarball...
../samplewhois.tgz
Traceback (most recent call last):
  File "../compile_app.py", line 200, in <module>
    ret_val, err_string = _install_app("{0}.tgz".format(app_dir))
  File "../compile_app.py", line 94, in _install_app
    import import_app as importer
  File "/opt/phantom/www/phantom_ui/import_app.py", line 40, in <module>
    django.setup()
  File "/usr/lib64/python2.7/site-packages/django/__init__.py", line 27, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "/usr/lib64/python2.7/site-packages/django/apps/registry.py", line 108, in populate
    app_config.import_models()
  File "/usr/lib64/python2.7/site-packages/django/apps/config.py", line 202, in import_models
    self.models_module = import_module(models_module_name)
  File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/var/lib/jenkins/workspace/3.8+_create_rpm_packages/www/phantom_ui/ui/models/__init__.py", line 1, in <module>
  File "/var/lib/jenkins/workspace/3.8+_create_rpm_packages/www/phantom_ui/ui/models/core.py", line 27, in <module>
  File "/var/lib/jenkins/workspace/3.8+_create_rpm_packages/www/phantom_ui/ui/models/action_run.py", line 16, in <module>
  File "/var/lib/jenkins/workspace/3.8+_create_rpm_packages/www/phantom_ui/ui/models/system.py", line 51, in <module>
  File "/var/lib/jenkins/workspace/3.8+_create_rpm_packages/www/phantom_ui/ui/models/user.py", line 20, in <module>
    wishes to do different things depending on the Python version.
ImportError: No module named validators
Labels (2)
Tags (1)
0 Karma
1 Solution

Splunk Employee
Splunk Employee

In Phantom, it is important to be mindful of the different versions, and the methods that change between them.

In newer versions of Phantom you need to run commands like the compile_app.pyc, under phenv.

The full command would be: phenv python2.7 /opt/phantom/bin/compile_app.pyc -i

,

View solution in original post

Communicator

Chris from Phantom support kindly helped out with this:

phenv python2.7 /opt/phantom/bin/compile_app.pyc -i

It looks like I was following an old tutorial (https://my.phantom.us/1.1/docs/appdev/tutorial) rather than https://my.phantom.us/4.1/docs/appdev/tutorial.

That phenv-based command suddenly made the samplewhois app from the 1.1 tutorial compile fine.

0 Karma

Splunk Employee
Splunk Employee

In Phantom, it is important to be mindful of the different versions, and the methods that change between them.

In newer versions of Phantom you need to run commands like the compile_app.pyc, under phenv.

The full command would be: phenv python2.7 /opt/phantom/bin/compile_app.pyc -i

,

View solution in original post

Splunk Employee
Splunk Employee

I completely agree with my understanding of the issue presented doesn't align with the problem presented. There should be a little further dialogue to explain why the output determined the incorrect usage of the compileapp.pyc command. Thanks for the help chrissplunk!

0 Karma

Communicator

Awesome - thanks.

Can you please get https://my.phantom.us/1.1/docs/appdev/tutorial removed - or marked as superceded in some way?

0 Karma

Splunk Employee
Splunk Employee

This shouldn't be removed, but your true source should be the platform you are working from as it's the one you are building to. https://my.phantom.us is for all the version and you need to be cognitive of the versions. As this moves to splunk.com, I am pretty sure the change will make versioning easier as they have in the Splunk versions.

0 Karma

Splunk Employee
Splunk Employee

I have submitted an internal ticket to handle wording for older doc sets.

Splunk Employee
Splunk Employee

I found an answer to your problem on the modules... https://my.phantom.us/3.5/docs/appdev/faq

Please review the install process for module and how to update your code for additional dependancies.

How do I handle python module dependencies for my app?

There is more than one way to manage dependencies on python modules. Do note that when an action is executed, the platform will add the following directories to the PYTHONPATH environment:

/opt/phantom/lib
/opt/phantom/www
/opt/phantom/apps/[app_install_directory]
/opt/phantom/apps/[app_install_directory]/dependencies

Use one of the following ways to manage dependencies:

  • Package with the App
  • Place all required modules as part of the app tarball. There are two ways to do this:
  • Install the modules into a sub directory of the app.
  • pip2.7 supports the --target command line switch that allows the modules to be installed at a specific location. Use this switch to install the modules into the app's subdirectory called dependencies. When the app tarball is installed on the platform the modules will be part of the app code.
  • Distribute the complete module, including the source, license file etc. in a sub directory of the app. In this case the app will need to append any folders in the PYTHONPATH it self.
  • The platform will install all the files present in the app tarball in the app install directory. Care should be taken while packaging modules this way. Many python modules are released under various licenses, so please make sure the license allows for such re-distribution. Packaging modules this way, makes the app self-sufficient.
  • Add PIP dependencies in the app json
  • One can specify app dependencies in the app JSON, that the platform will try to install with the app. Please see the section titled Specifying pip dependencies for more information.