topic Re: "ModuleNotFoundError: No module named 'syslog'" error on running python script in Splunk Dev

Why is there an error "ModuleNotFoundError: No module named 'syslog'" on running my python script?

sutom — Thu, 01 Dec 2022 23:22:59 GMT

Dear All,

I have created a TA to monitor a custom python script named log_parser_v1.py". Here is the configuration from /splunk/etc/apps/TA-logs/default/inputs.conf

[script://./bin/log_parser_v1.py] python.version = python3.9 interval = 300 disabled = false

But while running TA got failed with the error "ModuleNotFoundError: No module named 'syslog'"

So I am trying to debug with splunk cmd python, and it's throwing "ModuleNotFoundError: No module named 'syslog'" error -

[ss@localhost bin]$ ./splunk cmd python log_parser_v1.py Traceback (most recent call last): File "bin/log_parser_v1.py", line 7, in <module> import syslog ModuleNotFoundError: No module named 'syslog'

But the same script runs fine with the command python3.9 bin/log_parser_v1.py

Here are the few lines from the script with the import statement of the module "syslog" in the line 7-

[ss@localhost bin]$ cat log_parser_v1.py #!/usr/bin/env python import os, sys sys.path.append('/usr/bin/python3.9') sys.path.append('/usr/lib/python3.9/site-packages') sys.path.append('/usr/lib64/python3.9/site-packages') sys.path.append(os.path.dirname(os.path.abspath(__file__))) import json, logging, syslog, datetime, argparse, shutil, zipfile, tarfile, bz2, socket, sys, errno, time, gzip, hashlib from logging.handlers import SysLogHandler, SYSLOG_TCP_PORT from syslog import LOG_USER

To use python3.9. I append the python3.9 package path in script but it still is not picking the syslog module. here is the python3.9 path -

[ss@localhost bin]$ whereis python python: /usr/bin/python2.7 /usr/bin/python3.6 /usr/bin/python3.6m /usr/bin/python3.9 /usr/lib/python2.7 /usr/lib/python3.6 /usr/lib/python3.9 /usr/lib64/python2.7 /usr/lib64/python3.6 /usr/lib64/python3.9 /usr/include/python3.9 /usr/include/python2.7 /usr/include/python3.6m /usr/share/man/man1/python.1.gz

I also tried to import syslog package with ./splunk cmd python, but it got failed

[ss@localhost bin]$ ./splunk cmd python Python 3.7.11 (default, May 25 2022, 12:23:55) [GCC 9.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> import syslog Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'syslog' >>> exit()

And here is imported successfully with python3.9

[ss@localhost bin]$ python3.9 Python 3.9.7 (default, Sep 13 2021, 08:18:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import syslog >>> exit()

Guys, I am looking for your help to understand like what is missing. please help here.

Re: "ModuleNotFoundError: No module named 'syslog'" error on running python script

richgalloway — Fri, 11 Nov 2022 17:12:08 GMT

Splunk runs Python scripts using its own Python interpreter. If you develop code using a different interpreter, be sure to test with the Splunk interpreter to find out about errors like ths.

The fix is to add the missing module to the bin/lib or lib directory of your TA.

Re: "ModuleNotFoundError: No module named 'syslog'" error on running python script

sutom — Fri, 11 Nov 2022 18:14:35 GMT

@richgalloway Thanks for the response.

But in the past, I had the same issue for another script with BeautifulSoup, and that got resolved by appending the python3.9 path

sys.path.append('/usr/local/bin/python3.9') sys.path.append('/usr/local/lib/python3.9/site-packages') sys.path.append('/usr/lib64/python3.9/site-packages') from bs4 import BeautifulSoup

Here Also, I thought it was a similar one. but not working...

Another analysis is that if I am running ./splunk cmd python3.9 then syslog gets imported..

[ss@localhost bin]$./splunk cmd python3.9 Python 3.9.7 (default, Sep 13 2021, 08:18:39) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import syslog >>> exit()

also if I run script manually with the command - ./splunk cmd python3.9 log_parser_v1.py, it is running fine.

but It failing under the TA schedule.

Re: "ModuleNotFoundError: No module named 'syslog'" error on running python script

richgalloway — Fri, 11 Nov 2022 19:37:17 GMT

I don't have a "python3.9" in my Splunk installation, just "python3".

According to the Admin manual, "python3.9" is not a valid value for python.version.

If what you've done before isn't working this time then consider filing a support request. While you wait for that, try my suggestion. 🙂

Re: "ModuleNotFoundError: No module named 'syslog'" error on running python script

terminaloutcome — Thu, 01 Dec 2022 21:15:29 GMT

At that point you're either importing modules across python versions or running a non-standard interpreter, both of which are unsupported and asking for trouble. I'd definitely recommend @richgalloway 's suggestion of having the package in the app's path.

Re: "ModuleNotFoundError: No module named 'syslog'" error on running python script

sutom — Thu, 15 Dec 2022 13:41:29 GMT

Got a recommendation to use below app-

splunkbase.splunk.com/app/4322