Splunk Dev

Why is there an error "ModuleNotFoundError: No module named 'syslog'" on running my python script?

sutom
Path Finder

Dear All,

I have created a TA to monitor a custom python script named log_parser_v1.py". Here is the configuration from /splunk/etc/apps/TA-logs/default/inputs.conf

[script://./bin/log_parser_v1.py]
python.version = python3.9
interval = 300
disabled = false

But while running TA got failed with the error "ModuleNotFoundError: No module named 'syslog'"

So I am trying to debug with splunk cmd python, and it's throwing "ModuleNotFoundError: No module named 'syslog'" error -

[ss@localhost bin]$ ./splunk cmd python log_parser_v1.py
Traceback (most recent call last):
  File "bin/log_parser_v1.py", line 7, in <module>
    import syslog
ModuleNotFoundError: No module named 'syslog'

But the same script runs fine with the command python3.9 bin/log_parser_v1.py

Here are the few lines from the script with the import statement of the module "syslog" in the line 7-

[ss@localhost bin]$ cat log_parser_v1.py
#!/usr/bin/env python
import os, sys
sys.path.append('/usr/bin/python3.9')
sys.path.append('/usr/lib/python3.9/site-packages')
sys.path.append('/usr/lib64/python3.9/site-packages')
sys.path.append(os.path.dirname(os.path.abspath(__file__)))
import json, logging, syslog, datetime, argparse, shutil, zipfile, tarfile, bz2, socket, sys, errno, time, gzip, hashlib
from logging.handlers import SysLogHandler, SYSLOG_TCP_PORT
from syslog import LOG_USER

To use python3.9. I append the python3.9 package path in script but it still is not picking the syslog module. here is the python3.9 path -

[ss@localhost bin]$ whereis python
python: /usr/bin/python2.7 /usr/bin/python3.6 /usr/bin/python3.6m /usr/bin/python3.9 /usr/lib/python2.7 /usr/lib/python3.6 /usr/lib/python3.9 /usr/lib64/python2.7 /usr/lib64/python3.6 /usr/lib64/python3.9 /usr/include/python3.9 /usr/include/python2.7 /usr/include/python3.6m /usr/share/man/man1/python.1.gz

I also tried to import syslog package with ./splunk cmd python, but it got failed

[ss@localhost bin]$ ./splunk cmd python
Python 3.7.11 (default, May 25 2022, 12:23:55)
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import syslog
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'syslog'
>>> exit()

And here is imported successfully with python3.9

[ss@localhost bin]$ python3.9
Python 3.9.7 (default, Sep 13 2021, 08:18:39)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import syslog
>>> exit()

Guys, I am looking for your help to understand like what is missing. please help here.

Labels (1)
Tags (3)
0 Karma
1 Solution

sutom
Path Finder

Got a recommendation to use below app- 

splunkbase.splunk.com/app/4322

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Splunk runs Python scripts using its own Python interpreter.  If you develop code using a different interpreter, be sure to test with the Splunk interpreter to find out about errors like ths.

The fix is to add the missing module to the bin/lib or lib directory of your TA.

---
If this reply helps you, Karma would be appreciated.
0 Karma

sutom
Path Finder

@richgalloway  Thanks for the response.

But in the past, I had the same issue for another script with BeautifulSoup, and that got resolved by appending the python3.9 path

 

sys.path.append('/usr/local/bin/python3.9')
sys.path.append('/usr/local/lib/python3.9/site-packages')
sys.path.append('/usr/lib64/python3.9/site-packages')
from bs4 import BeautifulSoup

 

Here Also, I thought it was a similar one. but not working...

Another analysis is that if I am running ./splunk cmd python3.9 then syslog gets imported..

 

[ss@localhost bin]$./splunk cmd python3.9
Python 3.9.7 (default, Sep 13 2021, 08:18:39)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-3)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import syslog
>>> exit()

 

also if I run script manually with the command - ./splunk cmd python3.9 log_parser_v1.py, it is running fine.

but It failing under the TA schedule.

0 Karma

terminaloutcome
Path Finder

At that point you're either importing modules across python versions or running a non-standard interpreter, both of which are unsupported and asking for trouble. I'd definitely recommend @richgalloway 's suggestion of having the package in the app's path.


0 Karma

richgalloway
SplunkTrust
SplunkTrust

I don't have a "python3.9" in my Splunk installation, just "python3".

According to the Admin manual, "python3.9" is not a valid value for python.version.

If what you've done before isn't working this time then consider filing a support request.  While you wait for that, try my suggestion.  🙂

---
If this reply helps you, Karma would be appreciated.

sutom
Path Finder

Got a recommendation to use below app- 

splunkbase.splunk.com/app/4322

Get Updates on the Splunk Community!

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...

Edge Processor Scaling, Energy & Manufacturing Use Cases, and More New Articles on ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Get More Out of Your Security Practice With a SIEM

Get More Out of Your Security Practice With a SIEMWednesday, July 31, 2024  |  11AM PT / 2PM ETREGISTER ...