All Apps and Add-ons

Python script execution fails

arunpsz
New Member

Hello, I have an add-on which fetches emails from mail server. Due to O365 limitations, the add-on's python script required modification to support OAuth authentication.

 

Post modification, the script runs, however, it appears to be crashing once in a while randomly. After enabling the DEBUG for ExecProcessor, the following info is seen in the logs:

 

07-18-2023 15:29:34.771 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - PipelineSet 0: Got EOF from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py", uniqueId=20706
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - PipelineSet 0: Ran script: /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py, took 149.201812652 seconds to run, 0 bytes read 0 events read, status=done, exit=0, utime_sec=52.214078, stime_sec=7.235928, max_rss_kb=331052, vm_minor=606863, sched_vol=152097, sched_invol=3187
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - PipelineSet 0: Destroying ExecedCommandPipe for "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py" id=20706
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - ExecProcessorSharedState::addToRunQueue() path='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' restartTimerIfNeeded=1
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - adding "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py" to runqueue
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Added to run queue
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - Running: /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py on PipelineSet 0
07-18-2023 15:29:34.781 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - PipelineSet 0: Created new ExecedCommandPipe for "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py", uniqueId=20741
07-18-2023 15:29:58.340 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Not added to run queue
07-18-2023 15:30:58.337 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Not added to run queue
07-18-2023 15:31:58.339 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Not added to run queue
07-18-2023 15:32:58.339 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Not added to run queue
07-18-2023 15:33:58.340 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Not added to run queue
07-18-2023 15:34:58.342 +0530 DEBUG ExecProcessor [1181322 ExecProcessor] - cmd='/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py' Not added to run queue

 

I also noticed a few processes of script in interruptible sleep state since many days.

 

root@splunk:~# ps aux | grep splunk
root 303533 0.0 0.7 129636 121296 ? S Jul14 0:21 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
root 375980 0.0 0.4 98060 81632 ? S Jul14 0:18 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
....
root 1221826 0.0 0.0 3968 1336 ? S 18:28 0:00 /bin/sh -c /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
root 1221827 11.8 0.8 149052 140628 ? S 18:28 0:19 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
....
root 1536311 0.0 0.5 113036 96648 ? S Jul05 0:25 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
root 2278791 0.0 0.5 100092 91804 ? S Jul07 0:23 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
root 3247873 0.0 0.6 114140 106092 ? S Jul10 0:25 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py
root 3704217 0.0 0.8 148820 140472 ? S Jul12 0:19 /opt/splunk/bin/python3.7 /opt/splunk/etc/apps/TA-mailclient/bin/mail.py

 

arunpsz_0-1689675762863.png

 

Any help to debug further/fix the issue is highly appreciated. Thank you.

 

Regards,

Arun

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

WATCH NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If exploited, ...

Enter the Splunk Community Dashboard Challenge for Your Chance to Win!

The Splunk Community Dashboard Challenge is underway! This is your chance to showcase your skills in creating ...

.conf24 | Session Scheduler is Live!!

.conf24 is happening June 11 - 14 in Las Vegas, and we are thrilled to announce that the conference catalog ...