Getting Data In

After setting non-system default timezone external commands fail with error code 1 or "ImportError: No module named site"

IgorB
Path Finder

In new 4.3 instance running on Win2008 R2, external commands (e.g. sendemail) have started failing with errors like the one below.

03-01-2012 09:20:26.862 +0000 ERROR script - External search command 'sendemail' returned error code 1.
03-01-2012 09:20:26.831 +0000 ERROR ScriptRunner - stderr from 'C:\Program Files\Splunk\etc\apps\search\bin\sendemail.py': ImportError: No module named site

I can't pinpoint the problem to a particular application installation or configuration change, all I can say is that everything has worked fine when the instance was first installed, but at some point we've noticed that the mails have stopped coming and then we've found the errors like the one in the example.

Tags (2)
1 Solution

IgorB
Path Finder

We found what was triggering the problem. Comes out it's the per-user timezone setting (new in 4.3). So once you set the user's timezone to anything other than the system's default timezone, you'll encounter the following problems:

  1. Searches utilizing external search commands, e.g. xmlkv, will start failing showing red bar with messages like "External search command 'xmlkv' returned error code 1.". There will be no traces of an error in splunkd.log.
  2. 'Send Email' alert action and other scripts, e.g. "backfill data" in Deployment Monitor app will also fail, but at least will leave trace in splunkd.log:

03-01-2012 09:20:26.862 +0000 ERROR script - External search command 'sendemail' returned error code 1.
03-01-2012 09:20:26.831 +0000 ERROR ScriptRunner - stderr from 'C:\Program Files\Splunk\etc\apps\search\bin\sendemail.py': ImportError: No module named site

View solution in original post

smeier
Path Finder

I wanted to upvote IgorB's answer, but I don't have enough points 😛

Anyway- you saved us too, thank you!

0 Karma

patrikan
Engager

Thank you, i can confirm that setting the "System Default timezone" made my email alerts work again.
I dont know how you found it, but a big thank you for posting it, saved my day 🙂

IgorB
Path Finder

The credit should go to my customer - Eyal P.

0 Karma

IgorB
Path Finder

We found what was triggering the problem. Comes out it's the per-user timezone setting (new in 4.3). So once you set the user's timezone to anything other than the system's default timezone, you'll encounter the following problems:

  1. Searches utilizing external search commands, e.g. xmlkv, will start failing showing red bar with messages like "External search command 'xmlkv' returned error code 1.". There will be no traces of an error in splunkd.log.
  2. 'Send Email' alert action and other scripts, e.g. "backfill data" in Deployment Monitor app will also fail, but at least will leave trace in splunkd.log:

03-01-2012 09:20:26.862 +0000 ERROR script - External search command 'sendemail' returned error code 1.
03-01-2012 09:20:26.831 +0000 ERROR ScriptRunner - stderr from 'C:\Program Files\Splunk\etc\apps\search\bin\sendemail.py': ImportError: No module named site

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...