Sharing my experiences with the getting IMAP App (Linux/Splunk) to check an IMAP mailbox hosted on Microsoft Exchange 2007. Hope it helps.
cat /opt/splunk/etc/apps/imap/local/imap.conf
[IMAP Configuration]
debug = True
deleteWhenDone = False
disabled = 0
fullHeaders = False
includeBody = True
mimeTypes = text/plain
folders = Inbox, Inbox\Subfolder
noCache = False
server = 11.22.33.44
useSSL = False
user = erroremails@company.com
xpassword = U2FsdGVkX118jLJ2rc1fCJT4VY1KzOcWuTHxQ18kIOdDrcQ=
splunkuser = admin
splunkxpassword = U2FsdGV1kX119umt1wLRrUP1tRk4es3jiuiF587D16+jCTQ=
splunkHostPath = https://localhost:8089
Environment;
References;
bash /opt/splunk/etc/apps/imap/bin/genpass.sh
Enter the password you want to encrypt:
copy/paste the output to the appropriate variable in imap.conf
cd /opt/splunk/etc/apps/imap
python bin/getimap.py --debug
Traceback (most recent call last):
File "bin/getimap.py", line 39, in <module>
raise ConfigError('Environment variable SPLUNK_HOME must be set')
__main__.ConfigError: Environment variable SPLUNK_HOME must be set
Solution credits to gkanapathy
source /opt/splunk/bin/setSplunkEnv
echo $SPLUNK_HOME
Note: These environment variables help get the getimap.py to run from the CLI. They are not required for Splunkd or SplunkWeb.
Debug again. Pay close attention to all of the True/False values. Ensure they are obeying your local/imap.conf settings. Try using "True" or "False" instead of 0 or 1.
cd /opt/splunk/etc/apps/imap
python bin/getimap.py --debug
DEBUG:root:END_IMAP_BREAKER=EndIMAPMessage
DEBUG:root:bodySourceType=imapbody
DEBUG:root:body_separator=____________________ Message Body ____________________
DEBUG:root:debug=True
DEBUG:root:deleteWhenDone=False
DEBUG:root:folders=all
DEBUG:root:fullHeaders=False
DEBUG:root:headerSourceType=imap
DEBUG:root:imapSearch=(UNDELETED SMALLER 204800)
DEBUG:root:includeBody=True <-- make sure ALL true/false values match your imap.conf
DEBUG:root:mimeTypes=text/plain
DEBUG:root:noCache=False
DEBUG:root:password=
DEBUG:root:port=143
DEBUG:root:server=11.22.33.44
DEBUG:root:splunkHostPath=https://localhost:8089
DEBUG:root:splunkpassword=changeme
DEBUG:root:splunkuser=admin
DEBUG:root:splunkxpassword=U2FsdGVkX19u1mtwLRrUPtRk4e1s3jiuiF587D16+jCTQ=
DEBUG:root:timeout=10
DEBUG:root:useBodySourceType=False
DEBUG:root:useSSL=False
DEBUG:root:user=erroremails@company.com
DEBUG:root:version=1.2
DEBUG:root:xpassword=U2FsdGV1kX18jLJ12rcfCJT4VYK1zOcWuTHxQ8kIOdDrcQ=
DEBUG:root:VERSION = 1.2
DEBUG:root:about to dump mailbox Calendar
DEBUG:root:No module named splunk.auth
Traceback (most recent call last):
File "bin/getimap.py", line 405, in getMailbox
latestTime = self.getCacheIDForMailbox(box)
File "bin/getimap.py", line 234, in getCacheIDForMailbox
import splunk.auth as au
ImportError: No module named splunk.auth
Reason: Running "python" from the CLI is using your system's native python install, which doesn't include Splunk specific modules.
which python
/usr/bin/python <-- that's not Splunk's custom python...
Solution 1.
/opt/splunk/bin/python bin/getimap.py --debug
Solution 2. credits to gkanapathy
A quick way to set the right environments, paths, etc. to work as a script running inside Splunk is to use "/opt/splunk/bin/splunk cmd "
/opt/splunk/bin/splunk cmd python bin/imap.py
DEBUG:root:libxslt.so.1: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "bin/getimap.py", line 405, in getMailbox
latestTime = self.getCacheIDForMailbox(box)
File "bin/getimap.py", line 234, in getCacheIDForMailbox
import splunk.auth as au
File "/opt/splunk/lib/python2.6/site-packages/splunk/auth.py", line 1, in <module>
import splunk, rest, util
File "/opt/splunk/lib/python2.6/site-packages/splunk/rest/__init__.py", line 7, in <module>
import lxml.etree as et
ImportError: libxslt.so.1: cannot open shared object file: No such file or directory
Solution for Ubuntu 10.04 to install the missing library;
apt-get install python-lxml
The following NEW packages will be installed:
libxslt1.1 python-lxml
DEBUG:root:'SPLUNK_DB'
Traceback (most recent call last):
File "bin/getimap.py", line 405, in getMailbox
latestTime = self.getCacheIDForMailbox(box)
File "bin/getimap.py", line 234, in getCacheIDForMailbox
import splunk.auth as au
File "/opt/splunk/lib/python2.6/site-packages/splunk/auth.py", line 4, in <module>
import splunk.clilib.cli_common as comm
File "/opt/splunk/lib/python2.6/site-packages/splunk/clilib/cli_common.py", line 39, in <module>
splunk_db = os.path.normpath(os.environ["SPLUNK_DB"])
File "/opt/splunk/lib/python2.6/UserDict.py", line 22, in __getitem__
raise KeyError(key)
KeyError: 'SPLUNK_DB'
Solution is tied to the environment variables being set properly. Use setSplunkEnv;
source /opt/splunk/bin/setSplunkEnv
echo $SPLUNK_DB
Everything below needs to be run from the Exchange Management SHELL. Make sure to right/click, run-as administrator...
[PS] C:\>get-service msexchangeimap4
Status Name DisplayName
------ ---- -----------
Stopped msexchangeimap4 Microsoft Exchange IMAP4
[PS] C:\>start-service -name msexchangeimap4
[PS] C:\>set-service msexchangeimap4 -startuptype automatic
WARNING: Waiting for service 'Microsoft Exchange IMAP4 (msexchangeimap4)' to
finish starting...
I have only gotten PlainText working with the IMAP App so far. TLS/SSL coming...
TechNet Article - How to Configure Authentication for IMAP4
[PS] C:\>Set-IMAPSettings -LoginType PlainTextLogin
-LoginType <PlainTextLogin | PlainTextAuthentication | SecureLogin>
This parameter specifies the authentication setting that is used for the
Client Access server that is running the IMAP4 service. The possible
values are as follows:
* 1:PlainTextLogin
* 2:PlainTextAuthentication
* 3:SecureLogin
[PS] C:\>restart-service -service msExchangeIMAP4
WARNING: Waiting for service 'Microsoft Exchange IMAP4 (msExchangeIMAP4)' to
finish starting...
[PS] C:\>get-imapsettings | fl
ProtocolName : IMAP4
Name : 1
MaxCommandSize : 10240
ShowHiddenFoldersEnabled : False
UnencryptedOrTLSBindings : {:::143, 0.0.0.0:143}
SSLBindings : {:::993, 0.0.0.0:993}
X509CertificateName : SERVERNAME
Banner : The Microsoft Exchange IMAP4 service is rea
dy.
LoginType : PlainTextLogin
AuthenticatedConnectionTimeout : 00:30:00
PreAuthenticatedConnectionTimeout : 00:01:00
MaxConnections : 2000
MaxConnectionFromSingleIP : 2000
MaxConnectionsPerUser : 16
MessageRetrievalMimeFormat : BestBodyFormat
ProxyTargetPort : 143
CalendarItemRetrievalOption : iCalendar
OwaServerUrl :
EnableExactRFC822Size : False
ExtendedProtectionPolicy : None
EnableGSSAPIAndNTLMAuth : True
AdminDisplayName :
ExchangeVersion : 0.1 (8.0.535.0)
DistinguishedName : CN=1,CN=IMAP4,CN=Protocols,CN=SERVERNAME
,CN=Servers,CN=Exchange Administrative Grou
p (FYD123OHF23SPDLT),CN=Administrative Group
s,CN=DOMAIN,CN=Microsoft Exchange,CN=Servic
es,CN=Configuration,DC=company,DC=com
Identity : SERVERNAME\1
Guid : 06a16b81-04f4-4ed8-b8dd-c212328
ObjectCategory : onpeak.us/Configuration/Schema/ms-Exch-Prot
ocol-Cfg-IMAP-Server
ObjectClass : {top, protocolCfg, protocolCfgIMAP, protoco
lCfgIMAPServer}
WhenChanged : 11/19/2010 1:46:00 PM
WhenCreated : 11/24/2009 10:26:06 PM
OriginatingServer : SERVERNAME.company.com
IsValid : True
bash /opt/splunk/etc/apps/imap/bin/genpass.sh
Enter the password you want to encrypt:
copy/paste the output to the appropriate variable in imap.conf
cd /opt/splunk/etc/apps/imap
python bin/getimap.py --debug
Traceback (most recent call last):
File "bin/getimap.py", line 39, in <module>
raise ConfigError('Environment variable SPLUNK_HOME must be set')
__main__.ConfigError: Environment variable SPLUNK_HOME must be set
Solution credits to gkanapathy
source /opt/splunk/bin/setSplunkEnv
echo $SPLUNK_HOME
Note: These environment variables help get the getimap.py to run from the CLI. They are not required for Splunkd or SplunkWeb.
Debug again. Pay close attention to all of the True/False values. Ensure they are obeying your local/imap.conf settings. Try using "True" or "False" instead of 0 or 1.
cd /opt/splunk/etc/apps/imap
python bin/getimap.py --debug
DEBUG:root:END_IMAP_BREAKER=EndIMAPMessage
DEBUG:root:bodySourceType=imapbody
DEBUG:root:body_separator=____________________ Message Body ____________________
DEBUG:root:debug=True
DEBUG:root:deleteWhenDone=False
DEBUG:root:folders=all
DEBUG:root:fullHeaders=False
DEBUG:root:headerSourceType=imap
DEBUG:root:imapSearch=(UNDELETED SMALLER 204800)
DEBUG:root:includeBody=True <-- make sure ALL true/false values match your imap.conf
DEBUG:root:mimeTypes=text/plain
DEBUG:root:noCache=False
DEBUG:root:password=
DEBUG:root:port=143
DEBUG:root:server=11.22.33.44
DEBUG:root:splunkHostPath=https://localhost:8089
DEBUG:root:splunkpassword=changeme
DEBUG:root:splunkuser=admin
DEBUG:root:splunkxpassword=U2FsdGVkX19u1mtwLRrUPtRk4e1s3jiuiF587D16+jCTQ=
DEBUG:root:timeout=10
DEBUG:root:useBodySourceType=False
DEBUG:root:useSSL=False
DEBUG:root:user=erroremails@company.com
DEBUG:root:version=1.2
DEBUG:root:xpassword=U2FsdGV1kX18jLJ12rcfCJT4VYK1zOcWuTHxQ8kIOdDrcQ=
DEBUG:root:VERSION = 1.2
DEBUG:root:about to dump mailbox Calendar
DEBUG:root:No module named splunk.auth
Traceback (most recent call last):
File "bin/getimap.py", line 405, in getMailbox
latestTime = self.getCacheIDForMailbox(box)
File "bin/getimap.py", line 234, in getCacheIDForMailbox
import splunk.auth as au
ImportError: No module named splunk.auth
Reason: Running "python" from the CLI is using your system's native python install, which doesn't include Splunk specific modules.
which python
/usr/bin/python <-- that's not Splunk's custom python...
Solution 1.
/opt/splunk/bin/python bin/getimap.py --debug
Solution 2. credits to gkanapathy
A quick way to set the right environments, paths, etc. to work as a script running inside Splunk is to use "/opt/splunk/bin/splunk cmd "
/opt/splunk/bin/splunk cmd python bin/imap.py
DEBUG:root:libxslt.so.1: cannot open shared object file: No such file or directory
Traceback (most recent call last):
File "bin/getimap.py", line 405, in getMailbox
latestTime = self.getCacheIDForMailbox(box)
File "bin/getimap.py", line 234, in getCacheIDForMailbox
import splunk.auth as au
File "/opt/splunk/lib/python2.6/site-packages/splunk/auth.py", line 1, in <module>
import splunk, rest, util
File "/opt/splunk/lib/python2.6/site-packages/splunk/rest/__init__.py", line 7, in <module>
import lxml.etree as et
ImportError: libxslt.so.1: cannot open shared object file: No such file or directory
Solution for Ubuntu 10.04 to install the missing library;
apt-get install python-lxml
The following NEW packages will be installed:
libxslt1.1 python-lxml
DEBUG:root:'SPLUNK_DB'
Traceback (most recent call last):
File "bin/getimap.py", line 405, in getMailbox
latestTime = self.getCacheIDForMailbox(box)
File "bin/getimap.py", line 234, in getCacheIDForMailbox
import splunk.auth as au
File "/opt/splunk/lib/python2.6/site-packages/splunk/auth.py", line 4, in <module>
import splunk.clilib.cli_common as comm
File "/opt/splunk/lib/python2.6/site-packages/splunk/clilib/cli_common.py", line 39, in <module>
splunk_db = os.path.normpath(os.environ["SPLUNK_DB"])
File "/opt/splunk/lib/python2.6/UserDict.py", line 22, in __getitem__
raise KeyError(key)
KeyError: 'SPLUNK_DB'
Solution is tied to the environment variables being set properly. Use setSplunkEnv;
source /opt/splunk/bin/setSplunkEnv
echo $SPLUNK_DB
Everything below needs to be run from the Exchange Management SHELL. Make sure to right/click, run-as administrator...
[PS] C:\>get-service msexchangeimap4
Status Name DisplayName
------ ---- -----------
Stopped msexchangeimap4 Microsoft Exchange IMAP4
[PS] C:\>start-service -name msexchangeimap4
[PS] C:\>set-service msexchangeimap4 -startuptype automatic
WARNING: Waiting for service 'Microsoft Exchange IMAP4 (msexchangeimap4)' to
finish starting...
I have only gotten PlainText working with the IMAP App so far. TLS/SSL coming...
TechNet Article - How to Configure Authentication for IMAP4
[PS] C:\>Set-IMAPSettings -LoginType PlainTextLogin
-LoginType <PlainTextLogin | PlainTextAuthentication | SecureLogin>
This parameter specifies the authentication setting that is used for the
Client Access server that is running the IMAP4 service. The possible
values are as follows:
* 1:PlainTextLogin
* 2:PlainTextAuthentication
* 3:SecureLogin
[PS] C:\>restart-service -service msExchangeIMAP4
WARNING: Waiting for service 'Microsoft Exchange IMAP4 (msExchangeIMAP4)' to
finish starting...
[PS] C:\>get-imapsettings | fl
ProtocolName : IMAP4
Name : 1
MaxCommandSize : 10240
ShowHiddenFoldersEnabled : False
UnencryptedOrTLSBindings : {:::143, 0.0.0.0:143}
SSLBindings : {:::993, 0.0.0.0:993}
X509CertificateName : SERVERNAME
Banner : The Microsoft Exchange IMAP4 service is rea
dy.
LoginType : PlainTextLogin
AuthenticatedConnectionTimeout : 00:30:00
PreAuthenticatedConnectionTimeout : 00:01:00
MaxConnections : 2000
MaxConnectionFromSingleIP : 2000
MaxConnectionsPerUser : 16
MessageRetrievalMimeFormat : BestBodyFormat
ProxyTargetPort : 143
CalendarItemRetrievalOption : iCalendar
OwaServerUrl :
EnableExactRFC822Size : False
ExtendedProtectionPolicy : None
EnableGSSAPIAndNTLMAuth : True
AdminDisplayName :
ExchangeVersion : 0.1 (8.0.535.0)
DistinguishedName : CN=1,CN=IMAP4,CN=Protocols,CN=SERVERNAME
,CN=Servers,CN=Exchange Administrative Grou
p (FYD123OHF23SPDLT),CN=Administrative Group
s,CN=DOMAIN,CN=Microsoft Exchange,CN=Servic
es,CN=Configuration,DC=company,DC=com
Identity : SERVERNAME\1
Guid : 06a16b81-04f4-4ed8-b8dd-c212328
ObjectCategory : onpeak.us/Configuration/Schema/ms-Exch-Prot
ocol-Cfg-IMAP-Server
ObjectClass : {top, protocolCfg, protocolCfgIMAP, protoco
lCfgIMAPServer}
WhenChanged : 11/19/2010 1:46:00 PM
WhenCreated : 11/24/2009 10:26:06 PM
OriginatingServer : SERVERNAME.company.com
IsValid : True
Thanks for this, it helped me a lot!
A quick way to set the right environments, paths, etc. to work as a script running inside Splunk is to use /opt/splunk/bin/splunk cmd <command>
, e.g. /opt/splunk/bin/splunk cmd python bin/imap.py
. Pretty much that is what the command is intended for. You could also source /opt/splunk/bin/setSplunkEnv
into your shell.