topic Re: Splunk for IMAP stop indexing data in All Apps and Add-ons

Splunk for IMAP stop indexing data

jamesm84 — Tue, 25 Dec 2012 12:19:45 GMT

We installed and configured splunk for imap.
it worked and indexed data but from some reason it stopped indexing data after a few hours.

Troubleshooting:

verified that the mailbox contains new messages
verified that the mailbox was not full.
when I ran "/opt/splunk/bin/splunk cmd python /splunk/etc/apps/imap/bin/getimap.py --debug" it connected to the mailbox but from some reason did not find any new messages.
I've deleted some of the old messages and change imap.conf filtering to: imapSearch = UNDELETED instead of "imapSearch = UNDELETED SMALLER 204800"
After the changes splunk index the new messages

I've enabled debug in imap.conf but not sure what value it adds..

I want to know why it stopped and verify it won't happen again.

Where are the imap app log files located?
How can I troubleshoot it further?

Re: Splunk for IMAP stop indexing data

kristan — Wed, 23 Jan 2013 04:57:29 GMT

Bump. I have a similar issue and the same questions.

If I run via the commandline (as James did above) and pump the output to a log file the log file will get the IMAP entries for the mail in the folder, but the mail index in splunk never gets any data

Re: Splunk for IMAP stop indexing data

pbalsley — Fri, 04 Apr 2014 05:41:56 GMT

You can always search the splunk internal index for errors for the script.

index=_internal imap source="*splunkd.log"

See what you may find.

Also you can " tail -f var/log/splunk/python.log" file too.

I also noticed that if you don't delete your email after indexing "deleteWhenDone = True" in imap.conf, then the python script can take a looooong time to find the next set of emails to index. I noticed the script back logged for 2 hours on my install. I had to purge my mail box and then enable the delete option and things were ok again.