All Apps and Add-ons

Is there a timeout in the TA-dmarc add-on for Splunk that would cause the process to die before its finished processing the messages?

swharper79
Engager

We have a mailbox with a large number of emails we're attempting to ingest into Splunk (over 150,000). Before any data is ingested the process is timing out (connection reset by peer). Is there a timeout in the TA that would cause the process to die before its finished processing the messages?

2018-08-21 14:16:40,062 ERROR pid=21172 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/TA-dmarc/bin/ta_dmarc/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc_imap.py", line 88, in collect_events
    input_module.collect_events(self, ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/input_module_dmarc_imap.py", line 48, in collect_events
    filelist = i2d.process_incoming()
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 218, in process_incoming
    response = self.get_dmarc_message_bodies(new_messages)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 88, in get_dmarc_message_bodies
    response = self.server.fetch(messages, ['RFC822'])
  File "/opt/splunk/etc/apps/TA-dmarc/bin/imapclient/imapclient.py", line 971, in fetch
    tag = self._imap._command(*args)
  File "/opt/splunk/lib/python2.7/imaplib.py", line 872, in _command
    raise self.abort('socket error: %s' % val)
abort: socket error: [Errno 104] Connection reset by peer
Collapse
2018-08-21 14:16:40,059 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | Success deleting temporary directory /tmp/tmpIxrlSr
2018-08-21 14:16:39,906 INFO pid=21172 tid=MainThread file=base_modinput.py:log_info:293 | Start processing 154245 new messages of 154245 on 10.168.16.246
2018-08-21 14:16:39,905 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | filter_seen_messages: uids new       set([20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,.......
0 Karma

jorritf
Path Finder

Thanks for the report, I'll look into it when I have time somewhere next week.
Can you create an issue in the Github tracker?

0 Karma
Get Updates on the Splunk Community!

March Community Office Hours Security Series Uncovered!

Hello Splunk Community! In March, Splunk Community Office Hours spotlighted our fabulous Splunk Threat ...

Stay Connected: Your Guide to April Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars in April. This post ...

Want to Reduce Costs, Mitigate Risk, Improve Performance, or Increase Efficiencies? ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...