All Apps and Add-ons

Is there a timeout in the TA-dmarc add-on for Splunk that would cause the process to die before its finished processing the messages?

swharper79
Engager

We have a mailbox with a large number of emails we're attempting to ingest into Splunk (over 150,000). Before any data is ingested the process is timing out (connection reset by peer). Is there a timeout in the TA that would cause the process to die before its finished processing the messages?

2018-08-21 14:16:40,062 ERROR pid=21172 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/TA-dmarc/bin/ta_dmarc/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc_imap.py", line 88, in collect_events
    input_module.collect_events(self, ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/input_module_dmarc_imap.py", line 48, in collect_events
    filelist = i2d.process_incoming()
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 218, in process_incoming
    response = self.get_dmarc_message_bodies(new_messages)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 88, in get_dmarc_message_bodies
    response = self.server.fetch(messages, ['RFC822'])
  File "/opt/splunk/etc/apps/TA-dmarc/bin/imapclient/imapclient.py", line 971, in fetch
    tag = self._imap._command(*args)
  File "/opt/splunk/lib/python2.7/imaplib.py", line 872, in _command
    raise self.abort('socket error: %s' % val)
abort: socket error: [Errno 104] Connection reset by peer
Collapse
2018-08-21 14:16:40,059 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | Success deleting temporary directory /tmp/tmpIxrlSr
2018-08-21 14:16:39,906 INFO pid=21172 tid=MainThread file=base_modinput.py:log_info:293 | Start processing 154245 new messages of 154245 on 10.168.16.246
2018-08-21 14:16:39,905 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | filter_seen_messages: uids new       set([20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,.......
0 Karma

jorritf
Path Finder

Thanks for the report, I'll look into it when I have time somewhere next week.
Can you create an issue in the Github tracker?

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...