All Apps and Add-ons

Is there a timeout in the TA-dmarc add-on for Splunk that would cause the process to die before its finished processing the messages?

swharper79
Engager

We have a mailbox with a large number of emails we're attempting to ingest into Splunk (over 150,000). Before any data is ingested the process is timing out (connection reset by peer). Is there a timeout in the TA that would cause the process to die before its finished processing the messages?

2018-08-21 14:16:40,062 ERROR pid=21172 tid=MainThread file=base_modinput.py:log_error:307 | Get error when collecting events.
Traceback (most recent call last):
  File "/opt/splunk/etc/apps/TA-dmarc/bin/ta_dmarc/modinput_wrapper/base_modinput.py", line 127, in stream_events
    self.collect_events(ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc_imap.py", line 88, in collect_events
    input_module.collect_events(self, ew)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/input_module_dmarc_imap.py", line 48, in collect_events
    filelist = i2d.process_incoming()
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 218, in process_incoming
    response = self.get_dmarc_message_bodies(new_messages)
  File "/opt/splunk/etc/apps/TA-dmarc/bin/dmarc/imap2dir.py", line 88, in get_dmarc_message_bodies
    response = self.server.fetch(messages, ['RFC822'])
  File "/opt/splunk/etc/apps/TA-dmarc/bin/imapclient/imapclient.py", line 971, in fetch
    tag = self._imap._command(*args)
  File "/opt/splunk/lib/python2.7/imaplib.py", line 872, in _command
    raise self.abort('socket error: %s' % val)
abort: socket error: [Errno 104] Connection reset by peer
Collapse
2018-08-21 14:16:40,059 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | Success deleting temporary directory /tmp/tmpIxrlSr
2018-08-21 14:16:39,906 INFO pid=21172 tid=MainThread file=base_modinput.py:log_info:293 | Start processing 154245 new messages of 154245 on 10.168.16.246
2018-08-21 14:16:39,905 DEBUG pid=21172 tid=MainThread file=base_modinput.py:log_debug:286 | filter_seen_messages: uids new       set([20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55,.......
0 Karma

jorritf
Path Finder

Thanks for the report, I'll look into it when I have time somewhere next week.
Can you create an issue in the Github tracker?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...