topic [Bug] Download threat intelligence feed ignoring Timeout setting in Splunk Enterprise Security

[Bug] Download threat intelligence feed ignoring Timeout setting

FloSwiip — Thu, 19 Dec 2019 14:23:32 GMT

Hello,

Working on a threatq list which takes more than 1min to be generated, I was always looping in splunk with :

status="threat list download failed after multiple retries"

And I have discovered that in the Intelligence Download Settings, the field Timeout is a lie, as it is not used for real in the code behind.
In the script /opt/splunk/etc/apps/SA-ThreatIntelligence/bin/threatlist.py the value is set to a variable timeout

self.DEFAULT_TIMEOUT_INTERVAL = 30
(......)
IntegerField("timeout", "Timeout interval", "Time before regarding a download attempt as failed, in seconds.  [Defaults to {0}]".format(self.DEFAULT_TIMEOUT_INTERVAL), required_on_create=True, required_on_edit=True),

But after the call of /opt/splunk/etc/apps/SA-Utils/lib/SolnCommon/protocols.py we have :

_timeout = 30  # The timeout for queries conducted by this handler.
(......)
def set_options(self, *args, **kwargs):

    valid_keys = ['app', 'debug', 'owner', 'proxy_port',
                  'proxy_server', 'proxy_user', 'proxy_password',
                  'site_user', 'site_password', 'user_agent']
(......)
    try:
        response = urllib2.urlopen(request, timeout=self._timeout)

So at the end the http timeout is always to 30s max whatever you will set

¯\(ツ)/¯

Re: [Bug] Download threat intelligence feed ignoring Timeout setting

FloSwiip — Thu, 19 Dec 2019 14:26:47 GMT

I post my crap and dirty solution in case...

Edit /opt/splunk/etc/apps/SA-ThreatIntelligence/bin/threatlist.py look for the part :

        handler = handler_cls(self._logger, self._input_config.session_key, **handler_args)
        temp_checkpoint_filehandle = None

and add just after it

        handler._timeout = handler_args.get('timeout')

Re: [Bug] Download threat intelligence feed ignoring Timeout setting

FloSwiip — Wed, 22 Dec 2021 10:24:39 GMT

The bug is still there.

As the code was a bit reworked now the solution is to :

Edit /opt/splunk/etc/apps/SA-ThreatIntelligence/bin/threatlist.py
and add at line 497
handler._timeout = handler_args.get('timeout')