All Apps and Add-ons

Anomali ThreatStream App not processing snapshots from API?

guarisma
Contributor

Hello,

We've setup our Splunk Search Head to download snapshots from ThreatStream API directly, while troubleshooting, we observed that it was downloading the snapshots from hxxps://ts-optic.s3.amazonaws.com/snapshots/... but then had issues processing it.

 

 

 

 

2022-11-03 02:01:47,394 18860 ERROR threatstream_app - threatstream_kvstore> Autologin succeeded, but there was an auth error on next request. Something is very wrong.
2022-11-03 02:01:47,443 18860 ERROR threatstream_app - threatstream_kvstore> Failed at add_kvs_batch - sz == 1, collection_name: ts_md5, data: [{'date_last': '2016-02-21T14:52:32.000Z', 'id': '0', '_key': '99929352'}]
2022-11-03 02:01:47,443 18860 ERROR threatstream_app - threatstream_kvstore> Autologin succeeded, but there was an auth error on next request. Something is very wrong.
2022-11-03 02:01:47,464 18860 ERROR threatstream_app - threatstream_kvstore> Failed at add_kvs_batch - sz == 1, collection_name: ts_md5, data: [{'date_last': '2016-02-21T14:52:37.000Z', 'id': '0', '_key': '99929603'}]
2022-11-03 02:01:47,464 18860 ERROR threatstream_app - threatstream_kvstore> Autologin succeeded, but there was an auth error on next request. Something is very wrong.
2022-11-03 02:01:48,677 18860 INFO threatstream_app - ioc_loader> 193571 items with id="0" saved to kvs: ts_md5 for deletion, time: 35505.908512592316
2022-11-03 02:01:48,678 18860 INFO threatstream_app - ioc_loader> 193571 items with id="0" saved to kvs: ts_md5 for deletion, time: 35505.908512592316
2022-11-03 02:01:49,059 18860 ERROR threatstream_app - ts_ioc_ingest> failed to download optic intelligence: Autologin succeeded, but there was an auth error on next request. Something is very wrong.
2022-11-03 02:01:49,059 18860 ERROR threatstream_app - ts_ioc_ingest> failed to download optic intelligence: Autologin succeeded, but there was an auth error on next request. Something is very wrong.
2022-11-03 02:01:49,933 18860 ERROR threatstream_app - ts_ioc_ingest> Traceback (most recent call last):
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 290, in wrapper
    return request_fun(self, *args, **kwargs)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 71, in new_f
    val = f(*args, **kwargs)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 622, in delete
    response = self.http.delete(path, self._auth_headers, **query)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1169, in delete
    return self.request(url, message)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1255, in request
    raise HTTPError(response)
splunklib.binding.HTTPError: HTTP 401 Unauthorized -- call not properly authenticated

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 232, in _handle_auth_error
    yield
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 301, in wrapper
    return request_fun(self, *args, **kwargs)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 71, in new_f
    val = f(*args, **kwargs)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 622, in delete
    response = self.http.delete(path, self._auth_headers, **query)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1169, in delete
    return self.request(url, message)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 1255, in request
    raise HTTPError(response)
splunklib.binding.HTTPError: HTTP 401 Unauthorized -- call not properly authenticated

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/splunk/etc/apps/threatstream/bin/ts_ioc_ingest.py", line 284, in download_iocs
    TmDataManager(splunka=remote_splunk, logger=logger).process_data()
  File "/opt/splunk/etc/apps/threatstream/bin/ts/tm_data_manager.py", line 176, in process_data
    self._process_data()
  File "/opt/splunk/etc/apps/threatstream/bin/ts/tm_data_manager.py", line 245, in _process_data
    self.load_from_lookup_files()
  File "/opt/splunk/etc/apps/threatstream/bin/ts/tm_data_manager.py", line 508, in load_from_lookup_files
    iocs.load_iocs()
  File "/opt/splunk/etc/apps/threatstream/bin/ts/lookup_iocs.py", line 404, in load_iocs
    util.utils.remove_0_id_values(self.kvsm, kvs)
  File "/opt/splunk/etc/apps/threatstream/bin/util/utils.py", line 143, in remove_0_id_values
    remove_delete_id_values(kvsm, ioc_kvs_name, 'id', '0')
  File "/opt/splunk/etc/apps/threatstream/bin/util/utils.py", line 146, in remove_delete_id_values
    kvsm.delete_kvs(kvs, {id_name : delete_id_value})
  File "/opt/splunk/etc/apps/threatstream/bin/util/kvs_manager.py", line 286, in delete_kvs
    collection.data.delete(query=json.dumps(query_dict))
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/client.py", line 3678, in delete
    return self._delete('', **({'query': query}) if query else {})
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/client.py", line 3631, in _delete
    return self.service.delete(self.path + url, owner=self.owner, app=self.app, sharing=self.sharing, **kwargs)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 301, in wrapper
    return request_fun(self, *args, **kwargs)
  File "/opt/splunk/lib/python3.7/contextlib.py", line 130, in __exit__
    self.gen.throw(type, value, traceback)
  File "/opt/splunk/etc/apps/threatstream/bin/splunklib/binding.py", line 235, in _handle_auth_error
    raise AuthenticationError(msg, he)
splunklib.binding.AuthenticationError: Autologin succeeded, but there was an auth error on next request. Something is very wrong.

 

 

 

 

So I guess "Something is wrong"? but what?

Anyone knows a solution or at least the cause of this?

 

Labels (2)
Tags (3)
0 Karma

starcher
SplunkTrust
SplunkTrust

I would open a support ticket with anomali. That’s their code. Something in way they are trying to hit kvstore. 

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...