Splunk Enterprise Security

ES Threat Intelligence Download with POST argument

teresachila
Path Finder

I set up an Intelligence Download for https://threatfox-api.abuse.ch/api/v1  to use with the POST argument. However I am constantly getting the error: 

Caught HTTPError when querying https://threatfox-api.abuse.ch/api/v1: code=405 exc=HTTP Error 405: Method Not Allowed

I also see the log line:

file=threatlist.py:download_csv:333 | status="CSV download starting"

However this url does not return a csv. It will return a json and I am planning to use (?ms) in the extract regex to parse it. Is ES thinking that this is a csv and doing a GET instead of a POST? How do I control that? I have in the UI set the POST argument to be a json string required by the API. I am able to run curl and retrieve the output from this url.

Labels (1)
0 Karma

ownion
Path Finder

Dear @teresachila,

the API you are calling is returning results on a JSON format, in order to work maybe you have to set up a scripted input to fetch the data and then create a saved search to populate a lookup and reference this lookup in the Threat Intelligence Management

Or you can configure directly in the Threat Intelligence Management a new "Threat Intelligence Source" and use the link in the "download" label to ingest the type of data you need based on:

  • URLs
  • Domains
  • IP-Port
  • MD5 Hashes
  • SHA256 Hashes
  • Or Full data dump (all above joined toghether)

both in recent addition or full data dump, from this link https://threatfox.abuse.ch/export/#csv in a CSV mode.

Let me know if this solve your issue.

0 Karma

teresachila
Path Finder

I modified my POST argument format to be xx=yy and the 405 Method Not Allowed error is gone. However then it said no indicator found in the downloaded file. Unfortunately I can't see what is downloaded, and I can't tell if my POST arguments were accepted by the server. I am giving up trying to set this up in ES. Thanks for your help though.

0 Karma

venkatasri
SplunkTrust
SplunkTrust

Hi @teresachila 

Threat intel supports POST you could check here - https://docs.splunk.com/Documentation/ES/6.6.2/Admin/Downloadthreatfeed

The formats JSON seems not supported yet- this is the old post however still a good alternative solution for JSON -> https://community.splunk.com/t5/Splunk-Enterprise-Security/Splunk-Enterprise-Security-Is-there-a-way...

The above said url expects POST method, without POST arguments threat intel inputs might be assuming as GET method. 

{
    "query_status": "http_post_expected",
    "data": "The API expects a HTTP POST request"
}

---

An upvote would be appreciated if this reply helps!

 

Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...