ES Threat Intelligence Download with POST argument

[teresachila]

I set up an Intelligence Download for https://threatfox-api.abuse.ch/api/v1  to use with the POST argument. However I am constantly getting the error: 

Caught HTTPError when querying https://threatfox-api.abuse.ch/api/v1: code=405 exc=HTTP Error 405: Method Not Allowed

I also see the log line:

file=threatlist.py:download_csv:333 | status="CSV download starting"

However this url does not return a csv. It will return a json and I am planning to use (?ms) in the extract regex to parse it. Is ES thinking that this is a csv and doing a GET instead of a POST? How do I control that? I have in the UI set the POST argument to be a json string required by the API. I am able to run curl and retrieve the output from this url.

[ownion]

Dear @teresachila,

the API you are calling is returning results on a JSON format, in order to work maybe you have to set up a scripted input to fetch the data and then create a saved search to populate a lookup and reference this lookup in the Threat Intelligence Management

Or you can configure directly in the Threat Intelligence Management a new "Threat Intelligence Source" and use the link in the "download" label to ingest the type of data you need based on:

• URLs
• Domains
• IP-Port
• MD5 Hashes
• SHA256 Hashes
• Or Full data dump (all above joined toghether)

both in recent addition or full data dump, from this link https://threatfox.abuse.ch/export/#csv in a CSV mode.

Let me know if this solve your issue.

[teresachila]

I modified my POST argument format to be xx=yy and the 405 Method Not Allowed error is gone. However then it said no indicator found in the downloaded file. Unfortunately I can't see what is downloaded, and I can't tell if my POST arguments were accepted by the server. I am giving up trying to set this up in ES. Thanks for your help though.

[venkatasri]

Hi @teresachila 

Threat intel supports POST you could check here - https://docs.splunk.com/Documentation/ES/6.6.2/Admin/Downloadthreatfeed

The formats JSON seems not supported yet- this is the old post however still a good alternative solution for JSON -> https://community.splunk.com/t5/Splunk-Enterprise-Security/Splunk-Enterprise-Security-Is-there-a-way...

The above said url expects POST method, without POST arguments threat intel inputs might be assuming as GET method. 

{
    "query_status": "http_post_expected",
    "data": "The API expects a HTTP POST request"
}

---

An upvote would be appreciated if this reply helps!