Splunk Search

Firewall Log: How to send API calls to abuseIPDB and deal with results in splunk to get threat score?

Engager

Hi there,
I have been dealing with Splunk for two weeks now. My intention was to make firewall drops from an Unifi Security Gateway visible and evaluate them. The first step was successful and Splunk automatic field extraction made it easy to deal with.
The next step was to get detailed information about SRC IP addresses.
So I did: <search> | lookup dnslookup clientip as SRC OUTPUTNEW clienthost as hostname, added Geo-IP fields to a dataset and created a cluster map. However, the DNS lookup is very time consuming, but the result is nevertheless interesting. The next step is to investigate suspicious SRC IPs to get a threat score. So I tried the Splunk App "IP Reputation", but the results from the honeypot project are very poor. Only about one in 1000 attacker IPs is known. Much more information and known bad IPs come from abuseIPDB.com. Their documentation describes the API call:

# The -G option will convert form parameters (-d options) into query parameters.
# The CHECK endpoint is a GET request.
curl -G https://api.abuseipdb.com/api/v2/check \
  --data-urlencode "ipAddress=118.25.6.39" \
  -d maxAgeInDays=90 \
  -d verbose \
  -H "Key: $YOUR_API_KEY" \
  -H "Accept: application/json"

and the API response:

    This will yield the following JSON response:

  {
    "data": {
      "ipAddress": "118.25.6.39",
      "isPublic": true,
      "ipVersion": 4,
      "isWhitelisted": false,
      "abuseConfidenceScore": 100,
      "countryCode": "CN",
      "countryName": "China",
      "usageType": "Data Center/Web Hosting/Transit",
      "isp": "Tencent Cloud Computing (Beijing) Co. Ltd",
      "domain": "tencent.com",
      "totalReports": 1,
      "numDistinctUsers": 1,
      "lastReportedAt": "2018-12-20T20:55:14+00:00",
      "reports": [
        {
          "reportedAt": "2018-12-20T20:55:14+00:00",
          "comment": "Dec 20 20:55:14 srv206 sshd[13937]: Invalid user oracle from 118.25.6.39",
          "categories": [
            18,
            22
          ],
          "reporterId": 1,
          "reporterCountryCode": "US",
          "reporterCountryName": "United States"
        }
      ]
    }
  }

I tried a lot of combinations to embed the described API call into a Splunk search with no success. A curl call (with a valid API key) in a shell returns the expected response.
Is there anybody who could guide me to the right way how the Splunk search should look like and how to extract the response fields like "abuseConfidenceScore, countryName,..." in Splunk?
I hope that's not too ambitious for my current level of Splunk knowledge.

Currently, I have Splunk Enterprise 7.3.0 on Ubuntu 18.04.2 LTS and I want to continue with Splunk free version (advanced home usage).

Kind regards
Roland

Engager

Hi starcher,
thanks for reply. As I was afraid of, I see, I have to go much deeper in splunk to create custom search commands. I had hoped, that anybody has already solved this and can give me an example.
My main local DNS Server is a pi-hole which is pretty fast, delivers most DNS requests from cache and has a nice GUI to check this.

Kind regards
Roland

0 Karma

SplunkTrust
SplunkTrust

You need to look into custom search commands and write one to talk to the api and add fields back to the event.

Also always do a search | stats | then lookup on dns lookup so you are not doing the same lookup over and over. OR do local caching of DNS and ensure you use that.

http://dev.splunk.com/view/python-sdk/SP-CAAAEU2
http://www.georgestarcher.com/splunk-dns-lookup-performance-and-caching-with-dnsmasq/

Example streaming command
https://github.com/georgestarcher/TA-esreplacefields

0 Karma