Currently there is not a JSON parser built into the Threat Intelligence Framework in Splunk ES. That said, if the Threat Intel provider has an app or TA and you can get the the data into a Splunk index, you can use a saved search to push it into either a KVStore lookup or CSV lookup that the Threat Intelligence Framework already monitors.
As an example, the iSight Partners intelligence feed is JSON based, and their app sorts out the download and save to index part of the problem. To get this integrated into the Threat Intelligence framework of ES you can simply create a saved search similar to the following.
index="isightpartners" domain !=null | eval description="iSight Partners Intel" |table domain,description | outputlookup append=T local_domain_intel
The localdomainintel is a csv based lookup that is written to
That specifically covers domain intel, but the rules apply across all threat categories (also the "description" field I created with eval since it's option for all intel and generally speaking is likely not in the source JSON). Here's a list of the associated "fields" you will want to write to for each category of intel - note that these also have other associated csv backed look files in the same directory.
email_intel: src_user subject file_intel: file_hash file_name http_intel: http_referrer http_user_agent url ip_intel (by domain) domain ip_intel (by ip) ip process_intel process process_file_name registry_intel registry_path registry_value_name registry_value_text service_intel service service_file_hash service_dll_file_hash user_intel user certififate_intel certificate_issuer certificate_issuer_email certificate_issuer_organization certificate_issuer_unit certificate_serial certificate_subject certificate_subject_email certificate_subject_organization certificate_subject_unit
So if your JSON threat intel also contained info with file names and hashes, the aforementioned search could be tweaked for that case as well.
index="isightpartners" domain !=null | eval description="iSight Partners Intel" |table description, file_hash, file_name | outputlookup append=T local_file_intel
Please note that the fields in this case just happened to already have the right naming convention, you can always use eval to rename the raw, JSON-based intel to the right field names (as listed in the table above).
If you don't mind sharing, what is the threat intel provider you are using?
Thanks kcchamplin. The iSightPartner API now allows you pull STIX files directly to the monitored threat intelligence directories and Splunk picks them up very well (helps with getting the file based indicators into splunk).
The intelligence provider is ThreatGrid. Their API is currently JSON only...Right now I am toying with the idea of downloading the JSON files, converting them to CSV and having a monitored directory pick them up. I don't see a ThreatGrid App (TA or otherwise). Did I miss it?
Hey panovattack - very good to know regarding STIX direct download for iSight. As far as an app or TA for ThreatGrid, that would be awesome, but you're right there's not one currently available on Splunkbase. As far as rolling your own, you're on the right track for sure - you should be able to append to the previously listed files, there's also a custom inputs.conf stanza you can use:
[threatlist://my_custom_json_intel] delim_regex = , description = JSON_based_intel_test disabled = false fields = domain:"$1",description:"$2" ignore_regex = (^#|^\s*$|^Start) type = threatlist url = lookup://ip_intel_domain_lookup
Note that the fields again correspond to the ones outlined in my first post, and given its CSV, the "delim_regex" can remain a comma. Then you just use the "fields" stanza line to then map (in left to right order of your CSV file) the values to each intel category/component.
Since its using a
lookup:// for the URL, you'll want to have the lookup configured in your transforms.conf
[ip_intel_domain_lookup] filename = ip_intel_domain.csv
You can of course decide how your actual lookup file is formatted (if you decide that comma-separated is out of fashion, make sure to change your delim-regex to whatever your delimited is).