Splunk Enterprise Security

ES - Threat Intelligence - FS-ISAC Feeds

Attempting to ingest feeds from FS-ISAC into ES.
I can see in splunk that a file is created:
2018-06-19 17:01:28,107 INFO pid=23553 tid=MainThread file=stixparser.py:preprocess:154 | msg="Finished parsing STIX documents" filename="/opt/splunk/etc/apps/SA-ThreatIntelligence/local/data/threatintel/fsisacfilehashTAXIIfilehash2018-06-19T17-01-22.135143.xml" success="0" failed="0"

ls -lah /opt/splunk/etc/apps/SA-ThreatIntelligence/local/data/threatintel/
total 12K
drwx--x---. 2 splunk splunk 4.0K Jun 19 16:56 .
drwx--x---. 3 splunk splunk 25 Oct 17 2016 ..
-rw-------. 1 splunk splunk 483 Jun 19 16:56 fsisac
allhighTAXIIallhigh2018-06-19T16-56-22.004935.xml
-rw-------. 1 splunk splunk 483 Jun 19 16:56 fsisac
filehashTAXIIfilehash_2018-06-19T16-56-21.863297.xml

Below are the contents of the file:
2018-06-19T21:01:06.060327+00:00

2018-06-19T21:01:06.101416+00:00

Whats strange is the file is quickly deleted and ever poll, splunk re-creates the file, then deletes it again. I never see any of the threat intelligence, I've disabled all other feeds in an attempt to get this to work and I don't see anything on the "Threat Intelligence > Threat Activity" dashboard.

I've:
1. Created multiple feeds on analysis.fsisac[dot]com
2. Created multiple Threat Intelligence Downloads in an attempt to get data from any of them (see inputs below):

I don't see any errors associated with feeds.

Status of fsisac threatintelinternallogs:

eventtype=threatintelinternallogs fsisac | stats count by status
status count
TAXII feed polling starting 5450
continuing 5450
retrievedcheckpointdata 5300
Retrieved document from TAXII feed 4307
nocheckpointdata 150
Detected updated threatlist stanzas - ALL lookup gen searches will be executed 5

inputs for fsisac:

[threatlist://fsisac]
description = FS-ISAC threat intel
index = internal
initial
delay = 300
interval = 300
maxage = -1y
post
args = collection="Default" earliest="-1y" taxiiusername="redacted" taxiipassword="redacted" certfile="redacted.crt" keyfile="redacted.key"
retries = 3
retry_interval = 60
source = ModularInput:Threatlist
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50

[threatlist://fsisac2]
delim
regex = ,
description = FS-ISAC threat intel
ignoreregex = (^#|^\s*$)
index = _internal
initial
delay = 300
interval = 300
maxage = -1y
post
args = taxiiusername="redacted" taxiipassword="redacted" certfile="redacted.crt" keyfile="redacted.key"
retries = 3
retryinterval = 60
skip
header_lines = 0
source = fsisac
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/taxii-discovery-service
weight = 50

[threatlist://fsisacallhigh]
delimregex = ,
description = FS-ISAC threat intel
ignore
regex = (^#|^\s*$)
index = internal
initial
delay = 300
interval = 300
postargs = collection="allhigh" taxiiusername="redacted" taxiipassword="redacted" certfile="redacted.crt" keyfile="redacted.key"
retries = 3
retryinterval = 60
skip
headerlines = 0
source = fsisac
filehash
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50

[threatlist://fsisacfilehash]
delim
regex = ,
description = FS-ISAC threat intel
disabled = 0
ignoreregex = (^#|^\s*$)
index = _internal
initial
delay = 300
interval = 300
postargs = collection="filehash" taxiiusername="redacted" taxiipassword="redacted" certfile="redacted.crt" keyfile="redacted.key"
retries = 3
retry
interval = 60
skipheaderlines = 0
source = fsisac_2
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50

Path Finder

These settings worked for me:

NOTE: Make sure you put your .crt and .key file issued by FS-ISAC in the auth folder of the app directory you create the input inside of (e.g. /opt/splunk/etc/apps/DA-ESS-ThreatIngelligence/auth). You can verify you're connecting successfully by reviewing the threat intel download logs (index=_internal sourcetype=threatintel:download)

[threatlist://fs-isac-default]
delim_regex = ,
description = FS-ISAC system.Default feed
ignore_regex = (^#|^\s*$)
interval = 43200
is_threatintel = 1
max_age = -30d
post_args = collection="system.Default" earliest="-1y" taxii_username="<your_provided_username>" taxii_password="<your_password>" cert_file="<your_cert.crt>" key_file="<your_key.key>"
retries = 3
retry_interval = 60
sinkhole = 0
skip_header_lines = 0
timeout = 30
type = taxii
url = https://analysis.fsisac.com/taxii-discovery-service
weight = 1
0 Karma

Explorer

These generally worked for me as well, but I would note that some of the stanza's are invalid if you are not on a more current version of Splunk/ES combo. I would recommend starting out without sinkhole and is_threatintel otherwise the taxii polling won't even start. I found this out by restarting the splunk service and paying attention to the error streams that show up in the startup output. Errors will look like the following:

Invalid key in stanza
[threatlist://fs-isac-default] in
/opt/splunk/etc/apps/SA-ThreatIntelligence/local/inputs.conf, line 47: is_threatintel (value: 1).
Invalid key in stanza
[threatlist://fs-isac-default] in
/opt/splunk/etc/apps/SA-ThreatIntelligence/local/inputs.conf, line 52: sinkhole (value:0).

I overlooked the original note on .crt and .key files need to be placed in the auth/ folder within the app.

I found the following conf talk from 2017 as well which may help fill in some details for folks. Though it doesn't mention ISAC data specifically could be a good primer for others.

https://conf.splunk.com/files/2017/slides/enterprise-security-biology-dissecting-the-splunk-enterpri...

0 Karma

Explorer

Any updates on this thread?

0 Karma

Path Finder

"What's strange is the file is quickly deleted and ever poll, Splunk re-creates the file, then deletes it again. I never see any of the threat intelligence."

Under Threat Intelligence Management you can remove the sinkhole policy that deletes the files

0 Karma