Attempting to ingest feeds from FS-ISAC into ES.
I can see in splunk that a file is created:
2018-06-19 17:01:28,107 INFO pid=23553 tid=MainThread file=stix_parser.py:preprocess:154 | msg="Finished parsing STIX documents" filename="/opt/splunk/etc/apps/SA-ThreatIntelligence/local/data/threat_intel/fsisac_filehash_TAXII_filehash_2018-06-19T17-01-22.135143.xml" success="0" failed="0"
ls -lah /opt/splunk/etc/apps/SA-ThreatIntelligence/local/data/threat_intel/
total 12K
drwx--x---. 2 splunk splunk 4.0K Jun 19 16:56 .
drwx--x---. 3 splunk splunk 25 Oct 17 2016 ..
-rw-------. 1 splunk splunk 483 Jun 19 16:56 fsisac_all_high_TAXII_all_high_2018-06-19T16-56-22.004935.xml
-rw-------. 1 splunk splunk 483 Jun 19 16:56 fsisac_filehash_TAXII_filehash_2018-06-19T16-56-21.863297.xml
Below are the contents of the file:
2018-06-19T21:01:06.060327+00:00
2018-06-19T21:01:06.101416+00:00
Whats strange is the file is quickly deleted and ever poll, splunk re-creates the file, then deletes it again. I never see any of the threat intelligence, I've disabled all other feeds in an attempt to get this to work and I don't see anything on the "Threat Intelligence > Threat Activity" dashboard.
I've:
1. Created multiple feeds on analysis.fsisac[dot]com
2. Created multiple Threat Intelligence Downloads in an attempt to get data from any of them (see inputs below):
I don't see any errors associated with feeds.
Status of fsisac threatintel_internal_logs:
eventtype=threatintel_internal_logs fsisac | stats count by status
status count
TAXII feed polling starting 5450
continuing 5450
retrieved_checkpoint_data 5300
Retrieved document from TAXII feed 4307
no_checkpoint_data 150
Detected updated threatlist stanzas - ALL lookup gen searches will be executed 5
inputs for fsisac:
[threatlist://fsisac]
description = FS-ISAC threat intel
index = _internal
initial_delay = 300
interval = 300
max_age = -1y
post_args = collection="Default" earliest="-1y" taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
source = ModularInput:Threatlist
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50
[threatlist://fsisac_2]
delim_regex = ,
description = FS-ISAC threat intel
ignore_regex = (^#|^\s*$)
index = _internal
initial_delay = 300
interval = 300
max_age = -1y
post_args = taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
skip_header_lines = 0
source = fsisac
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/taxii-discovery-service
weight = 50
[threatlist://fsisac_all_high]
delim_regex = ,
description = FS-ISAC threat intel
ignore_regex = (^#|^\s*$)
index = _internal
initial_delay = 300
interval = 300
post_args = collection="all_high" taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
skip_header_lines = 0
source = fsisac_filehash
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50
[threatlist://fsisac_filehash]
delim_regex = ,
description = FS-ISAC threat intel
disabled = 0
ignore_regex = (^#|^\s*$)
index = _internal
initial_delay = 300
interval = 300
post_args = collection="filehash" taxii_username="redacted" taxii_password="redacted" cert_file="redacted.crt" key_file="redacted.key"
retries = 3
retry_interval = 60
skip_header_lines = 0
source = fsisac_2
sourcetype = ModularInput:Threatlist
target = threatlist
timeout = 30
type = taxii
url = https://analysis.fsisac.com/
weight = 50
... View more