Please notice I used 2 Servers here, one is Splunk forward , another is Splunk enterprise. "Cisco eStreamer eNcore Add-on for Splunk" install on Splunk forward. "Cisco Firepower eNcore App for Splunk" install on Splunk enterprise. "Geo Location Lookup Script (powered by MAXMIND)" install on Splunk enterprise. GeoLite2 databases unzip on Splunk enterprise. Installation Download "Cisco eStreamer eNcore Add-on for Splunk" from https://splunkbase.splunk.com/app/3662/ (cisco-estreamer-encore-add-on-for-splunk_356.tgz) Download "Cisco Firepower eNcore App for Splunk" from https://splunkbase.splunk.com/app/3663/ (cisco-firepower-encore-app-for-splunk_353.tgz) Download "Geo Location Lookup Script (powered by MAXMIND)" from https://splunkbase.splunk.com/app/291/ Download GeoLite2 databases from http://geolite.maxmind.com/download/geoip/database/GeoLite2-City.tar.gz down XXXX.pkcs12 from FMC and upload it on Splunk forward here: $SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/client.pkcs12 PKCS12 file must be renamed as “client.pkcs12” Configuration on Splunk forward Edit "/opt/splunk/etc/apps/TA-eStreamer/bin/configure.sh" Modify "exec &>configuration.log" to "exec >>configuration.log 2>&1" Navigate to app settings in Splunk – from the home page, click the “cog” icon Find Cisco eStreamer eNcore for Splunk and click “Set-up” At a minimum: ■ enter the “FMC hostname or IP address” and IP XXX.XXX.XXX.XXX Port 8302 ■ check the “Process PKCS12 file?”. No password here Note: Each time you load this page, “Process PKCS12 file” is reset to “no” and the password is not saved. It should be used once to process the PKCS12 file using openSSL and store a public-private key pair. Select "Packets? Packet logs can be large and use up storage" and "Connections? This is a very high-volume option and may consume significant network and storage usage" Click "Save" Enable the data inputs on Splunk forward Navigate to Settings > Data Inputs > Files & Directories and enable the single TA-eStreamer app input (cisco:estreamer:data) – this is the where the main output data files are saved Navigate to Settings > Data Inputs > Scripts and enable the three TA-eStreamer inputs ■ cisco:estreamer:clean – this script has no output but is used to delete data files older than 12 hours ■ cisco:estreamer:log – this script uses the stdout of eNcore to take program log data. This becomes very useful where things are not going to plan ■ cisco:estreamer:status – this script runs periodically to maintain a clear status of whether the program is running or not Execution on Splunk forward Once you have fully configured the collector and enabled the inputs, navigate back to the set-up page in app settings, enable eNcore (“is enabled?”) and press save. To check the status, search for sourcetype="cisco:estreamer:status" To check more detailed log output, search for sourcetype="cisco:estreamer:log" To look for eStreamer data, search for sourcetype=" cisco:estreamer:data" Troubleshooting If you see Error while posting to url=/servicesNS/nobody/TA-eStreamer/encore/configure/main when you press Save in the setup screen, then please search the logs for more information: Search: index=_internal source="*splunkd.log" AdminManagerExternal If you are getting less data than you are expecting or just want to see what the eStreamer client is doing, then search: sourcetype="cisco:estreamer:log" (ERROR OR WARNING). To see more detail, remove the ERROR and WARNING constraints. replace GeoLite2-City.mmdb on Splunk Enterprise upload "GeoLite2-City_yyyymmdd.tar.gz" to the Splunk server path /opt/splunk/share tar -xvf GeoLite2-City_yyyymmdd.tar.gz cd GeoLite2-City_yyyymmdd cp GeoLite2-City.mmdb ../ filter the logs on Splunk forward cd /opt/splunk/etc/apps/TA-eStreamer/local ==props.conf== [cisco:estreamer:data] TRANSFORMS-forward-data-to-receiver = forward_receiver [cisco:estreamer:log] TRANSFORMS-drop-data = setdrop [cisco:estreamer:status] TRANSFORMS-drop-data = setdrop ==transforms.conf== [forward_receiver] REGEX = sec_intel_list1 DEST_KEY = _TCP_ROUTING FORMAT = default-autolb-group [setdrop] REGEX = . DEST_KEY = queue FORMAT = nullQueue configure forward on Splunk forward settings->Forwarding and receiving->Configure forwarding "Add new" Host: xxx.xxx.xxxx.xxx:9997 configure receiving on Splunk Enterprise settings->Forwarding and receiving->Configure receiving "Add new" Listen on this port:9997 ... View more

YES, the "Connection Events" is on the FMC side. TA-eStreamer is only used to receive the Events from the FMC and can do some filter. I don't remember the other configurations on the TA-eStream configuration page. I'll check my documents on Monday and give you more information. Yes, I used "props.conf" and "transforms.conf" to only forward the "Intelligence Events" to the Splunk. You can follow the Splunk's documents to modify it by yourself, if you need some other filter conditions. https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/propsconf https://docs.splunk.com/Documentation/Splunk/7.2.6/Admin/Transformsconf ... View more

@hatalla you need to choose the "Connection Events" Login FMC, go to "System" -> "Integration" -> "eStreamer" -> "Connection Events" But be careful, the data grows very fast. I'm using a Splunk forward as a filter, only forward the Intelligence Event to the Splunk Enterprise. And you also need to set the "Forward data" and "Receive data" in the Splunk forward and Splunk Enterprise /opt/splunk/etc/apps/TA-eStreamer/local# cat props.conf [cisco:estreamer:data] TRANSFORMS-forward-data-to-receiver = forward_receiver [cisco:estreamer:log] TRANSFORMS-drop-data = setdrop [cisco:estreamer:status] TRANSFORMS-drop-data = setdrop /opt/splunk/etc/apps/TA-eStreamer/local# cat transforms.conf [forward_receiver] REGEX = sec_intel_list1 DEST_KEY = _TCP_ROUTING FORMAT = default-autolb-group [setdrop] REGEX = . DEST_KEY = queue FORMAT = nullQueue ... View more

how about the dynamic field name? such as : {"Troj/DocDl-QUA": 4, "CXmail/OleDl-AU": 44, "CXmail/EncDoc-B": 6, "Troj/DocDl-QVV": 10, "Troj/DocDl-QVQ": 10, "Troj/DocDl-QWV": 7, "CXmail/OleDl-AD": 341, "Troj/DocDl-QWN": 25, "Troj/DocDl-QVF": 4, "Troj/DocDl-QVC": 8} {"Troj/DocDl-QUA": 4, "CXmail/OleDl-AU": 591, "Troj/DocDl-QOQ": 10, "Troj/DocDl-QUE": 9, "Troj/RtfExp-EQ": 1, "Troj/DocDl-QTD": 5, "Troj/DocDl-QUI": 21, "CXmail/OleDl-AD": 868, "CXmail/OleDl-V": 9, "Troj/PDFDown-X": 4} I want to transfer it like: Troj/DocDl-QUA 4 CXmail/OleDl-AU 44 CXmail/EncDoc-B 6 Troj/DocDl-QVV 10 Troj/DocDl-QVQ 10 Troj/DocDl-QWV 7 CXmail/OleDl-AD 341 Troj/DocDl-QWN 25 Troj/DocDl-QVF 4 Troj/DocDl-QVC 8 Troj/DocDl-QUA 4 CXmail/OleDl-AU 591 Troj/DocDl-QOQ 10 Troj/DocDl-QUE 9 Troj/RtfExp-EQ 1 Troj/DocDl-QTD 5 Troj/DocDl-QUI 21 CXmail/OleDl-AD 868 CXmail/OleDl-V 9 Troj/PDFDown-X 4 so I can sum the same virus type and draw chart. Thanks! ... View more

You are correct! I reviewed my other data also in json-format but not as json-array, only use "rename" can makes it work fine. Seems I need spend more time on json and splunk extract data. Thanks again! ... View more

Thanks somesoni2. It works, but why it's so complicated! following is the final search language: sourcetype="cisco:opendns:api" | rename organizations{}.name AS group, organizations{}.requests.allSecurity AS allSecurity, organizations{}.requests.blocked AS blocked, organizations{}.requests.contained AS contained, organizations{}.requests.prevented AS prevented, organizations{}.requests.security AS security, organizations{}.requests.total AS total | table group, allSecurity, blocked, contained, prevented, security, total | head 1 | eval temp=mvzip(mvzip(mvzip(mvzip(mvzip(mvzip(group,allSecurity,"#"), blocked,"#"),contained,"#"),prevented,"#"),security,"#"),total,"#") | table temp | mvexpand temp | rex field=temp "(?[^#]+)#(?[^#]+)#(?[^#]+)#(?[^#]+)#(?[^#]+)#(?[^#]+)#(?[^#]+)" | fields group, allSecurity, blocked, contained, prevented, security ... View more

Thanks Maciep! Remove the "spath" command can resolve this issue. ... View more

Thanks for replying. But this splunk is just a forwarder, don't have any license in it. Can I still using the "indexQueue"? ... View more

[cisco:estreamer:data] TRANSFORMS-send-data-to-null-queue = setnull_2, setnull_1 [cisco:estreamer:log] TRANSFORMS-drop-data = setdrop [cisco:estreamer:status] TRANSFORMS-drop-data = setdrop [setnull_1] REGEX = ^fw_rule_reason\=N\/A$ DEST_KEY = queue FORMAT = nullQueue [setnull_2] REGEX = ^((?!rec_type\=71).)*$ DEST_KEY = queu FORMAT = null Queue [setdrop] REGEX = . DEST_KEY = queue FORMAT = nullQueue Seems the issue is not about the REGEX, it's about how to make multi-REGEX to work. Every time when I restart the service to test it, I only can have one correct logs. rec_type not equal "71" works or fw_rule_reason=N/A works. ... View more

the regex pass the "https://regex101.com/" online test, but still doesn't work. Does anyone know why? ... View more

[setnull_2] REGEX = ^((?!rec_type\=71).)*$ DEST_KEY = queu FORMAT = null Queue This website is really helpful "https://regex101.com/" ... View more

I made a stupid mistake. The NullQueue is like drop something. So the the REGEX should be "sec_intel_event=No'. Thanks all your help, guys! ... View more

props.conf and transforms.conf are in heavy forwarder. Still not working, all "cisco:estreamer:data" forwarded to the indexer. I only want "sec_intel_event=Yes" forward to indexer. /opt/splunk/etc/apps/TA-eStreamer/local# cat props.conf [cisco:estreamer:data] TRANSFORMS-send-data-to-null-queue = setnull /opt/splunk/etc/apps/TA-eStreamer/local# cat transforms.conf [setnull] REGEX = (sec_intel_event=Yes) DEST_KEY = queue FORMAT = nullQueue ... View more

Shit, it's the Firefox problem. I'm using Firefox 61.0.1 (64-bit), looks like compatible issue. I switch to Chrome and login successfully!!! Hope you guys can repeat this bug with the same version of Firefox. ... View more

It doesn't work, I did it before but set the password as "changeme" and this time I set the password by using random characters. Same issue. CLI authentication successful but web interface failed. Strong password policy issue? password length? special characters? any limits? My random password has 10 characters and contains upper-case and lower-case letters, numerical digits and special characters. ... View more

I changed the props.conf and transforms.conf as following. But seems no event is forwarded. ===props.conf=== [cisco:estreamer:data] TRANSFORMS = setnull ===transforms.conf=== [setnull] REGEX = (sec_intel_event=Yes) DEST_KEY = queue FORMAT = nullQueue ... View more

Do I need to escap the "=" in REGEX such as "REGEX = (sec_intel_event=Yes)" ... View more

I want to forward "Security Intelligence Events", props.conf and transforms.conf are saved in "/opt/splunk/etc/apps/TA-eStreamer/local" props.conf [monitor://$SPLUNK_HOME/etc/apps/TA-eStreamer/data] TRANSFORMS-set = setnull Transforms.conf [setnull] REGEX = (sec_intel_event=Yes) DEST_KEY = queue FORMAT = nullQueue But it doesn't work, who can help me? Thanks! ... View more