We have had this same issue in our environment.  The fix we have come up with is to create an app for the 4300X events that are specific to cisco:ftd where we parse out all the fields (you can use kvmode=auto for these, but some fields like url don't get extracted correctly since they oftentimes have '=' in the url string).   In order to address the other message IDs that match number/format with cisco:asa, we make a 'custom' app and update it each time we update Splunk_TA_cisco-asa.  In that app (we named it Splunk_TA_cisco-asa-ftd) we just copy over the /default/ and /local/ props.conf files and change the sourcetype declaration from [cisco:asa] to [cisco:ftd].  The other files (like transforms) aren't needed because splunk already has those definitions in the Splunk_TA_cisco-asa app, you just need to tell it to do all the eval/extract/transform/etc functions from props.conf for the other sourcetype.  If you use eventtypes, you should also update that. We updated Splunk_TA_cisco-asa/local/eventtypes.conf to use 'sourcetype IN (cisco:asa, cisco:ftd)' to address that issue in the 'standard' app. I know that seems like a lot of customization, but after doing the customization / upgrade a few times, it's not so bad.  🙂  ... View more

Were you able to get this resolved?  We are seeing the same issue with some of our users after upgrade to v9.1.2.   Thanks! ... View more

Having the SPLUNK_HOME/etc/.ui_login file on the host DOES make the banner go away (as noted in the linked articles above). We did a bit of testing (since some of our servers had the file and others did not)... Logging in with a LOCAL account (non-LDAP/SAML/etc) automatically generates the .ui_login file.  Once we logged in with the LOCAL account, the .ui_login file existed and the "First time signing in?..." banner was gone. ... View more

We are having this same issue and for us the resolution was to remove $decideOnStartup from the host field when creating the input. ... View more

We were able to find a temp fix... $SPLUNK_HOME/etc/splunk-launch.conf has a setting PYTHONHTTPSVERIFY that allows for some additional security when setting it to 1 (part of the TLS security fixes from June 2022). Setting it to 0 (using the default value from $SPLUNK_HOME/etc/splunk-launch.conf.default) causes the 'service unavailble' issues to go away when exporting. However, this is not the "FIX" for the issue... just a temp fix for your users while you (we?) try to figure out what the real issue is with the self-signed cert validation errors (the 'real' issue) in the logs. ... View more

We are also having this same error after our upgrade from v8.2.7.1 to v9.0.3. We are at a loss as to what happened. Did you guys ever figure it out @sh254087 @splunkadmin5678 ? ... View more

Me again... While deploying this we noted that the TA-eStreamer/bin/encore/data/splunk directory being the data directory causes more problems than not. The newest problem being that the /bin directory is replicated, so if the heavy-forwarder has any searchpeers, it will cause bundle replication issues because Splunk will be attempting to replicate 200gb+ data directory all over the place. We have opted to move the data directory back to the old location of TA-eStreamer/data in the TA--eStreamer/bin/encore/estreamer.conf file.  This addresses the issues with: /bin/splencore.sh clean() location being incorrect /default/inputs.conf monitor location for data files being incorrect /bin/encore/data/splunk being 200gb+ causes replication issues if the hfw has any searchpeers   ... View more

I found one more thing today when I was testing the v4.0.11 update.  I noticed that the estreamer.conf process wasn't stopping when I stopped splunk and that the .pid file wasn't getting deleted when splunk stopped either.  It was almost like the estreamer process wasn't dependent on the splunk service. After running a diff command against the estreamer.conf from v3.6.8 and the new one for v4.0.11, I noticed that was exactly what happened.  The part of the script noting that it should be depending on splunk has been removed. Adding lines 2-4 back to the TA-eStreamer/bin/encore/estreamer.conf file re-added the splunk service dependency. { "conditions": [ "splunk" ], ... View more

I have become intimately familiar with the eStreamer TA over the last couple of years.  Let me see if I can help with some of these. setup.xml was removed in v4.0.x, so the configuration that was previously done with two passes through setup.xml in the GUI or TA-eStreamer/local/encore.conf now has to be done by manually editing the TA-eStreamer/bin/encore/estreamer.conf file, which is not nearly as easy-peasy as using the GUI. Packets, Connections, & Metadata (not mentioned earlier - but seems worth noting since it could be a data hog and is completely left out of the new instructions) In addition to manually enabling and setting the hosts in TA-eStreamer/bin/encore/estreamer.conf, you also have to manually enable/disable packets, connections, and metadata options that were previously available via checkboxes on the bottom of the setup page. packets are enabled by default - which could be problematic since packet data is quite large these options are in the "records" section of estreamer.conf as info, our previous configuration which had connections enabled, but packets and metadata disabled is below.   "records": { "connections": true, "core": true, "excl@comment": [ "These records will be excluded regardless of above (overrides 'include')", "e.g. to exclude flow and IPS events use [ 71, 400 ]" ], "exclude": [], "inc@comment": "These records will be included regardless of above", "include": [], "intrusion": true, "metadata": false, "packets": false, "rna": true, "rua": true } },     Data Directory Change (affects inputs.conf & clean() function of splencore.sh script) As noted above the data directory has changed to TA-eStreamer/bin/encore/data/splunk/ The filename format has also changed from encore.EPOCHTIME.log to encore.logEPOCHTIME There are multiple ways you can address this.  Either change where the data lives or point everything to the new locale. Change Where the Data Lives Update the "uri" in the "handler" section of TA-eStreamer/bin/encore/estreamer.conf back to the old value:   "uri": "relfile:///../../data/encore.{0}.log"   Update Where the App Looks Add a new monitor stanza in TA-eStreamer/local/inputs.conf for the new data path:   [monitor://$SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/data/splunk]   Update the path in the clean() stanza of the TA-eStreamer/bin/splencore.sh script to the new data path:   clean() { # Delete data older than 12 hours -> 720mins # find ../../data -type f -mmin +720 -delete # correcting path to new path in new version 4.0.11 of TA find $SPLUNK_HOME/etc/apps/TA-eStreamer/bin/encore/data/splunk -type f -mmin +720 -delete }     first_pkt_sec EVAL Error The EVAL statement triggering the error looks like it was a FIELDALIAS that someone switched over to an EVAL without actually switching it.   The culprit:   EVAL-first_pkt_sec = event_sec as first_pkt_sec​   The fancy EVAL we wrote to address this coalesces several time fields to ensure the 'first_pkt_sec' field gets populated.   EVAL-first_pkt_sec = coalesce(first_pkt_sec, connection_sec, event_sec)​   You could also accomplish this with a simple eval that will override the EVAL triggering the issue.   EVAL-first_pkt_sec = event_sec ​     Other Props Fixes We also noted that the search-time props had conflicting FIELDALIAS functions, no KV_MODE, and a few other things; so we added some additional flare to address those issues. Just in case this might also be helpful.   [cisco:estreamer:data] #### Setting the time format to epoch time (not set in TA) TIME_FORMAT = %s #### Setting KV_MODE #### KV_MODE = auto #### Splunk CIM - Intrusion Detection Fields #### EVAL-severity = coalesce(severity, priority) EVAL-signature = coalesce(case(signature="",null(),true(),signature), detection, msg) #### Splunk CIM - Malware Fields #### EVAL-url = coalesce(url, uri)     If it wasn't clear - the first_pkt_sec and "other props fixes" were all applied to our TA-eStreamer/local/props.conf file. ... View more

Yes, we did. We have been able to get it working like a champ. However, I would be a poor Splunk Admin if I didn't say this, as well: THIS IS HUGELY RESOURCE INTENSIVE. Ok. Now that that's out of the way... 🙂 Inputs > data_integrity_checks/local/inputs.conf [script://./bin/integrity_checks.sh] disabled = 0 interval = 3 10 * * 6 index = system_events sourcetype = audit:integrity Script > data_integrity_checks/bin/integrity_checks.sh #!/bin/bash IDX='/opt/splunk/etc/slave-apps/data_integrity_checks/local/dataintegrity_indexes.txt' for index in $(<$IDX); do $SPLUNK_HOME/bin/splunk check-integrity -index $index -verbose 2>&1; done The script runs based on the list of indexes on the dataintegrity_indexes.txt file, or you could just pass a list of index names. ... View more

Did you just manually break the event after msg= ? ... View more

We are having the same issue. Did you ever get this fixed? I've heard suggestions from someone that it could be the syslog message length is too short, but I cannot find any guidance from CyberArk for how to set that. Someone also suggested that it could be the UseLegacySyslogFormat parameter, but again - not sure what the current settings are and am not seeing much guidance one way or the other (except for this article and its unaccepted answer). Note - our events were correct at one time, but must have been borked with an upgrade or some other configuration change. However, we have also lost our CyberArk admin since then and the new admins are - new. ... View more

Seems like the long way around - but it works like a charm! ... View more

The only thing I can think to try would be to have two totally different, yet exactly the same 'setnull' stanzas in transforms.conf > setnullA for sourcetypeA and setnullB for sourcetypeB each with their own 'setparsing' as you have done already. ... View more

We are trying to do something similar with a loop to catch all of our indexes: #!bin/bash for path in /opt/data/*/*; do splunk check-integrity -index ${path##*/} -verbose; done However, I have noticed that the results of the command are only output to splunkd.log. I can find the output of the commands in the "_internal" index, but not in the index specified in inputs.conf Do you have any guidance on how to get the results ingested into an index other than _internal? ... View more

I am having this same issue - v7.2.1. Has there been any progress on a fix for this? ... View more

I tried this out with "host=$host$" in my collect statement and no-dice. Any other ideas? ... View more

We are having the same issue. Creating an alert, giving it a logevent action, giving it a triggered_alerts action... but nothing is written to the indicated index, which is a custom index defined within our environment. I noticed quite a few parameters under 'advanced edit' that are mostly referenced in one of these two places: https://docs.splunk.com/DocumentationStatic/CshrpSDK/2.2.8/Splunk.Client/html/1c2f4fe3-c78c-40bc-1853-c37a3ac37b5e.htm http://dev.splunk.com/view/javascript-sdk/SP-CAAAEKZ Can anyone help with what parameters need to be defined to: 1. Write to the 'summary' index? 2. Write to custom index 'index_A'? 3. Appear in Triggered Alerts under the activity pulldown in the top nav bar. ... View more

We were having a similar situation in our environment and the issue ended up being that the group in question lived in an Active Directory branch outside the defined groupBaseDN value. Once we added that branch to the groupBaseDN filter, access worked like a champ. Original Value: groupBaseDN = OU=Server_Groups,DC=Enterprise,DC=MyDomain,DC=com Updated Value: groupBaseDN = OU=Server_Groups,DC=Enterprise,DC=MyDomain,DC=com;OU=Access_Groups,DC=Enterprise,DC=MyDomain,DC=com ... View more

We had the same error, but with a totally different cause. When we deployed the bundle with the updated Splunk_TA_windows app from the Deployer to the Search Head Cluster, we used the "save lookups" option (as we always do) which caused the new lookup files not to be pushed to the SHC. We were unsure whether a redeployment without the save-lookups option would accidentally overwrite other lookup directories that SHOULD be saved, so we manually pushed the new Splunk_TA_windows/lookups directory to each of the SHC members. ... View more