I am trying to get our Add-on that was developed for standalone Splunk to work in a SHC environment. The Add-on takes input from the user in a setup view and saves the configuration values via custom endpoint using the Splunk JS SDK. When Set up is run on a standalone instance we get custom fields from the system we are connecting to and create the modular alert html using the custom REST endpoint (also stored in /data/ui/alert/sa_myapp.html). Is there a way to replicate the modular alert html across the search had cluster members if running Setup from the Deployer? As far as I can tell the Setup needs to be run on each search head member to generate the html for that node and this conflicts with SHC best practices with Setup run only on the deployer and pushing the conf files to the SHC members. Setup may need to be rerun for the Add-on if custom fields are added or deleted in the system we are connecting to, to change the html used for mapping the fields between Splunk and our system. Is there a solution so that Setup can only be run on the deployer? How can I replicate the html across the cluster members? In my investigation the file /data/ui/alert/sa_myapp.html is not replicated across the search heads. If Setup is run on each search head cluster member the html is generated. It is my understanding that Setup should not be run on the SHC members but only on the deployer.  Can Setup run on the deployer post to the custom endpoint on each SHC member? ... View more

I am trying to get our Add-on that was developed for standalone Splunk to work in a SHC environment. The Add-on takes input from the user in a setup view and saves the configuration values via the REST API using the Splunk JS SDK.  I am able to replicate  our sa_our_app.conf by adding this stanza in server.conf: [shclustering] conf_replication_include.sa_our_app = true We are able to replicate the setup view in the UI across the search  head members. The Add-on also uses a custom REST endpoint during setup to write the modular alert html (stored in /data/ui/alert).  Is there a way to replicate this html across all members of the SHC? ... View more

I am working on a Splunk Add-on that was developed on stand alone Splunk that I am now testing in a search head cluster environment. The Add-on uses a modular script that sends Splunk search data or notable event data to a 3rd party app when a saved search is matched. Our Add-on ships 2 saved searches for users to use to verify that the add-on is configured properly with the 3rd party app. On stand alone Splunk (with Splunk ES also installed) both of these saved searches are triggered when there is a failed Splunk server login that is logged in splunkd: index=_internal sourcetype=splunkd ERROR UiAuth                                            (for Splunk without Splunk ES) index=_internal sourcetype=splunkd ERROR UiAuth | `get_event_id`        (for Splunk ES) In the SHC environment, when there is a failed login, we are only seeing one of the two rules triggered (it is not consistent which one is triggered). Can anyone explain why only one rule would be triggered in SHC environment? I’m trying to figure out if the SHC is not setup properly, if there is something the Add-on is not doing properly or if there is some other issue? We are currently testing on Splunk 8.1.0 and Splunk ES 6.1.1, CIM 4.15.0. Do you think upgrading Splunk to 8.2 would make any difference? Thanks for any help! AnnMarie ... View more

A customer has been using our Splunk add-on with a reverse proxy and they reported that the  add-on stopped working with our latest release that uses a custom REST endpoint for setup of the add-on. In the screenshot below, I can see that the reverse proxy root_endpoint (/esa01) is used in the first 2 calls to GET when setting up that dashboard (these errors are usually there).  However in the subsequent GET and POST calls with use of the custom endpoint the root_endpoint is not used.  The add-on does not specify the base URL when using a custom REST endpoint, so I am wondering if there is an issue with the Splunk JavaScript SDK?  Or is our add-on missing something to support reverse proxy? ... View more