I have two apps which have the same saved searches defined in their own savedsearches.conf files, however, rather than one overriding the other based on standard conf file precedence, I see both versions listed in the web site and also have duplicate reports being sent when scheduled. I would assume that only version of each saved search would be active, as they use the same stanza names in the savedsearches.conf files. Is there something that I am missing? Does file precedence not apply to savedsearches.conf.
As background information, I have a main app which is used to contain all my custom configurations, which includes saved searches. These saved searches include scheduled reports, scheduled lookups generations. All of these work as expected. I will call this app "Prod"
On the same Splunk server, I have a copy of "Prod" App and placed in a new app called "UAT". This app was created by taking a copy of the "PROD" app and modifying the app.conf to change the app name and also modified the savedseach.conf file - specifically the request.uidispatchapp value to add the new app name. The "UAT" app is then loaded through the GUI using the "Manage Apps" interface. This installation worked successfully and I am able to use this as expected.
So the "Prod" app should have precedence over the "UAT" app. When I run btool for the savedsearch.conf file, I only see one entry which should be the correct result, but both versions of each search are visible, active and manageable through the saved searches web interface. For the saved searches that produce lookup files, both run and update the correct lookup folder under each app.
Here's a post on how to limit the scope of .conf stanzas -
Basically, the UAT version should be in a folder at the app level, where the prod version is global.
Other than that, if you're getting duplicate reports, check to see whether the UAT version is scheduled as well as the PROD version. To find out, you could temporarily kill the UAT's access to the indexes (for example) and see whether you still got dups when the prod one ran.
If I understand correctly, then this is not an issue with precedence, but with application scope. As these saved searches are defined only within the app scope (they have not been globally published) - then it would make sense that from each app's perspective, these searches are unique and so each app will run these. Is that correct?
I was approaching this issue more from a perspective that the conf files were all being processed once and that then defines the configuration Splunk uses- similar to the output we get from btool which does not take app/user context into account- but re-reading the documentation, it seems to be more dynamic than that - so depending on the app/user context different settings will "win" and therefore be used and you can have multiple activities running in parallel using different configurations.
What I want to achieve is that I can have the same settings in both and only one will be active. I think this can be achieved by making saved searches from both apps global in scope and therefore only one can be active at any one time. This makes it easier to promote UAT to Prod, as I know the Prod version will then be the active one. And only new searches that I add to UAT will be active.
Thanks for the information - I will start testing the scope settings.
If they are only to be defined at the app level, then probably both of them should be placed in the app-level folders.
Be extra careful when promoting any tested changes, though. (A word to the wise is sufficient).