Best practice for isolating/abstracting which indexes an app uses, to make it easy to use different indexes?

[Graham_Hanningt]

I've developed an app that contains dashboards.

For now, I have deliberately not limited the searches to specific indexes. Two reasons (but not the only reasons):

• I developed the app in Splunk Free
• I'd like to avoid hardcoding index names into the searches, because I don't (can't) dictate/predict what index names customers/users will want to use

I've now copied that app to an instance of Splunk Enterprise (Developer Trial license), where I have defined various indexes.

For testing purposes, I now want to limit the dashboards to searching specific indexes; I don't want the results to be "polluted" by other indexes that contain data of the same sourcetype in the same time range.

I'm also looking ahead to the possibility of publishing the app on Splunkbase. In which case, I might include with the app one or more indexes of sample data.

I'd like to define my searches in the app in a way that makes it easy to "redirect" those searches to use different indexes without requiring me (or a user who downloads the app) to edit index=myindex conditions in multiple search strings (whether in multiple saved searches, or in base searches defined in the Simple XML of the app's dashboards).

One thought that occurs to me is defining tokens at the app level (that can then be used in searches), but, as far as I know, that feature is just wishful thinking.

This question is similar to a question from 2011, "How do I set a default search index per app?".

[esix_splunk · Accepted solution]

One of the best methods for this is to define a macro in your app. And in that macro, you define the indexes and/or sourcetypes used..

So your macro, named 'test' in this case, would be

index=mycustom_index sourcetype=mysourcetype

Then your dashboards and reports can all be..

`test` | stats count by sourcetype

etc.. etc..

When you distribute / deploy the app, all you have to do is adjust the macro instead of all the searches in the dashboards and reports..

That's within the best practices and will get you the most mileage...

[Graham_Hanningt]

Thanks very much!

My app uses data of various source types that might, depending on the user's preferences, be stored in different indexes or stored together in a single index, such as main.

For what it's worth - to match my particular requirements - here's what I've added to my app's macros.conf (where myapp is just a placeholder for my actual app name):

# Edit these definitions to point the app searches to your indexes
[myapp_search_start_default]
definition = index=main

[myapp_search_start_cmf]
definition = `myapp_search_start_default`

[myapp_search_start_dtr]
definition = `myapp_search_start_default`

[myapp_search_start_ims]
definition = `myapp_search_start_default`

[myapp_search_start_smf]
definition = `myapp_search_start_default`

where each macro name suffix (except for "_default") corresponds to a "family" of source types. Actual example source types are more specific, such as ims_ca01, smf_4601.

I've inserted a reference to the appropriate macro at the start of each query in the Simple XML of my app's dashboard definitions.

For now, I've left the sourcetype= conditions hardcoded in the queries, after the reference to one of these macros. I'm still thinking about that; I can see pros and cons to having fixed sourcetype values.

I've defined the macros this way to allow for the possibility of storing different source types in different indexes, but so that you only have to edit a single macro if you're storing all source types in a single index.

I deliberately decided to prefix ("namespace") the macro names with the app name to avoid potential clashes with macros that might be defined elsewhere. Actually, I made that decision before deciding on these relatively long macro names.

I had originally imagined creating a macro named index that would take a single parameter, and I'd use that parameter value in a case function in the macro definition. Something like this (not working code):

[index(1)]
args = arg
definition = (fuzzy notion as to what to put here) case( \
    $arg$=cmf, "index=index_containing_cmf",\
    $arg$=dtr, "index=index_containing_dtr",\
    TRUE, "index=default_fallback_index")

but it dawned on me that, as far as I can tell - although it might be possible, which is why I'm bothering mentioning this discarded approach - there is no way to inject the result of that case function back into the search appropriately. I'm reasonably happy with the multi-macro approach I've ended up with, but I remain curious about the possibility of using case.