Use different lookups based upon environment in same search

[harry2007gsp]

How can I use same search for 2 different lookup?
For ex: lookup_qa.csv and lookup_prod.csv.
I wanna use them in search (for ex: search_testing) so that when I use the search for qa data, it uses lookup_qa.csv and when i use search for prod data , it uses lookup_prod.csv.
Basically rest of the code of that search is gonna be same for both environments but only lookups would switch.
Please help.

[elliotproebstel · Accepted solution]

Let's say you write this search so it can query source="qa_data.csv" or source="prod_data.csv" and then you want to use lookup_qa.csv for the first source and lookup_prod.csv for the second. If you have interesting_field in both sources and want to compare them to a field called interesting_lookup_field in your lookup tables, I'd structure my query like this:

index=something source="qa_data.csv" OR source="prod_data.csv"
| eval qa_lookup_field=if(source="qa_data.csv", interesting_field, NULL)
| eval prod_lookup_field=if(source="prod_data.csv", interesting_field, NULL)
| lookup lookup_qa.csv lookup_field AS qa_lookup_field OUTPUT something_from_lookup
| lookup lookup_prod.csv lookup_field AS prod_lookup_field OUTPUT something_from_lookup
| fields - qa_lookup_field prod_lookup_field

This separates out the interesting_field into two separate fields so that they are looked up in their respective lookup files.

[harry2007gsp]

my main data is coming from index which has only 1 column common with both lookup tables. So these lookup tables are used to just join additional column in the already existing data (from indexes). So I want that when qa index is in use, use | lookup lookup_qa.csv and ignore lookup_prod.csv and when prod index is in use and do the vice versa.

[elliotproebstel]

Sure thing. This search structure should work for that, because the field qa_lookup_field will remain null if you are searching prod data, and prod_lookup_field will remain null if you are searching qa data. Consequently, the lookups for null fields will get skipped, and you'll only get data from the lookup you actually wanted to use, based on the source/index.

[woodcock]

Schedule this search to run every night:

|inputlookup lookup_qa.csv
| eval index="search_testing"
| appendpipe [
|inputlookup lookup_prod.csv
| eval index="search_prod" ]
| outputlookup lookup_combined.csv

Then, in your search do this:

.... | lookup lookup_combined.csv index your other input fields here OUTPUT your output fields here

[harry2007gsp]

I wanna keep my lookup data separate.
I need to run search only with qa data with lookup_qa.csv when qa index is provided as parameter to the saved search.

[somesoni2]

How can you identify the environment in which the search is running? Is there any field data that identifies it, or in your Splunk server name??

[harry2007gsp]

Index is gonna identify the environment
We have different indexes for different environments.

[somesoni2]

So in your query, there is one more thing different, index name?? Can you share the base search portion (everything before first pipe symbol) of the searches you run?