Hello. I'm fairly new to Splunk and SPL so bear with me here.
I have the following scenario:
I have an existing lookup file that was created by a search and is then updated daily by a similar saved search.
So to sum it up, run a search, append contents of the lookup file, remove old events, and finally output the data to the lookup file again overwriting the old contents of the lookup file. If the search with the appended lookup file data and after clean-up results in zero events I still want the lookup file to remain.
Now, when reading the Splunk docs I get a bit confused regarding create_empty and override_if_empty optional arguments.
For create_empty, Splunk docs state "If set to true and there are no results, a zero-length file is created." So since outputlookup normally overwrites the file if it already exists is this the case even when writing no results?
Same question for override_if_empty, which seems to be doing something similar. If override_if_empty is set to false, does outputlookup overwrite the lookup file with a zero length list when the search has no results?
My saved search to update the lookup file looks approximately like this:
| "get external data"
| fields blah blah blah
| fields - _*
| rename blah blah blah
| eval time=now()
| inputlookup "my existing lookup file" append=true
| sort 0 - time
| where time > relative_time(now(), "-7d@d") OR isnull(time)
| outputlookup "my existing lookup file"
So do I need to add create_empty=true and override_if_empty=false?
Or do I just need one of them, and if so which one?
Grateful for any clarification on this matter. Thanks in advance.
If the lookup file already exists then the create_empty setting doesn't apply.
The override_if_empty setting was simple to test and does not touch the lookup file when set to false and there are no results.
Thanks for your reponse.
So assume the following scenario. My search returns zero results, and the events in the lookup file are all older than a week and as such are removed by the housekeeping part of the query. I still want a file to exist even though it's empty. But create_empty will not help and the zero results I have won't be written to the existing file due to it already existing. And override_if_empty won't touch the existing file either.
Is it possible to chain the two parameters somehow so that the old file is removed and a new, empty file is created with the same name? Like:
override_if_empty=true (delete the old file) and then create_empty=true (to create a new file with the same name)
If that works, will it create problems with lookup definitions, or anything else?
You could experiment with various combinations of settings to see if one does want you need. Unfortunately, SPL has limited conditional logic abilities that don't include the situation you describe.
Thanks for trying to help. I wish this information was readily available in docs considering search driven lookups are a thing in Splunk.
Having to experiment to try and figure out how certain basic parameters and functions operates is not optimal in my opinion. It is kind of crucial to understand how to manage different scenarios like a saved search outputting no data to a lookup file for example.