I have a use case problem and could appreciate a peer review. My use case is to use a kvstore lookup as a journal to track events that have a specific rule that triggers an addition action/search.
The logic workflow is as follows;
1. event comes into Splunk a couple of times a day
2. a splunk search runs 4 times a day and identifies the event for handling.
3. for the first instance of event
4. splunk is used to normalize/creating fields for event so that the value of is mapped to Field3
5. a table command is used to verify that all required fields for follow up processing (eg field1, field2, field3, field4) are formatted correctly
6. a lookup is then used to query the kvstore based on Field3 for values in field5 (eg. | lookup kvstore field3 output field5)
7. a filter is then used to identify event ABC if it has a value in field5 that is null. (eg. I where where isnull(field5)
8. a custommand command is run; which generates a value for field5
10. the results are sent back to the kvstore using a outputlookup (| outputlookup | outputlookup append=t kvstore)
When the second splunk search runs and picks up event again, how do I tweak the above logic so as to prevent splunk from picking up that event and processing it with the customcommand? Meaning how do I prevent the duplication?
How do I use lookup to identify an existing match of a kvstore entry on top of the one that I have already defined? Or is there a better way?
How do I an outputlookup where I can drop kvstore entries after a period of time eg, like after 2 weeks?
(note all fields in my collections.conf are currently strings). Do I need to setup my kvstore definition as a temporal lookup, taking one of my fields in my collections.conf and make it a time/numeric value?
(let me try this again, was replying in the wrong place).
For #1; that means using | eval key = _key; right?
If so, how do I include that in my logic?
Is that through a lookup to match the key to my field that I have identified as unique?
Review the dev docs on kvstore. if you expose _key field you can set it to whatever you want. if you set it same each time then you are updating the row. Such as an IP address.
Thanks. One point of clarification;
For #1, that is using | eval key = _key, right? If so, how do I insert that into my logic?
Is that lookup kvstore key as Field3? (where field3 is the field of the unique value of my event)?