Exclude events that do not occur each time a file is indexed

[lbogle]

Hello Splunkers,
Question. I index and report on my departments IT Service Level Agreement ticket stats for them using Splunk. I have a number of queries that are successfully returning the desired information but I have noticed that as tickets get transferred out of our ticket queue, tickets are still showing up in the reports from the time they were indexed initially. Because Splunk saw it once but does now not see it, the record of the ticket still persists in the reports. I guess my question is: How do write a query to exclude something that is not there?

[lguinn2 · Accepted solution]

Once the data is indexed in Splunk, it remains in the index until it ages out. Each time you upload a csv, it does not replace the prior information, it simply adds to it.

If you don't want to keep the data, or you want to replace it each day/month/week/whatever, you should make the csv file a lookup file. As a lookup, you would be using only the latest information in your searches. Here is a tutorial on Splunk field lookups.

Think of the Splunk index as containing event data: "This thing happened at this time." The index is not normally a good place to store most other kinds of data, such as "This is the current state" (although it is great for taking a snapshot: "This is the state at this time.")

[lbogle]

I was hoping it would be you that responded 🙂
Your answers are always well thought out and well written.
That was exactly how I understood things to work and a lookup table may work here perfectly here now that you mention it.
Thank you very much!

[lguinn2]

Thanks LB :$ (blushing)

[lbogle]

The file in question is a report file (.csv) containing data only for my department that is manually generated and exported and then indexed into Splunk. The same index is used to collect total resolved/historical tickets.