Whats the best practice provide data to display current indicators

[marcfelden]

Hi there,
i have a ticketing system which uses a Database to store ticket information.
On a splunk dashboard I want to present current informations about how many open tickets, the escalation status of the tickets, time a ticket spent without someone taking action, etc.

Question:
At the moment I am connecting every minute to the database, getting the data and putting the result in a CSV file to a path that splunk then uses as a lookup input. The file gets overwritten. it works but does not give me any history.

Is there a more elegant way to handle data like this?

[richgalloway]

If you index the CSV instead of treating it as a lookup file you'll have history. You'll have to edit your dashboard to pull only the most recent data, however.

[marcfelden]

Thanks for the quick answer. The problem with that is that it would create tons of dublicates depending on the frequency by which I pull my data. And how can I possibly know which are the "most recent" data? I'm interested in the "best" solution for this problem. As I am using a PHP Script to fetch the data and give it to splunk I can do everything with the data before giving it to splunk.

[richgalloway]

It's the "tons of duplicates" that give you the history you seek. Perhaps your PHP script can extract only new information from the database to help reduce the amount of duplicate data. Find the most recent data using the dedup or stats latest() command.