topic Re: Update indexed data after a monitored file has been changed in Splunk Search

Update indexed data after a monitored file has been changed

mkohl — Fri, 27 Sep 2019 08:13:18 GMT

Dear friends,

with my company besides investigating log-data we are getting ready to roll-out splunk for the Business Team in order to monitor the business. We got some tables which we would like to display as dashboard visualizations, e.g.

ID Name ID2 Name2 Date Volume
1 A1 11 B1 01.01.2019 30000
2 ...

We managed to monitor the csv-files with the upper table, search for relevant data and plot it for dashboard visualiztion.
Now comes the question: How can we update the plots correctly, after the monitored files have been updated?
The data is mainly static and will only be altered inline, eg.
ID Name ID2 Name2 Date Volume
1 A1 11 B1 01.01.2019 99999 (before: 30000)
2 ...

Is there a function which can easily do the job, or do we always have to clean the index an reload/re-index all the files?
We want to avoid the commands: splunk stop/splunk clean eventdata/splunk start

Best regards
Matthias

Re: Update indexed data after a monitored file has been changed

gcusello — Fri, 27 Sep 2019 08:30:51 GMT

Hi mkohl,
indexed data aren't upgradable!
But you can index data and show only the last in a period: you cannot think to Splunk as a DB: Splunk indexes logs, so you can display them thinking to the time distribution.
In other words, if you have some values to display you can use stats command, something like this:

index=my_index
| stats latest(ID) AS ID latest(Name) AS Name latest(ID2) AS ID2 latest(Name2) AS Name2 latest(Date) AS Date latest(Volume) AS Volume

and automatically run this search every e.g. five minutes to have an always updated situation.

Bye.
Giuseppe

Re: Update indexed data after a monitored file has been changed

techiesid — Fri, 27 Sep 2019 08:33:51 GMT

Hello,

You can use dedup command to achieve this.

For more informationplease find the below link,

https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/dedup

Sid