topic Re: best way to index automatically and update dashboard with unique file name in Dashboards & Visualizations

best way to index automatically and update dashboard with unique file name

Javo222 — Mon, 28 Sep 2020 17:56:42 GMT

Hi,

I have a script that uploads a file named file.log to a folder on my computer (let's say C:\Users\Public\Documents\splunk_data). I have managed to extract some fields like CPU_usage or Memory_usage and draw it in a dashboard, using the following string:

source="file.log" Total_CPU_Usage=* | timechart span=1h max(Total_CPU_Usage)

That works fine, however now my script will upload a new file.log to this folder and I would like splunk to Index it automatically and also update the dashboard with this new data ONLY. I don't want to see the old data.

How can I:
- Make sure Splunk indexes automatically this file? It seems it's not indexed using "Continuously index data from a file or directory this Splunk instance can access" to "C:\Users\Public\Documents\splunk_data".
- Make sure the old data is removed from the server, or at least not shown in the dashboard.

At the moment, I'm renaming any new file into fileX.log and uploading it manually. Then I go to the dashboard and change the source to fileX.log . I'm sure there is a better way to do that 🙂

thanks!

Re: best way to index automatically and update dashboard with unique file name

somesoni2 — Mon, 28 Sep 2020 17:54:35 GMT

Assuming you have access to physical server where Splunk resides and both the file and splunk are on same server, do the following:-

Go the machine where Splunk is installed and go to the local folder of the splunk app where you're creating your dashboard. (it would be something like c:\Program Files\Splunk\etc\apps\YourAppName\local.
Create/update a file name inputs.conf and put following

[monitor://C:\Users\Public\Documents\splunk_data\file.log]
disabled = false
followTail = 0
sourcetype = YourSourceType
index = YourIndex
Restart your splunk instance. This will enable automatic monitoring on the file C:\Users\Public\Documents\splunk_data\file.log and will update all changes to splunk automatically. The same can be done from Splunk Web as well See this (http://docs.splunk.com/Documentation/Splunk/6.1.4/Data/UseSplunkWeb)
Change your dashboard query to this to get the latest indexed file.

source="file.log" Total_CPU_Usage=* _indextime=[search source="file.log" | head 1 | eval Date=_indextime | return $Date] | timechart span=1h max(Total_CPU_Usage)

Updated

source="file.log" Total_CPU_Usage=*  [search  source="file.log" | head 1 | eval Date=_indextime | eval search="_indextime=".Date | table search] | timechart span=1h max(Total_CPU_Usage)

Re: best way to index automatically and update dashboard with unique file name

Javo222 — Fri, 17 Oct 2014 21:42:22 GMT

Yes I have access to the server and the file. I've tried your instructions but the search seems wrong, I can't see what is missing. I get the following error:

Error in 'search' command: Unable to parse the search: Comparator '=' is missing a term on the right hand side.

Re: best way to index automatically and update dashboard with unique file name

somesoni2 — Fri, 17 Oct 2014 21:56:57 GMT

Try with updated query.