topic Re: Is there a way to have a list of forwarded files in Splunk? in Splunk Search

Is there a way to have a list of forwarded files in Splunk?

mawomommoh — Tue, 13 Mar 2018 20:48:34 GMT

I have multiple xml files which have been forwarded to Splunk from my machine. Each file has its own data which is used to create a dashboard using the Status Indicator app. My goal is have a centralized location in Splunk such that I can see all the files that have been forwarded to Splunk (a list) and when I click on each file in the list, it should take me to the dashboard unique to that specific file.

Is there a way that this can be done in Splunk? Is there any place I can view the list of forwarded items in Splunk which can help me for this purpose?

Re: Is there a way to have a list of forwarded files in Splunk?

richgalloway — Tue, 13 Mar 2018 22:01:11 GMT

You can see all of the sources using the metadata command.

| metadata type=sources

or with this search

| tstats values(source) where host=foo

We'll need to know more about how your sources map to dashboards to help with the drilldown.

Re: Is there a way to have a list of forwarded files in Splunk?

gjanders — Tue, 13 Mar 2018 22:32:15 GMT

Meta Woot! might be useful if you want to track which sources are sending in data over time, richgalloway's answer is also great, although I'd do something more like:

| tstats max(_time) AS mostRecent groupby source, sourcetype, host
| eval mostRecent=strftime(mostRecent, "%+")

Re: Is there a way to have a list of forwarded files in Splunk?

mawomommoh — Wed, 14 Mar 2018 14:14:53 GMT

Great! These queries show me the list of my files. Thanks.

Regarding how my sources map to dashboards:
- I have a drop folder on my machine with a universal forwarder monitoring it.
- I currently have a dashboard in the Status Indicator app which makes use of the data from the latest file that has been forwarded from my drop folder. So when a new xml file is dropped into that folder, and it gets forwarded to Splunk, the dashboard updates using the latest info. This is the query I use for the dashboard in the Status Indicator app:

host=XXXX index=XXXX | eval NewTime=strptime(StartTime,"%Y-%m-%dT%H:%M:%S.%3N") | eval _time=NewTime | eventstats max(_time) AS latestScan by Description  | where _time=latestScan | stats  count by StepResult, Description| sort Description | eval  StepResult = Description

This is how my sources are being mapped to the dashboard. File > Drop folder > File forwarded to Splunk > Dashboard updates based on query
At the moment, I have included a time picker to the dashboard so I can switch between the latest dashboard being displayed to the previous dashboards i.e. previous forwarded source files (using a date/time range), but this is a bit tedious because I always need to know the date/time of the source file (dashboard) I would like to view.
With the list of source files being displayed for that specific host and index, I would like to be able to drill down to the Status Indicator dashboard of a specific source file when I click on it from the list.

Re: Is there a way to have a list of forwarded files in Splunk?

mawomommoh — Wed, 14 Mar 2018 14:18:00 GMT

This works well. Thanks!

The next thing is to carry out the drill down (you can see my reply to richgalloway's answer below). If you have any ideas pertaining to that, it would be much appreciated.

Thanks once again.