topic How can I use a source folder as a input token? in Splunk Search

How can I use a source folder as a input token?

external_alien_ — Thu, 08 Sep 2016 11:51:20 GMT

Hi guys!
I have a bunch of test data in JSON files as my sources and they're structured in the following way:
"/MyFolder/ProjectName/RunID/jsonFile" such as for example "/MyFolder/test1/47/ErrorMessages.json".
I want to populate two drop down menus in my dashboards with ProjectName and RunID.
So that the first drop down get populated with Project names that the user can select. I then use this selection as a token in the second drop down menu and populate it with all the runs for that project. I now have two tokens that I can use for searching.
How can I best go about this? If there's a search I can do directly in Splunk to table all the project names and RunIDs this would of course be the easiest and most ideal. Otherwise I guess I'd have to start looking into creating a regEx or a lookup, but I'm not very proficient at those =p
Any help is greatly appreciated!
Thanks you!

Re: How can I use a source folder as a input token?

somesoni2 — Thu, 08 Sep 2016 18:23:26 GMT

Since the ProjectName and RunID as part of the source/file path is something custom to your requirement, there will not be any readily available table to get you that.

Once this you can try is to run a metadata/tstats search to get list of all sources (for your index/sourcetype of course) and use field extraction to get those values listed/made available for dashboard dropdowns. Following search can give you list of ProjectName adn RunID from the source:-

| tstats count WHRE index=PutYourIndex sourcetype=PutYourSourceType by source | rex field=source "^\/[^\/]+\/(?<ProjectName>[^\/]+)\/(?<RunID>[^\/]+)" | stats count by ProjectName RunID | table ProjectName RunID

Now you can either use the search directly for the dropdown OR setup a scheduled search to write this data into a lookup file and then use the lookup file for the dropdown.

Re: How can I use a source folder as a input token?

astalv — Fri, 09 Sep 2016 07:31:07 GMT

do you want something like this?
| rex field=source "\/(?P[^\/]+)\/(?P[^\/]+)\/(?P[^\/]+)"

Re: How can I use a source folder as a input token?

external_alien_ — Fri, 09 Sep 2016 13:37:13 GMT

Beautiful, worked perfectly! Exactly what I needed!

Thank you! 😃