So now that I've been making good progress of getting data into Splunk 4.2, I'm running into a different issue, managing knowledge that's contained there.
Many of the people that will be using Splunk may not know what the log source will actually be, or even what host will have the data. Searching for strings that will be contained in the log may match many different applications.
So I can create tags for them: Tag=TEST_WEB, or Tag=your_app. But how would the users know that they exist?
But what is everyone else doing to manage this? Separate webpage containing a table of what is configured? I see this is my biggest problem in the near future, how to provide this information back to them on how to easily find the data they're looking for.
One thing we've started on (but haven't finished) is form searches based on lookups. We run a scheduled search to incrementally populate some lookups with well-known apps. Our next goal is to use those lookups to make form searches where they can have a degree of drop-down selection to hint them into host=xyz, sourcetype=pqr.