How do YOU use splunk! (Search/Query Examples)

[ktrumpol]

Hello everyone,

Our company just started using Splunk, and after experimenting with some basic commands it certainly proves to be a powerful yet simple to use search processor. Since our team is so new to this experience, we were curious how everyone else was utilizing Splunk for their servers! Any general, abstract overview of what you use it for is appreciated. Also what would be very helpful are some techniques, and example search and query custom codes that you use to perform such actions in your company.

Any response is much appreciated!

[hagjos43_2]

There are two sites that exist that I know of that have Splunk Query examples:
http://www.bbosearch.com. Though it is usually full of spam, but still have useful (and rather advanced) splunk queries.
https://gosplunk.com. It's been around for a few years and has quite a few queries categorized by sourcetype.

[lukejadamec]

I use Splunk to answer questions about what went or is going on, and to solve problems presented by management regarding the review and analysis of system logs.

The questions vary as much as the deployed environment, and are often trigged by review of Splunk reports and or alerts: Things like – “Look at that. What the h3!! Was going on there?” These clues to problems can range from single lower case items to all caps sentences.

In the “What the h3!!” cases, I usually start by reviewing information about the system in question, the search that generated the report, and then create some crafty search to elucidate the situation, or I just click on the result in the report to drill down. I try not to make it look too easy.

When management decides there are problems that Splunk can solve, I create searches that address the problems, and then combine those searches into reports and or alerts. Management problems are generally pretty vague, so the searches specifically address what management is really looking for from the data – it does take a bit of vision to bridge the gap. Of course, management does like reports and alerts.

[chris]

What does your company do? There are lots of different use cases. Have a look at the apps section of splunk base to get some ideas. Connecting to databases, becoming PCI Compliant, monitoring VMware or Exchange there is a lot .... a lot of apps are free and you can learn from the saved searches in the apps. This might also be interesting http://demos.splunk.com/

[ssankeneni]

http://www.splunk.com/web_assets/v5/book/Exploring_Splunk.pdf is a book from splunk and will help you to learn more about search commands.