Splunk Search

How to document a Splunk query

sonam
Explorer

I am writing a Splunk query to search logs generated by a middleware system for anything 'exceptional'. Basically, the approach I'm taking is filtering out entries for 'known issues'. Anything left behind is then an 'unknown issue' by definition (and worthy of attention). The Splunk query examines the previous day's logs each night and emails results for review in the morning.

The query looks like this... as you can see, it just a large set of 'NOT' terms:

index=middleware
NOT SalesForce* 
NOT SSL_DEBUG
NOT "Cache cleared for service *" 
NOT "Service Thread Pool" 
...
(20 more exclusions and growing)
...

My questions :

  1. How can I document this query?
    Specifically, I'd like to explain each 'NOT' exclusion above.

  2. Is this a reasonable approach for reviewing logfiles?
    My Splunk admin is concerned about the performance impact of 'NOT' terms.

jbsplunk
Splunk Employee
Splunk Employee

You could add it into savedsearches.conf, and then just add the comments in above that, specifying they are comments by beginning each comment line with a # symbol. In terms of a query expense, your returning the entire set of results, and then filtering based on that. It is better if you can specify a time frame, and the results that you'd like to see being as specific as possible prior to filtering out events.

0 Karma

sonam
Explorer

My impression is there is no functionality available to a Splunk end-user, to document Splunk artifacts in Splunk.

The only option seems to be to copy/paste the saved searches/events, etc... from Splunk into a Wiki or Word files or whatever knowledgebase you use, and document it there.

0 Karma

sonam
Explorer

Hmm. Thanks for that. You mentioned a configuration file (savedsearches.conf). However, I'm just a poor user, not a Splunk sysadmin so I don't have access to this file. (Am I wrong?) I can only save searches and event.

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...