Splunk Search

Is the information in the search job inspector stored in Splunk internally so I can query this data and set up dashboards?

feickertmd
Communicator

I have been tasked to find a way to report on the overall query load to our Splunk system by customers that we have using it. The information I need shows up in the job inspector. Is that information stored in the _internal index anywhere, or is there another data source which I can query to set up dashboards for this purpose?

Tags (2)

emiller42
Motivator

What information are you looking for out of the Inspector? That information is actually coming from the search artifact in the dispatch directory, and goes away when the search expires.

However, you also get metrics in the _introspection index. (index=_introspection sourcetype=splunk_resource_usage component=PerProcess) and there are built-in dashboards that might be helpful. (Activity > System Activity, or Settings > Distributed Management Console)

The Splunk on Splunk app also gives insight into search metrics, although there is overlap between it and the built-in dashboards mentioned above.

0 Karma

woodcock
Esteemed Legend

It might be here:

| rest /services/search/jobs

sfmandmdev
Path Finder

Is this data logged? Is there a config param that can be enabled to write this info to logs?
Would be nice to analyze when looking at long running / expensive searches.

0 Karma

woodcock
Esteemed Legend

This data is not logged in this level of detail anywhere but you can save it yourself in a csv/lookup or in a summary index.

landen99
Motivator

Anything can be sent to a summary index with a scheduled search.

0 Karma

sfmandmdev
Path Finder

Is there an answer for this?

0 Karma

Anonymous
Not applicable

Hi

 

I know it`s a bit late for this answer. But in my defence I was too looking for the same thing.
And ended up on this post.

The inspector does not go trough the search pipeline because then it would impact the result of the search. 

The Inspector does a lot of checks and balance and it could interfere the search outcome if it was too in the search pipeline. 

Maybe in the future there would be a export to csv on the website. 

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...