Monitoring Splunk

How to calculate the memory Splunk consumes while running a query?

pramit46
Contributor

guys,

How can I find out how much memory does a Splunk Query consume?

0 Karma
1 Solution

rsennett_splunk
Splunk Employee
Splunk Employee

A quick and dirty way to do this, would be to steal the "Top Memory Consuming Searches" from the SOS app.
(This is a handy app that you should have installed anyway. http://apps.splunk.com/app/748/ Everyone should use Splunk on Splunk (SOS)

Under the Resource Usage Menu, choose "CPU/Memory". The last panel on the bottom of the dashboard is the one you want.

Hover your mouse over the lower left hand corner of the panel and you'll see a tiny magnifying glass... click it.
It will open the search in a new window.
This search has a lot of stuff in there that you don't need if you are pinpointing one particular search... but rather than pulling it apart... you can insert the SID from the search in question.

Run it in another tab, click "Job Inspector" and get the SID (it's right at the top). Find the following line in the search:

| search sid=* sid!="subsearch*" search!=typeahead* search!="|history*"]

Be careful, because this is the end of an append

Right after | search sid=*

insert your SID so it looks like this:

| search sid=* sid="YOURSIDHERE" sid!="subsearch*" search!=typeahead* search!="|history*"]


That will create a lovely report showing you just the stats for that one particular search.



Again... rather than breaking it... I just added the "Search FOR this thing" with all the "Search for NOT this stuff"

Of course... next thing to do is pick it apart and learn what it's doing. 🙂

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

View solution in original post

rsennett_splunk
Splunk Employee
Splunk Employee

A quick and dirty way to do this, would be to steal the "Top Memory Consuming Searches" from the SOS app.
(This is a handy app that you should have installed anyway. http://apps.splunk.com/app/748/ Everyone should use Splunk on Splunk (SOS)

Under the Resource Usage Menu, choose "CPU/Memory". The last panel on the bottom of the dashboard is the one you want.

Hover your mouse over the lower left hand corner of the panel and you'll see a tiny magnifying glass... click it.
It will open the search in a new window.
This search has a lot of stuff in there that you don't need if you are pinpointing one particular search... but rather than pulling it apart... you can insert the SID from the search in question.

Run it in another tab, click "Job Inspector" and get the SID (it's right at the top). Find the following line in the search:

| search sid=* sid!="subsearch*" search!=typeahead* search!="|history*"]

Be careful, because this is the end of an append

Right after | search sid=*

insert your SID so it looks like this:

| search sid=* sid="YOURSIDHERE" sid!="subsearch*" search!=typeahead* search!="|history*"]


That will create a lovely report showing you just the stats for that one particular search.



Again... rather than breaking it... I just added the "Search FOR this thing" with all the "Search for NOT this stuff"

Of course... next thing to do is pick it apart and learn what it's doing. 🙂

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!

rsennett_splunk
Splunk Employee
Splunk Employee

Oh! I didn't see that! 🙂

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma

pramit46
Contributor

@rsennett_splunk, I also found that in the job inspector page it shows the memory space the query had consumed.

Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...