Monitoring Splunk

deployment monitor MB received SLOOOOOOWWWW

mcbradford
Contributor

If I run the All Sourcetypes dashboard, the MB received panel for the past 24 hours, the panel takes just over nine minutes to complete. I studied the search and it is made up of three macros that form this search:

index="_internal" source="*license_usage.lo*" type!=*Summary | eval lastReceived = _time | rename s as source st as mysourcetype h as host b as bytes o as originator | eval my_splunk_server = splunk_server | fields lastReceived source mysourcetype host bytes pool originator my_splunk_server source | bin _time span=10m | stats sum(bytes) as bytes max(lastReceived) as lastReceived by mysourcetype _time pool host | eval kb = bytes/1024 | eval mb = kb/1024 |timechart minspan=10m bins=200 sum(mb) as mbytes by mysourcetype

If I run this search manually the results are returned within 1 minute.

Any idea about what is going on????

0 Karma

mkinsley_splunk
Splunk Employee
Splunk Employee

The macro powering the search in the "MB Recevied" Panel is:

sourcetype_metrics_timechart

You can see what is consuming all the time in your search by inspecting the job. Here is what you'll want to do:

  1. Reload the "All Sourcetypes" panel and go on a small coffee break ( not too long of a break or the job details will get cleaned up).

  2. Click on the "Jobs" link in the upper right corner

  3. Click "Inspect" on the entry for "sourcetype_metrics_timechart"

It sounds like the search might not be using Report Acceleration correctly. Do you see a message indicating that sumaries are being used?

It would look something like the following:

DEBUG: [my.host.name] Using summaries for search, 

If Search summaries are being used, then you may have run into a bug in the core product with search acceleration. In that case, I would recommend opening a support case.

araitz
Splunk Employee
Splunk Employee

Not enough information. Can you open a support case please?

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...