If I run the All Sourcetypes dashboard, the MB received panel for the past 24 hours, the panel takes just over nine minutes to complete. I studied the search and it is made up of three macros that form this search:
index="_internal" source="*license_usage.lo*" type!=*Summary | eval lastReceived = _time | rename s as source st as mysourcetype h as host b as bytes o as originator | eval my_splunk_server = splunk_server | fields lastReceived source mysourcetype host bytes pool originator my_splunk_server source | bin _time span=10m | stats sum(bytes) as bytes max(lastReceived) as lastReceived by mysourcetype _time pool host | eval kb = bytes/1024 | eval mb = kb/1024 |timechart minspan=10m bins=200 sum(mb) as mbytes by mysourcetype
If I run this search manually the results are returned within 1 minute.
Any idea about what is going on????
The macro powering the search in the "MB Recevied" Panel is:
sourcetype_metrics_timechart
You can see what is consuming all the time in your search by inspecting the job. Here is what you'll want to do:
Reload the "All Sourcetypes" panel and go on a small coffee break ( not too long of a break or the job details will get cleaned up).
Click on the "Jobs" link in the upper right corner
Click "Inspect" on the entry for "sourcetype_metrics_timechart
"
It sounds like the search might not be using Report Acceleration correctly. Do you see a message indicating that sumaries are being used?
It would look something like the following:
DEBUG: [my.host.name] Using summaries for search,
If Search summaries are being used, then you may have run into a bug in the core product with search acceleration. In that case, I would recommend opening a support case.
Not enough information. Can you open a support case please?