How to improve performance of stats sum

[bapun18]

Hi I want to improve my search for better search performance, please find the attachment enclosed.![alt text

[niketn · Accepted solution]

@bapun18 only possibility in the query seems like you can just delete | search from your search so that filter for field x-vf-trace-source can be applied while fetching data from index. Please try out and confirm!

If the query performs for a day but not for multiple days, you can try using daily summary indexing.
If you can have index extraction for field x-vf-trace-source you can use tstats which would work way faster.
If indexed extraction is not possible you can explore data model acceleration.

Refer to documentation: https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Aboutsummaryindexing

[to4kawa]

index=myvdf_smapi_de_db sourcetype=smapi_collector_adnroid_myvf_de x-vf-trace-source="android:com.appseleration.android.selfcare"
|eval bytes=len(_raw)
|timechart span=1d sum(bytes) as Total_bytes

Hi, You searched twice because of search in the second line. Let's remove this.

 |tstats sum(bytes) where index=myvdf_smapi_de_db sourcetype=smapi_collector_adnroid_myvf_de x-vf-trace-source="android:com.appseleration.android.selfcare"

Creating a data model so that this search can be used is one of the solutions.

Design data models

[arjunpkishore5]

Looking at the Events tab, it looks like you are in "Verbose Mode". Change your search from "Verbose Mode" to "Fast Mode" (on the bottom right of your search panel). That should speed things up

Please mark as answer if this helps

Cheers

[adonio]

seems like you are trying to calculate the size of each event and then sum them up to get total usage per day ...
why not look at the _internal index for license usage of this particular source in this particular index?

there are tons of answers around this forum regarding license usage monitoring
also, no need to do the | search its redundant, just add all your filters

[arjunpkishore5]

He has a filter for a specific type of event. I would guess that's the reason he needs to calculate the size of each event. 🙂

[adonio]

if he needs the size of each events, he wouldnt sum it up by time (1d)

[arjunpkishore5]

Apologies for the confusion. Let me try to rephrase. It looks like they want to calculate the amount of data generated by a specific type of event by day. As far as I'm aware, License usage provides metrics at metadata level and does not provide metrics for a subset of the data within the index.

[richgalloway]

It would have been helpful to copy-and-paste the query into your question so we can test it ourselves.
What about this query needs improving. The screen shot does not show any performance information so we don't know what needs to change.
Have you looked at the Job Inspector?