Optimising the Populating of Data in my Dashboard Panel for Daily size Count of Events Indexed for 7 days.

[bapun18]

Hi,
I have prepared a Weekly Monitoring dashboard for Size count of Events Ingested Daily for a specific index and sourcetypes But as it populates it takes alot of time to load as the Count of eevents is pretty huge. I can't use tstats as i used eval exists in the query for summation purpose .Suggest me any solutions to Quick run the dashboard panel populated data .
Below is the Query which i have implemented :

index=myvdf_smapi_de_db sourcetype=smapi_collector_android_myvf_de
| eval bytes=len(_raw)
| bucket span=1d _time
| stats sum(bytes) by _time

Time stamp i need is weekly.

[gcusello · Accepted solution]

Hi bapun18,
did you explored the summary index approach?
In few words, you have to schedule (e.g. every night at 1.00 AM) a search putting results in a summary index and then you can use this index for your searches with very much performant time.
So you could schedule at 01.00 AM a search like the following:

index=myvdf_smapi_de_db sourcetype=smapi_collector_android_myvf_de earliest=-d@d latest=@d
| eval bytes=len(_raw)
| bucket span=1d _time
| stats sum(bytes) as bytes by _time
| eval Mbytes=bytes/1024
| collect index=my_summary

Then run a search, like the following:

index=my_summary earliest=-7d@d latest=@d
| table _time Mbytes

Obviously if there are additional information useful for you, you can add tho the scheduled search.

Ciao.
Giuseppe

[bapun18]

Hi Arjun ,
Thanks for the Response but i was wondering apart from creating a summary index for a single panel data of the the whole dashboard isn't their any method to fine tune the query and load the dashboard panel data quickly like Some Splunk commands similar to Tstats which considers certain evaluation as well as optimize the result time.

[bapun18]

Hi gcusello ,
Sorry wrongly posted your name.

[arjunpkishore5]

This data is already available in the _internal index. For weekly, set your span to 1w

index=_internal source=*license_usage.log idx=myvdf_smapi_de_db st=smapi_collector_android_myvf_de 
| bucket span=1w _time 
| stats sum(b) by _time

Hope this helps.

Cheers

[bapun18]

Hi Arjun ,
Nothing is getting populated if i run the above mentioned query.