Can I add 15M entries into a summary index on a daily basis?

[mvanderlist_spl]

Running into challenges with monthly reporting, and need to figure out how to use the right Splunk tool for the job.

Need to report on fieldA and distinct the devices. Raw events contain deviceId="xys" fieldA="something"

A monthly report against the raw data with ... | stats dc(deviceId) by fieldA takes 9 hours to run - it works, but resource consuming.

With the search ... | stats count by deviceId, fieldA over 1 day producing 15M events per day.

Can I add 15M entries into a summary index on a daily basis? And then run a search index=summary | stats dc(deviceId),fieldA ?

[woodcock]

Summary Index is probably your only practical option.

Schedule this search to run every hour (or day or whatever your minimum granularity is):

... | sistats dc(deviceId) by fieldA 

When you save it, click "Summary Index" and give it an SI in which to put the data, then run the backfill command to go backwards in time and summarize your data. Then use this to get the data back out:

index=MySummaryIndex| stats dc(deviceId) by fieldA