Splunk Search

Fastest way to count records per day

Communicator

Hello,

hopefully this has not been asked 1000 times.

I'd like to count the number of records per day per hour over a month. We have ~ 100.000.000 - 150.000.000 records per day. So a simple "* | timechart span=1h count" takes veeery long.

Was the fastest way to do this?

Regards,

Jens

Tags (1)

Champion
| tstats count where index=* groupby index,_time span=1d

Splunk Employee
Splunk Employee

SplunkTrust
SplunkTrust

+1 for tstats which will be faster then metadata

0 Karma

Splunk Employee
Splunk Employee

Using the advanced charting view may speed it up a bit.

Just getting that many buckets (timeslots) may end up being problematic.

If you want to solve this problem for the future, summary indexing might be the right approach.

If you want a very very rough idea, you could use the commands which probe index state such as are used to back the searches for the indexing dashboard. These tell you things about where your buckets exist in time, and then some fancy calculations could give you an upper bound on the number of events over that month.

If you want just a total, you could try date_month=1 date_year=2011 | stats count by date_day

State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!