Splunk Search

Fastest way to count records per day

JensT
Communicator

Hello,

hopefully this has not been asked 1000 times.

I'd like to count the number of records per day per hour over a month. We have ~ 100.000.000 - 150.000.000 records per day. So a simple "* | timechart span=1h count" takes veeery long.

Was the fastest way to do this?

Regards,

Jens

Tags (1)

the_wolverine
Champion
| tstats count where index=* groupby index,_time span=1d

jmeyers_splunk
Splunk Employee
Splunk Employee

MuS
Legend

+1 for tstats which will be faster then metadata

0 Karma

jrodman
Splunk Employee
Splunk Employee

Using the advanced charting view may speed it up a bit.

Just getting that many buckets (timeslots) may end up being problematic.

If you want to solve this problem for the future, summary indexing might be the right approach.

If you want a very very rough idea, you could use the commands which probe index state such as are used to back the searches for the indexing dashboard. These tell you things about where your buckets exist in time, and then some fancy calculations could give you an upper bound on the number of events over that month.

If you want just a total, you could try date_month=1 date_year=2011 | stats count by date_day

Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...