Splunk Search

How to get a daily count of distinct users over a large time range without re-running from beginning of time

e_sherlock
Explorer

I have a simple "| stats dc()" command to get a cumulative sum of distinct users; however, I don't want to have to run this query from the beginning of time every day it runs for performance reasons.

These couple routes came to mind, but don't seem best...

  1. Summary indexes can write out the distinct users per day, but seems like couldn't compute distinct over the whole set of days.
  2. Write the distinct values to lookup and only adding the new values each day -- a kinda hacky incremental approach.

Any ideas on the optimal solution?

Tags (2)
0 Karma
1 Solution

gkanapathy
Splunk Employee
Splunk Employee

The best way would be to use sistats and a summary, or alternatively, use report acceleration. With report acceleration, just set up your search and tell Splunk to accelerate, and that should do it.

If you summarize yourself, then on a daily basis (and you can backfill later), you run a;

... | sistats dc(user) by x,y,z 

and store that to a summary. Then to get your counts:

index=my_summary_index name=my_summary_job | stats dc(user)

(or ... | stats dc(user) by x,y,z or ... | stats dc(user) by x,y).

the sistats command will have saved the right data, and the stats command will know how to handle what sistats did. Yes, it is doing slightly clever things under the hood.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

The best way would be to use sistats and a summary, or alternatively, use report acceleration. With report acceleration, just set up your search and tell Splunk to accelerate, and that should do it.

If you summarize yourself, then on a daily basis (and you can backfill later), you run a;

... | sistats dc(user) by x,y,z 

and store that to a summary. Then to get your counts:

index=my_summary_index name=my_summary_job | stats dc(user)

(or ... | stats dc(user) by x,y,z or ... | stats dc(user) by x,y).

the sistats command will have saved the right data, and the stats command will know how to handle what sistats did. Yes, it is doing slightly clever things under the hood.

Get Updates on the Splunk Community!

This dashboard view is deprecated and will be removed in future versions of Splunk ...

After upgrading to Splunk Enterprise 9.0 I do get the following message from several Dashboard.This dashboard ...

How to ingest a selection of JSON fields

I have a dump.json file that collects events in JSON format:<BR ...

Why getting timeout error while adding data to the Splunk cloud index from REST API?

Hello Team,<BR /><BR />I am getting timeout error while adding data to Splunk cloud index from REST API. I am ...